Diagnosis
Program evaluations are often unsatisfactory, not because they fail to determine whether an outcome has been achieved or an impact observed, but rather because they tell us very little about why a particular outcome or impact occurred. At the end of a program we may be able to determine that it has been successful; however, if we do not know the reasons for this success (if we have not fully appreciated the complex dynamics operating within and upon this program) then we have little information that is of practical value. We have very few ideas about how to sustain or improve the program, or about how to implement a successful program somewhere else. All we can do is to continue doing what we already have done. This choice is fraught with problems, for conditions can change rapidly. Programs that were once successful may no longer be so.
Over the years, Michael Quinn Patton has been among the most influential evaluators in his emphasis on the pragmatic value inherent in a diagnostic focus. Coining the phrase “utilization-focused evaluation,” Patton suggests that:
Unless one knows that a program is operating according to design, there may be little reason to expect it to produce the desired outcomes. . . . When outcomes are evaluated without knowledge of implementation, the results seldom provide a direction for action because the decision maker lacks information about what produced the observed outcomes (or lack of outcomes). Pure pre-post outcomes evaluation is the “black box” approach to evaluation.
A desire to know the causes of program success or failure may be of minimal importance if an evaluation is being performed only to determine success or failure or if there are no plans to continue or replicate the program in other settings. However, if the evaluation is to be conducted while the program is in progress, or if there are plans for repeating the program somewhere else, evaluation should include appreciative procedures for diagnosing the causes of success and failure.
Download Article 1K Club
