Documents/TM4RG/3: EVALUATION/III.A: CONCEPTUAL FRAMEWORK

III.A: CONCEPTUAL FRAMEWORK

Other Information:

In a recent paper, Fukuyama described the poor state of assessing governance worldwide, which he explains as resulting from the lack of any conceptual framework: [W]e cannot measure what we cannot adequately conceptualize, we have to start with the concept first." A related challenge in identifying a logic model or conceptual framework for governance innovation is linked with the diversity of goals underlying citizen engagement and data-sharing in governance. Goals may be as broad as enhancing democracy or as narrow as helping a specific agency operate more efficiently. Abelson and Gauvan, for instance, have this to say about goals of public participation and democracy generally: "Democratic theory tells us that public participation is undertaken for different purposes and with different underlying goals. Tensions exist between views of participation as an essential element of successful democracy (and inherently desirable in its own right) and participation as a means for achieving something else, be it a specific decision outcome, a desire for more informed, accountable or legitimate decision making, or perhaps to delay or share the blame for a difficult decision. Lying somewhere between is the desire for public participation to contribute to a more educated and engaged citizenry." They go on to discuss how different parties to an intervention may have differing goals: "[D]ifferent evaluation perspectives ...may exist among interested parties. For example, sponsors and taxpayers tend to be interested in value for money. But sponsors and organizers of public participation should also be interested in questions of efficacy and effectiveness (if the purpose is summative evaluation) and whether the public participation method implemented was successful as measured against its goals (to address a formative evaluation purpose). Participants themselves are increasingly interested in whether their involvement makes a difference (i.e., policy impact) and, as taxpayers, they also want to see that their involvement was meaningful given that investments in public participation are typically made at the expense of direct service and program delivery. These differing perspectives are integrally linked to the different underlying goals for public participation." Chess has identified three different approaches that are routinely taken to assess public participation: "1) user-based evaluation, which assumes that different participants will have different goals and that the evaluation must take these different goals into account; 2) theory-based evaluation which is driven by theories and models of public participation and applies normative criteria universally to any public participation effort; and 3) goal-free evaluation which is not constrained by any stated goals and is conducted in the absence of any theory."

Indicator(s):