III.B: METRICS & INDICATORS

Develop a set of common standards for assessing citizen engagement initiatives.

Other Information:

Selecting indicators and metrics of success for citizen engagement in governance is a value-laden process with abundant opportunity for misinterpretation. It is not clear whether a universal framework for success can be achieved, but some have suggested that there is a need to develop a set of common standards for assessing citizen engagement initiatives. Each initiative may be so different that the desired outcomes will vary widely depending on the program. However, it may be possible to develop thematic buckets within each subset of the participatory government space, defining a set of metrics for each area. Some researchers assessing open government have suggested that rigid distinctions between quantitative and qualitative assessments are no longer helpful. The SUNY Albany Center for Technology in Government suggests that measures of the return on investment for government interventions should be judged only on whether they are "valid and useful in the relevant context of measurement." The Open Government initiative of the Obama administration provides a good example of the evolution in measurement and metrics needed in this field. To assess the success of this effort, the Openthegovernment.org organization developed agency evaluations using a scale of 1 to 3 to judge adherence to the directive. But that exercise is only able to determine whether agencies made their data available to the public. Missing from the directive itself, and from initial evaluation efforts, was any measurement of progress on the overall goal of the directive: to engage the public in governance. As these initiatives move forward, additional metrics are being developed to measure more substantive issues of participation. As the report on this study explained, " ...directional policy without benchmarks or specific increments of improvement is akin to diagnosing fever without a thermometer." Several other organizations have been carrying out evaluations/assessments of open participatory government interventions, some of which already have case studies that give an indication of the effectiveness of various methods of evaluation. They include the following examples:

Stakeholder(s):

• Coalition for Evidence-based Practice: The Coalition for Evidence-based Practice issued the report, "Which Study Designs Can Produce Rigorous Evidence of Program Effectiveness?" with an appendix that includes a number of case studies of interventions in government where RCTs have been effective.


• Institute of Development Studies: The Institute of Development Studies examined various evaluations in the open/participatory government arena. One of the reports looks at assessments of transparency and accountability initiatives in the fields of Service Delivery, Budget Processes, Freedom of Information, Natural Resource Governance, and Aid Transparency.


• Center for Technology in Government: The Center for Technology in Government at SUNY Albany has conducted a number of studies in related fields (see e.g., The Dynamics of Opening Government Data and Developing Public Value Metrics for Returns to Government ICT Investments). Taewoo Nam, also at SUNY Albany, reviewed a number of studies of evaluation specifically in the area of citizen sourcing using Web 2.0 technologies and discusses frameworks that researchers can use to evaluate the impacts of citizen sourcing.


• OpenDataResearch.org: Exploring the Emerging Impacts of Open Data in Developing Countries (ODDC) is a recent project aimed at investigating ways in which open data improve governance, support citizens' rights and promote more inclusive development in developing countries. They are attempting to share information on evaluation and assessment of open data initiatives. Their website presents a number of case studies evaluating the impacts of open data.


• World Bank: The World Bank, through its Striking Poverty program, is engaged in a debate on evaluating citizen involvement not only in transparency and accountability, but also in interventions that have an impact on people's lives.


• Nesta: Nesta's initative "Making Evidence Useful" includes a network of centres in the U.K. that will develop and evaluate assessment methods to identify what works.


• Project on Government Oversight: The Project on Government Oversight highlights best practices in open government in the U.S., including a list of resources on open government initiatives.

Indicator(s):

Indicator:1

Content