II: METRICS & METHODS

Learn from evaluation.

Other Information:

LEARNING FROM EVALUATION: METRICS AND METHODS -- Before we try to change how we solve problems and plan new initiatives we need to know what has worked already and what hasn't. Over the past few decades there have been renewed calls to measure the impact of societal initiatives in order to allocate resources wisely and understand what to continue, scale up or cancel. With government and philanthropic funding shrinking, both government agencies and charities see outcome-based evaluation as an urgent need. There has also been an increased demand to understand with more precision and nuance whether programs are successful, how they are successful and what factors have led to that success. Even when an initiative demonstrates positive outcomes, it is still helpful to understand: * Whether there are different impacts for some populations and objectives than for others. * How much better off people are as a result of the initiative—only slightly, or significantly? * Whether a program works differently in different geographical regions. * Why a program has been effective and what factors have contributed to its success. It can also be helpful to understand what doesn't work, despite the incentive to report only successful elements of a program in order to maintain funding. Engineers Without Borders Canada, for instance, produces an annual Failure Report which shares learnings about less effective innovations in a step to "create a culture of creativity and calculated risk taking." With a clear understanding of both successes and failures, more effective and more innovative initiatives can be designed and better opportunities for intervention can be identified. Evaluation is a complex and challenging task that requires judgment to choose how, what and when to measure along with the criteria for an intervention to be deemed successful. Our review of the existing literature on evaluation has highlighted several issues, challenges and approaches.

Stakeholder(s):

• Engineers Without Borders Canada

Objective(s):

II.A: CONCEPTUAL FRAMEWORK

II.B: METRICS & INDICATORS

II.C: METHODOLOGY

II.C.i: Experimental Design

II.C.ii: Quasi-Experimental Design

II.C.iii: Non-Experimental Techniques

II.C.iv: Performance Measurement

II.C.v: Process Evaluation

II.D: NEW TRENDS & TOOLS

Content