In search of an evaluation approach to social innovation

18.09.2012 Blog

In recent years, the social sector has neared consensus on the need to improve measurement and evaluation of its work and impacts.  Although several different methods are currently in use, it is widely accepted that more appropriate approaches and practices to monitor performance, track progress, and assess the impact of organisational strategies, initiatives, and grants should be developed. When it comes to approaches to social problems, traditional evaluation methods have proved to be insufficient. The challenge for evaluators is that not all solutions to social problems follow logical steps; they are often complex, involving parameters which are difficult to understand. This is the place where social innovators usually find themselves.

Social innovators are experimenting with new ways of tackling intractable social challenges that existing structures, policies and institutions have failed to address. Compared to traditional interventions, social innovations are new solutions (products, services, models, markets, processes etc.) that involve actors working together from different sectors. Traditional evaluation approaches like formative evaluation and summative evaluation are considered to be poor for assessing social innovation projects which combine complex variables, relationships and uncertain results. Such evaluations are not structured in a way that provides timely and appropriate information and data to respective decision makers. When a formative or summative evaluation approach is applied to an innovation that is still evolving, it can stifle the adaptation and creativity which is fundamental for its success. In this respect, Developmental Evaluation (DE) could be seen as the most appropriate approach for evaluating social innovations.

According to Michael Quinn Patton, “Developmental evaluation supports innovation development to guide adaptation to emergent and dynamic realities in complex environments. … Informed by systems thinking and sensitive to complex nonlinear dynamics, developmental evaluation supports social innovation and adaptive management. Evaluation processes include asking evaluative questions, applying evaluation logic, and gathering real-time data to inform ongoing decision making and adaptations.”

His main argument is that social innovation initiatives are often in a state of continuous development. Although still at an embryonic stage, DE is suggested as an alternative approach to address the real challenges of evaluating innovative social change projects. It can be applied to an ongoing process of innovation in which both the path and the destination are evolving. Although ex-post evaluations have an important role, DE can be effective in cases where an innovation is still unfolding. The main objective of DE is not only to assess the results of a project but also to contribute to its success and development with the provision of continuous feedback from key stakeholders throughout the entirety of the innovation process.

Given that there is no accepted method specifically designed to evaluate social innovations and based on the fact that a single approach is probably impossible to develop, Patton’s suggestion on developmental evaluation represents a useful approach that should be taken into account when it comes to the complex environments where social innovations appear. But, since DE is not appropriate to all types of initiative, a model should be developed to indicate the most appropriate evaluation method taking into account the objectives, results and different conditions under which a project is implemented. The wider application of DE, the specific circumstances under which it can be used as well as its combination with traditional evaluation approaches should be a matter of further discussion. These are some of the issues that will be further examined as part of the TEPSIE project.

References
Preskill, H., Beer, T. (2012) Evaluating Social Innovation, Washington DC: FSG & Center for Evaluation Innovation.

Patton, M.Q. (2011) Developmental evaluation: Applying complexity concepts to enhance innovation and use. New York: Guilford Press.

Gamble, J. A. (2008) Developmental Evaluation Primer, J.W. McConnell Family Foundation.