Measuring and mapping social innovation

15.04.2013 Blog

On Friday April 12, Social Innovation Exchange (SIX) and TEPSIE hosted a TelePresence discussion exploring the potential for mapping and measuring social innovation. We heard presentations from two ends of the measurement spectrum: Georg Mildenberger and Eva Bund from the University of Heidelberg presented on measurement on the macro scale; Filipe Santos from INSEAD and Nuno Frazão from the Social Entrepreneurship Institute introduced their methodology for mapping specific social innovations in Portugal.

Georg and Eva kicked off the discussion by taking us through their current thinking on the measurement of social innovation as part of their work for the TEPSIE project. Georg emphasised that to be clear and useful, any kind of measurement has to be clear about the object that it is measuring. Their work therefore has strong links to the TEPSIE  definition of social innovation which includes the following elements: social innovations must meet a social need, have some novel element (or at least be new to their context), must be implemented (they are more than just ideas) and they must work (they are more effective than existing solutions). They must also lead to new or improved capabilities, assets and/or relationships, so that they enhance the capacity of society to act in the future.

The team in Heidelberg has done extensive work reviewing existing indicators that might have relevance for social innovation; their approach has been to build on existing indicators rather than to propose creating an entirely new ones. Two types of existing indicator are particularly relevant – those that measure innovation more broadly in the economy (for example the Innovation Union Scoreboard) and those that are less focused on innovation and more on measuring social outcomes (for example the OECD Better Life Index). Reflecting on some of these existing indicators, they propose a framework for measuring social innovation using existing instruments, with the following three parts:

  • Enabling framework conditions (for example the political climate, investment framework)
  • Core entrepreneurial activities (creation of ideas, mobilisation of resources)
  • Field specific outcomes and outputs (measurements of improvements in health, education etc.).

This framework will be developed into a blueprint for a social innovation indicator suite – which will be published later this year.

Filipe and Nuno then talked us through the methodology of their project which aims to map social innovation and social entrepreneurship in Portugal. Their overall goal is to identify 200-300 high potential social entrepreneurship projects from around 2000 that they are currently tracking. The unit of analysis of their research is new solutions themselves (as opposed to a focus on organisations or individual entrepreneurs). They have employed a snowballing technique whereby they begin with interviews with what they call ‘privileged observers’ in a region (this could be a business person with a great network, the head of a charity or a church leader). They ask these individuals: what are the most interesting initiatives you are seeing in your region? At stage two, they make a screening call to all the people identified through the snowballing technique to better understand their project or initiative. Stage three involves an in-depth survey to track the initiative – where it came from and how it developed – which has resulted in a rich bank of case studies. Then finally, a panel comes together to decide whether they think a particular project should be identified as a ‘high potential initiative’. The research team are agnostic about legal structure, but projects rated as ‘high potential’ should meet the following criteria:

  • They have a clear social or environmental mission
  • They have impact on their chosen issue
  • They involve and empower local stakeholders
  • They are innovative – they differ from existing solutions
  • They are replicable in other contexts or could be scaled up

Projects do not need to be strong in all criteria but they should be strong in at least three of these elements to be included in the ‘high potential’ category. Now that this methodology has been trialled in Portugal, there are plans to use an adapted version in Mozambique.
Following each of the presentations there was a wide ranging and rich discussion of both projects and wider issues around measurement. Some of the themes that stood out from the discussion were:

Do we know what we are measuring?
Discussions of measurement inevitably bring up issues of definition. If what we are trying to measure is rather fuzzy, then this makes identifying relevant indicators all the more complex. There was some discussion about whether our understanding of ‘social needs’ as a core part of the definition of social innovation is sufficient. Filipe commented that there needs to be more discussion around what constitutes a social need. He suggested it might be better to focus on the idea of ‘neglected’ or ‘unaddressed’ needs rather than trying to make an (often false) distinction between social and economic or other types of need. Related to the issue of definition, Geoff Mulgan noted that this issue is not unique to social innovation as a field. It is equally challenging to define what we mean by innovation in the business sphere, and this creates similar problems for measurement tools.

Who needs measurement?
Who are the different audiences who would benefit from tools to measure social innovation? Georg argued that the key interest for academics would be in helping to answer important questions about where social innovation comes from – does it tend to originate where there are the most resources or in places where social needs are the most pressing?

Georgina Harrison from the Department of Premier and Cabinet New South Wales noted that for government policymakers, a way to measure social innovation would be useful to the extent it would enable government to know where innovation is already happening in society, and where their intervention is most needed.

Katherine Danton from UnLtd questioned whether isolating innovation via a measurement tool should be our goal. Many funders will be far more interested in establishing whether a particular project has created social impact, and less concerned about whether it was innovative or not.
There are also questions about the usability of measurement tools by different groups. Geoff highlighted that the most complex and nuanced measurement instruments might be the most accurate, but they are unlikely to be widely used.

How far can existing approaches take us?
Several participants argued that traditional indictors may be quite limited for capturing a dynamic and changing environment. It may be that new types of approach – for example doing large scale surveys with the populace to ask them about their experience of innovation in their work – might yield more interesting results. 

Martin Stewart Weeks also highlighted a potential danger with top down attempts to measure social innovation. We need to be cautious that macro level, academic attempts to capture what is happening on the ground do not smother what is only an emergent field of activity.
Many thanks to all our participants, to Geoff for chairing and to SIX for facilitating a great discussion. TEPSIE will be hosting a further TelePresence later this year looking at measuring the impact of individual social innovations – keep an eye on our website and newsletters for details.

Participants
Aabyhoj, Denmark
John Lauritzen Keller, DTI

Barcelona
Josep Pascual, City Council of Barcelona

Berlin
Rafael Ziegler, Ernst-Moritz-Arndt-Universitat Greifswald
Justus Lodemann, Ernst-Moritz-Arndt-Universitat Greifswald

Brussels
Marianne Paasi, European Commission

Budapest
Annamaria Inzelt, Financial Research Ltd
Laszlo Csonka, Financial Research Ltd
Anna Horvath, NESsT

Eschborn, Germany
Georg Mildenberger, University of Heidelberg
Eva Bund, University of Heidelberg
Georg Licht, ZEW

Lisbon
Luis Jeronimo, Gulbenkian Foundation
Nuno Frazao, Social Entrepreneurship Institute
Filipe Santos, INSEAD
Mafalda Frazao, Social Entrepreneurship Institute

London
Geoff Mulgan, NESTA (Chair)
Louise Pulford, SIX
Kine Nordstokka, SIX
Julie Simon, the Young Foundation
Anna Davies, the Young Foundation
Kieron Kirkland, Nominet Trust
Katharine Danton, UnLtd
Neil Reeder, Head and Heart Economics,
Ian Vickers, Middlesex University

Melbourne
Jane Treadwell, Australian National University

Sydney
Martin Stewart Weeks, CISCO
Andrew Young, UNSW
Les Hems, Tomorrow's Agenda Research Institute
Stephen Brady, Department of Premier and Cabinet, NSW
Georgina Harrison, Department of Premier and Cabinet, NSW
Haley Raey, Department of Premier and Cabinet, NSW
Mark Webb, Public Service Commission
Kevin Cosgriff, NSW Treasury