Measuring the value of social innovation
I’ve just got back from an interesting workshop on “Measuring the Value of Social Innovation” hosted by the Work Foundation, an independent research centre which explores issues around innovation, economic growth, labour market disadvantage and health and wellbeing, among other things. They are about to start a programme of work around measuring social innovation so this workshop was an opportunity to get a sense of the key players, approaches and issues in the field.
Lizzie Crowley from the Work Foundation talked about the need for rigorous and robust approaches to measuring social innovation and social value – particularly in terms of making the case for investment in social programmes and preventative actions but also in the UK context where new forms of commissioning and the new Public Services (Social Value) Act is putting extra pressure on organisations with public service delivery contracts to demonstrate their social value. In addition, calls for better metrics are emerging from the growing social investment market.
However, as Lizzie pointed out, measuring impact is notoriously difficult - one needs to distinguish between outputs, outcomes and impact; evaluations can be very costly; and there are, inevitably, more fundamental issues around causality. She highlighted four central issues or questions facing the field: whether we need standardised measurement systems; or social auditing systems; whether commissioners should be setting guidelines for measuring social value and; how government, ventures and other stakeholders can be more transparent and share their information.
Cliff Prior, Chief Executive of UnLtd referred to the measurement of social innovation as the ‘holy grail’ – it is both magical and mythical. And yet, metrics are of great importance: social investors need to know what social returns they’re getting and to know this, we need more rigor, better definitions, more effective tools and approaches. But there are challenges – in many cases, the main beneficiaries of social innovations are not the customers (in many cases, a social venture’s actual customer will be the state). In these cases, measurements are often decided by the state, rather than those using the services, or even those delivering the services. This can skew priorities and practices. Also, many metrics focus on intentions – what organisations would like to do or think they’re going to do. But these measures leave out unintended consequences (which can be both negative and positive). These externalities also need to be measured and taken into account.
But most importantly, Cliff highlighted the fact that the kinds of metrics which are appropriate to later stage social ventures will not be appropriate for early stage social innovations. Indeed, social innovations require new measures – measures which are contextual, subjective and developed with beneficiaries, front line staff and a range of other stakeholders. These metrics need to be developed iteratively, with feedback loops and so on. Cliff suggested some possible approaches including improving data literacy among social sector organisations, radical transparency (especially for smaller organisations that can’t bear the costs of more formal evaluations) and making customers beneficiaries where possible (in order that metrics are most closely aligned with their needs and experiences).
Kieron Kirkland from the Nominet Trust started with an important question - what is the purpose of an evaluation? Partly they enable organisations to reflect (internally) on their practice, which in turn promotes continuous improvement, but they can also help to articulate an organisation’s value to a range of external stakeholders. In this way, the term evaluation is much more helpful than the term ‘impact assessment’. Kieron talked of the highly subjective nature of social value and therefore warned against any shared metrics or indicators. And in particular, argued that early stage innovations need to focus on reflection as a tool for development. He also talked of Nominet’s experience and their new evaluation model (which will come out in the New Year) which draws on the work of Cognitive Edge. This brings together complexity thinking (or an awareness that the social challenges and social interventions being measured are highly complex) and storytelling (collecting a range of ‘data points’) to evaluate an organisation’s development from a range of viewpoints.
A range of other issues were also discussed - like what is the role of open data and big data? Is the drive for better metrics coming from funders and investors? How can we reconcile the needs of investors for shared metrics with the need for subjective and contextual metrics? Another issue which we at TEPSIE need to address isn’t only how we measure the ‘social’ in social innovation – we also need to measure the ‘innovation’. If we value innovation then we also need to be able to measure it. But, how do we do this? How can we measure or capture the innovativeness of particular programmes? This adds another layer of complexity but is just as important. If funders, investors and governments want to invest in social innovation there do need to be some approaches to measurement which capture the innovativeness of new ideas, projects and services – otherwise, many funding programmes will fund projects which yield social value but are not innovative. Worse still, they might fund more of the same under the banner of social innovation.