Case study: Assessing an impact investment fund using an impact audit
23 July 2019 at 8:24 am
How do we know what difference impact investing makes? Nesta commissioned an impact audit to assess its impact investment fund, writes Nesta CEO Geoff Mulgan as part of a series of case studies highlighting different approaches to capturing impact.
When Nesta Impact Investments 1 (NII1) was launched in 2012 it was one of the first impact investment funds in the UK. At the core of its investment approach was a commitment to rigorous evidence; ensuring that each investment would have a measurable positive social impact as well as a financial return.
In 2016, based on lessons learned over four years of impact investing, Nesta Impact Investments revised its strategy for impact. The key points of this revised strategy were:
- an explicit focus on alignment between impact and financial goals;
- a broadened concept of “impact risk” to include commitment to measuring impact and, for the first time, a framework for selecting potential investments;
- a renewed emphasis on quarterly support on managing impact to investees, underpinned by high quality monitoring data; and
- a commitment to transparency as a way both to demonstrate the performance of our fund and to contribute to the credibility of impact investment as an industry.
The impact audit was commissioned as part of this commitment to transparency. The audit was intended to assess the extent to which our new strategy was embedded in practice and to compare our approach to other approaches in the industry, in order to understand how it might be strengthened.
We believe this to be the first learning-focused audit of its kind in the UK and the first publicly available impact audit of a UK impact investment fund.
The method
Our auditors, Itad, conducted this audit using a rapid, participatory and learning-focused approach, engaging stakeholders and gaining feedback as the audit progressed. Itad interviewed all NII staff and members of the Investment Committee and three investees; reviewed all documentation including frameworks, tools, and reports along with external reporting; observed key meetings and forums including Investment Committee meetings and an annual general meeting; conducted a literature review of selected recent industry impact monitoring and management frameworks and industry practices; and, facilitated an analysis workshop with NII and Nesta colleagues. The scope and depth of the audit was tailored to fit a proportionate budget and short timeframe.
The audit
We welcomed the publication of what we consider to be a well-researched and thoughtful report from our impact auditors, Itad.
The key findings for us were:
- Overall, the auditors found that we had implemented our strategy well and that it met standards set by initiatives such as The Impact Management Project. In particular:
- Our partnership-oriented and flexible approach was highlighted as a particular strength. This is an area where NII has invested a lot of resources and we work hard to develop theories of change, impact plans and bespoke performance indicators for each of our investments.
- Our investment in building up knowledge in our particular focus areas was also picked out by the auditors as an area where we were doing well. This specialisation means we have a clear understanding of what “good” looks like in each sector, which provides a strong underpinning for our due diligence process.
- The auditors also helped to highlight areas where our approach could be improved. We agree with their assessment of our weaknesses, with a couple having particular resonance:
- First, the need to consider impact on systems as well as individuals, and unintended impacts on beneficiaries (positive or negative), both at the due diligence stage and after the point of investment. Currently, we do this in an informal and ad hoc manner and so may be missing some important aspects of the impact of our portfolio.
- Second, data collection has often lagged behind the intention in our impact plans and, as a result, our impact reporting has emphasised scale rather than depth of impact. This largely reflects the time it takes to transition to our new strategy but also the need to systematically update impact plans so that we maintain a focus on regular, high-quality data collection.
What we learnt
This impact audit has helped to set our priorities for the development and refinement of our impact strategy over the next 12 months. Our top three priorities will be to:
- develop an exit strategy for impact as the time for managed exits from our fund approaches;
- introduce a systematic approach for considering unintended impacts on beneficiaries, and for considering systemic impact; and
- review and upgrade our tools and processes to improve data collection from investees.
Our goals for this audit were:
- to validate our impact approach and identify areas for improvement
- to increase our transparency and build public credibility for our work
- to test a cost-effective approach that would enable impact investors to ensure they are meeting appropriate standards.
The first goal has certainly been met. The benefit to our internal learning has been clear and the audit has provided us with reassurance about our strengths as well as a clear pathway for improvement.
Whether this provides the external credibility and transparency we desired is too soon to tell, but we have received a lot of interest from stakeholders even before the report has been published. Itad proposed that, for our next audit, we appoint an independent audit panel with representation from investees, investors and other fund managers. We agree that this would serve to strengthen the transparency of the process and will certainly consider this for our next audit in around two years from now.
The benefit to our internal learning has been clear and the audit has provided us with reassurance about our strengths as well as a clear pathway for improvement.
In terms of cost-effectiveness, the financial cost was also relatively low compared to other evaluation approaches. If this audit was carried out every two years, it would have the same annual cost as a financial audit. As expected, the audit increased the workload for the impact director for a few weeks but the extra workload for the rest of the team was minimal. If we seek to treat social impact as seriously as financial success then this is entirely proportionate. We believe this is a very promising approach for actors in the impact investment industry to be sure that standards are being met.
About the author: Geoff Mulgan has been chief executive of Nesta since 2011. Nesta is the UK’s innovation foundation and runs a wide range of activities in investment, practical innovation and research.
This article was first published by Nesta.
It is one of a series of eight examples provided by Nesta to highlight different approaches to capturing impact they have used in recent years.
Read Geoff Mulgan’s article here about why it’s essential to try and track what is being achieved.
See also:
Case study: Capturing the impact of a challenge prize
Case study: Using matched-control groups to evaluate a volunteering scheme
Case Study: Randomised control trial of SME business support vouchers
Case Study: Embedding standards of evidence into grant-making
