Executive summary
Purpose
Mott MacDonald were commissioned by Transport Scotland to identify methods of appraising the return on investment for zero emission / decarbonisation mobility innovation projects to support future development of an innovations assessment methodology.
At present, the assessment of the impact of innovation work in this area is ad hoc, therefore a standard approach would assist project development and evaluation and help prioritise zero emission / decarbonisation mobility projects.
This study reviews cross-sector best practice, across a wide range of research and innovation literature, to provide a framework from which a future assessment methodology can be developed for zero emission / decarbonisation mobility innovation projects.
Approach
The methodology for the literature review was systematic, in that it was replicable and transparent whilst providing a comprehensive, methodical, and critical assessment of the scope and quality of available evidence from the literature.
A long-list of 156 documents was compiled, with 32 selected for detailed review. Following this review, and the extraction and collation of evidence, a stakeholder workshop was held with Transport Scotland to discuss the findings.
The findings from the literature review, and insights from the workshop, are captured within this report.
Summary of findings
The collated evidence highlighted a range of assessments that have been undertaken to examine the value of research and innovation projects in economic and social terms. This encompassed alternative evaluation techniques to establish the strength of relationships between investment in research and subsequent outcomes.
A wide range of individual outputs, outcomes and impacts from research and innovation investment are listed across the reviewed literature, alongside specific metrics to quantifiably capture the benefits.
The reviewed papers also presented evidence on the monetary scale of returns on research investment. Despite a broad range being presented across the papers, a lower and upper bound of 10% and 85% rate of return is suggested, with a central estimate of 20% to 40%.
However, the papers also identify a series of key factors that influence benefit realisation. These factors suggest both positive and negative effects and provide a balanced insight into what may help or hinder the causal chain of research and innovation from input to impact.
Framework for estimating returns
The literature review indicates there is a wide scale of potential returns from research and innovation (Section 3.4) and that there are a significant number of factors that affects the outcomes and impacts (Section 3.5). There are a range of potential logic chains by which bespoke investment in research and innovation can feed through from outputs into outcomes and impacts (Section 4.2) and the mechanisms for achieving wider economy, societal, governmental, and international impacts are complex, reliant upon individual context, and have not readily quantified (Section 4.3).
Utilising this information, a framework has been developed that seeks to identify a check list of elements that would be used to evaluate each individual zero emission / decarbonisation mobility project to flag the likely areas where impacts may be derived. This has been done by considering:
- What are the main factors that are likely to determine the benefits that can be derived from zero emission and decarbonisation mobility research and innovation?
- What are the implications for forecasting the scale of potential impacts?
Six broad categorises of factors have been identified that can affect the scale of potential returns:
- The type of research and innovation being undertaken
- The component element of the research approach and development process
- The precise technical nature of the research and innovation
- The area of research and the networks and talent available to support the delivery of projects and maximise the outcomes.
- The context and conditions of the sectors / industry in which the outcomes from the research and innovation may be applied
- The context and conditions of the country of origin of the research and innovation, in this case being Scotland.
These six categories encompass a total 21 individual component factors that the literature review has indicated will impact upon the scale of returns from zero emission / decarbonisation mobility research and innovation projects. Alongside the evidence of quantified economic rates of return, these can be used to forecast the potential impacts of public sector investment within individual research and innovation projects.
The central case forecast of economic and social rates of return of between 20% to 40% offers a generic starting point for implied research and development impacts. By then considering the extent to which an individual zero emission / decarbonisation mobility project is likely to perform against the 21 identified component factors, it would then be feasible to make a judgement as to whether the rates of return for that project are likely to be greater or lower than core estimates.
A potential approach would be to establish a proforma of questions relating to each of the 21 component elements. The responses to each component would provide an indication as to whether benefits are likely to be higher or lower than average. Summing this cumulatively would provide a basis upon which to conclude whether, overall, a project is likely to deliver higher, lower, or comparative rates of return to the core estimates.
An indicative proforma of questions has been developed as a basis for adjusting the forecast rate of return range. This is presented within Table 4.2 in the main body of the report.
Next steps
It has always been recognised that this literature review of best practice will form an initial, albeit important, phase of a wider process in developing an assessment methodology for research and innovation into zero emission / decarbonisation mobility projects. The following three aspects have been identified as key next steps in the process:
- Social impacts. Collate additional case study evidence from individual zero emission / decarbonisation mobility projects to improve the understanding of causal relationships from inputs and outputs from these projects leading through to wider social impacts relating to topic areas, such as health, wellbeing, social capital, and environment.
- Subject the provisional framework to some initial practical testing on case study examples. This can then lead to a period of refinement and further development of the framework, and specifically the proforma, to ensure it becomes a viable practical tool that can be readily applied across all future projects.
- Consider how the framework can be best integrated within existing appraisal tools. Part of this will involve the dissemination of the approach to relevant stakeholders to ensure a consistent understanding of the framework, but also obtain feedback on how it aligns with other existing appraisal frameworks. Particularly consideration will need to be given to how the framework would align to Scottish Transport Appraisal Guidance and its potential integration.