Approach

Introduction

To identify methods of appraising return on investment in zero emission / decarbonisation mobility innovation projects, a review of best practice was undertaken. The methodology for the literature review was systematic, in that it was replicable and transparent whilst providing a comprehensive, methodical, and critical assessment of the scope and quality of available evidence from the literature. Following the review and collation of evidence, a stakeholder workshop was held with Transport Scotland to discuss the findings.

Literature review methodology

A literature review was undertaken to collate evidence relating to appraising the return on investment for zero emission / decarbonisation mobility innovation projects. The methodology provided a comprehensive methodical and critical assessment of the scope and quality of available evidence from the literature. The literature review was comprised of the following seven steps:

  1. An inception workshop to confirm the extent of literature to be included, the type of decarbonisation projects and key outcomes to explore.
  2. Identifying literature to compile a long-list of candidate material for closer examination. A bibliography of 156 pieces of literature was identified which included government, industry, and academic publications. This was a two-stage process, with an initial focus on papers which focused specifically upon zero emission / decarbonisation mobility projects. However, on review it was acknowledged that the initial long-list did not encompass enough information on the impacts of projects on the economy and society. As a result, a broader secondary search of literature was undertaken focusing on the economic and social impact of innovation projects.
  3. Prioritising the longlist to determine an agreed shortlist of articles to be reviewed. The long-list of 156 papers were scored by two reviewers, with a score between zero (lowest) to four (highest) based on their abstract, to prioritise those most relevant to the research question. From the long-list, 32 papers scored a four and, upon agreement with transport Scotland, were taken through to a short-list for a more detailed review.
  4. Creation of a review template to ensure that insights from reviewing the literature could be appropriately structured. This included defined categories and a combination of population lists (to ensure consistency in reporting key metrics) and open text fields (to provide more descriptive commentary).
  5. Primary review of articles by a team of a three primary reviewers and completion of the review template for each article. A pilot review of eight papers was used to test the extraction framework and amend accordingly. An additional high-level review was also undertaken by two technical leads to verify the information presented and ensure consistency across the primary reviewers.
  6. Secondary detailed review of those papers considered to offer the greatest insight and to draw out further detailed information (above and beyond the review template). 18 core papers were considered to provide the greatest insight into causal linkages originating from research and innovation, provided evidence of specific metrics, and provided empirical estimates of the scale of impacts. Alongside an additional six papers that provided insights into specific topic areas, the secondary review of these 24 papers provided the main narrative for the development of the logging mapping process.
  7. Synthesis of findings to collate evidence across key dimensions from the data extraction sheet and identify how many studies provide evidence across different outcome measures and what approaches were used.

Stakeholder workshop

Following on from the literature review stage, a stakeholder workshop was held on the 22nd April 2024, attended by the project team and subject matter experts within Transport Scotland. The workshop included a process of open and constructive dialogue to ensure the input of stakeholder expertise, and contained the following activities:

  • The project team presented the preliminary findings identified to date from the literature review. This included the headline outputs, outcomes, impacts and metrics.
  • Discussion on the implications of the findings for appraising zero emission / decarbonisation research and innovation:
    • What is the evidence of quantified social returns?
    • What are the logic chains to delivering impacts?
    • Consideration of the transferability from other geographies and sectors?
    • Evidence gaps and how they might be filled?

Reporting

The findings from the literature review, and insights from the workshop, are captured within this report. This includes the development of the logic mapping process and a framework for establishing the returns from research and innovation into zero emission / decarbonisation mobility projects.