Larkhall – Milngavie Railway Project Evaluation Study Final Report

1. Executive Summary

1.1. SYSTRA were commissioned by Transport Scotland in March 2014 to undertake an evaluation of the Larkhall – Milngavie railway project and to provide recommendations for improvement of the draft Rail Evaluation Guidance.

1.2. The evaluation found that the project has been a success in terms of standard Transport Economic Efficiency (TEE) measures with the project's benefits outweighing its costs. This is primarily due to higher than expected demand. However, there is only limited evidence to support the success of the project's wider objectives.

1.3. To conduct the evaluation, data was gathered and analysed to provide an evidence base to establish the extent to which the project has met its original objectives and the five Scottish Transport Appraisal Guidance (STAG) criteria.

1.4. The six project objectives were:

Project Objective 1: to reconnect Larkhall to the rail network to allow the introduction of a half-hourly service;

Project Objective 2: to double the frequency of services between Hamilton and central Glasgow and between Milngavie and central Glasgow to four trains per hour;

Project Objective 3: to remove an operational bottleneck on the North Suburban line;

Project Objective 4: to increase the attractiveness of Larkhall and Kelvindale and the surrounding areas for inward investment and land development;

Project Objective 5: to offer social inclusion benefits for residents; and

Project Objective 6: to encourage a modal shift towards public transport.

1.5. To establish an evidence base, a variety of techniques were employed, using a combination of primary and secondary research including:

  • a User Survey of rail passengers to understand their characteristics and the impacts of the project on their travel behaviour;
  • a Business Survey to understand the impact of the project on local businesses;
  • accessibility analysis to assess how public transport journey times have changed since the completion of the project; and
  • secondary data sources to establish actual station demand and revenue, and examine trends in local socio-economic indicators such as population and employment.

1.6. The first three operational objectives have all been achieved. For the remaining three, in the absence of quantitative targets, it was more difficult judge the extent to which they have been achieved. There was evidence from the two surveys undertaken and the accessibility analysis that positive contributions have been made towards social inclusion benefits and a modal shift towards public transport (objectives 5 and 6), particularly in Larkhall.

1.7. For objective 4, whilst there was some evidence from the User Survey to suggest that Larkhall and Kelvindale have become more attractive places to live and work, there was limited evidence of inward investment, land development for commercial purposes and Wider Economic Benefits (WEBs) from the Business Survey. Examination of local socio-economic indicators also showed there was no conclusive evidence to suggest that the Larkhall – Milngavie project has had a significant or measurable wider economic or social impact, other than data to support an increase in home building in the Larkhall area. It is however acknowledged that these are long-term impacts and may yet still materialise.

1.8. In terms of utilisation, the project has been a success with actual passenger demand exceeding forecasts. For example, in 2012/13, actual passenger demand at the four new stations was 26% higher than forecast. Reasons for the difference were explored. It is thought that exogenous drivers of rail demand in general such as increasing employment in city centres and car dis-benefits such as congestion, parking constraints and high fuel prices, as well as population being higher than forecast have been key factors. Other factors may include not accounting for the inclusion of Park & Ride facilities and the method by which demand abstraction from other stations was modelled.

1.9. The project's original Benefit Cost Ratio (BCR) was 0.66, indicating that the project's monetised benefits would not outweigh its costs (although it was acknowledged that the project would bring other non-monetised benefits such as regeneration impacts which would make the project value for money). Owing to changes in appraisal methodology since the original BCR was calculated and the availability of actual values to replace some of the forecast values, the BCR was recalculated. The new value was calculated as 2.77 which is significantly higher than the original appraisal value and indicates that the project has delivered value for money. Even when reverting to the original appraisal assumptions, the project's benefits still outweigh its costs with a BCR of 1.83.

1.10. Drawing from the learning points encountered whilst undertaking this evaluation study, a number of recommendations for the improvement of the draft Rail Evaluation Guidance have been made. The key recommendations are to:

  • develop 'SMART' project objectives which can be effectively and continuously monitored post-project completion;
  • conduct the Process Evaluation soon after project completion;
  • identify the data required to effectively appraise, monitor and evaluate the project including the use of surveys to better understand the characteristics and behaviour of users and potential users both before and after project completion;
  • ensure data collection is an on-going exercise rather than a task that is only considered as part of the Outcome Evaluation;
  • consider innovative survey design including the use of new technology and social media to ensure a more targeted yet cost-effective survey approach;
  • ensure all project documentation is comprehensively archived and safe-guarded to make sure the relevant and correct information is readily accessible which will aid the future monitoring of the project;
  • ensure all demand modelling assumptions made and outputs prepared at the appraisal stage are comprehensively documented; and
  • undertake sensitivity tests using a range of economic conditions when preparing demand forecasts to reflect the inherent uncertainty in forecasting.