12. evaluation study conclusions
12.1. This chapter summarises the main findings and conclusions with respect to the two main study objectives:
- an assessment of the extent to which the project has met its objectives and the five STAG criteria, and whether the project has offered value for money;
- provide recommendations for improvement of the draft Rail Evaluation Guidance.
12.2. The evaluation found that the project has been a success in terms of standard Transport Economic Efficiency (TEE) measures with the project's benefits outweighing its costs which is primarily due to higher than expected demand. However, there is only limited evidence to support the success of the project's wider objectives.
Achievement of Objectives
12.3. The six project objectives were:
- Project Objective 1: reconnect Larkhall to the rail network to allow the introduction of a half-hourly service;
- Project Objective 2: double the frequency of services between Hamilton and central Glasgow and between Milngavie and central Glasgow to four trains per hour;
- Project Objective 3: remove an operational bottleneck on the North Suburban line;
- Project Objective 4: increase the attractiveness of Larkhall and Kelvindale and the surrounding areas for inward investment and land development;
- Project Objective 5: offer social inclusion benefits for residents; and
- Project Objective 6: encourage a modal shift towards public transport.
12.4. The first three operational objectives have all been achieved. For the remaining three, in the absence of quantitative targets, it was more difficult judge the extent to which they have been achieved. There was certainly evidence from the two surveys undertaken and the accessibility analysis that positive contributions have been made for objectives 5 and 6, particularly in Larkhall:
- the rail improvements have led to reductions in public transport journey times to key destinations across the project area, improving accessibility and promoting social inclusion;
- there is evidence from the survey findings that the rail improvements have encouraged a mode shift towards public transport which has likely resulted in abstraction of car trips as well as a small net decrease in car ownership.
12.5. For objective 4, although there was some evidence from both the User and Business Surveys that the project had increased the attractiveness of Larkhall and Kelvindale, this was not supported by examination of local economic indicators; this suggested that overall the rail project has not had a significant or measurable wider economic or social impact. There is some evidence however of the rail project being a factor in the increased levels of homes being built in the Larkhall area in particular. It is also acknowledged that the full benefits of the rail improvements could take many years for to materialise.
12.6. The analysis of WEBs generated by the project also suggested that these too had been limited to localised, small-scale impacts.
12.7. Nevertheless, overall the project can be considered a success in terms of utilisation, with actual passenger demand exceeding forecasts and the recalculation of the BCR showed this was significantly higher than that reported in the original appraisal and indicates that project has delivered 'value for money'.
Recommendations for Rail Evaluation Guidance
12.8. The recommendations listed in chapter 11 were developed through encountering a number of issues when conducting this evaluation study.
12.9. The main recommendations are to:
- develop 'SMART' project objectives which can be effectively and continuously monitored post-project completion;
- conduct the Process Evaluation soon after project completion;
- identify the data required to effectively appraise, monitor and evaluate the project including the use of surveys to better understand the characteristics and behaviour of users and potential users both before and after project completion;
- ensure data collection is an ongoing exercise rather than a task that is only considered as part of the Outcome Evaluation;
- consider innovative survey design including use of new technology and social media to ensure a more targeted yet cost-effective survey approach;
- ensure all project documentation is comprehensively archived and safe-guarded to make sure the relevant and correct information is readily accessible which will aid the future monitoring of the project;
- ensure all demand modelling assumptions made and outputs prepared at the appraisal stage are comprehensively documented; and
- undertake sensitivity tests using a range of economic conditions when preparing demand forecasts to reflect the inherent uncertainty in forecasting.
12.10. Inclusion of these recommendations will promote a more robust,
evidence-based evaluation of rail projects. This will enhance the ability to demonstrate that the observed project outcomes and impacts have been caused by the intervention rather than external influences.