8. Conclusions
8. Conclusions
8.1 This chapter summarises the main findings and conclusions from each of the chapters in turn.
8.2 Overall the findings from the process evaluation suggest that the delivery and implementation of the Laurencekirk station reopening was a success. The project was delivered on time and on budget. A number of factors contributed to this success. In particular:
- consideration was given to the organisation best placed to deliver the project and effectively manage the risks
- there was a clear scope that was fully understood by all delivery partners
- elements of the procurement process were specifically tailored to deliver value for money
- there was a carefully thought out consultation process designed to allow continued engagement with stakeholders and the local community that helped to generate buy in and avoid delays to implementation
8.3 The latter may have been aided by strong local support for the project.
8.4 Another important conclusion from the process evaluation is that it should be carried out much earlier and preferably during the delivery stage itself. The main reasons include being able to learn early lessons that can be implemented to improve the ongoing delivery and ensure that those best placed to provide input are in post at the time.
8.5 The aim of the outcome evaluation was to measure the performance of the project against the four transport planning objectives of:
- Link rural commuters to centres of employment, educational establishments and other facilities;
- Encourage greater use of public transport by connecting the township of Laurencekirk and its surrounding area to the rail network;
- Encourage modal shift from private car to public transport by constructing a Park and Ride facility serving the new station; and
- Improve road safety by encouraging a reduction in trips made by road and through reduced road traffic.
8.6 Overall the reopening of Laurencekirk railway station has performed well against the objectives.
8.7 On the first, it is clear that the project is helping to link rural communities to the various services and facilities. The journey times by public transport have been reduced, in places significantly, and many more journeys are being made by public transport than before the station was reopened.
8.8 On the second objective, this has also been met, with the evidence showing that over half of journeys being undertaken via the station were previously being made by car.
8.9 Similarly, on the third objective, many of the users of the station Park and Ride facility would previously have undertaken their journey by car and currently live too far from the station to access it by foot. This suggests that without the Park and Ride they would continue to use the car to travel to their destination.
8.10 The evidence supporting the final objective is less conclusive. While the survey results point to a reduction in car kilometres it is difficult to demonstrate that this has led directly to an improvement in road safety.
8.11 While the evidence suggests that the transport planning objectives are being met, it is difficult to measure the extent of the success due to the objectives not being SMART. Future appraisals of rail projects should design SMART objectives, or at least indicators or targets, which reflect the extent of the identified transport problems so that the evaluation can quantify the success of a project.
8.12 At the outset, consideration needs to be given to the evaluation so that plans can be put in place to gather data that may be required but is not necessarily readily available. Without this information, particularly on the counterfactual, it is difficult to carry out an effective evaluation.
Review of Laurencekirk STAG Appraisal
8.13 Other important elements of the study were to review the economic elements of the original 2004 STAG appraisal of the project, with a view to understanding:
- why the passenger and revenue figures for Laurencekirk station exceeded those forecast in the STAG appraisal;
- Revised STAG, including Transport Economic Efficiency / Benefit Cost Ratio (BCR) for the scheme; and
- the ‘Wider Economic Benefits’ (WEBs) of the project.
8.14 On the first bullet, there was a lack of clarity within the STAG report on how the revenue forecasts were calculated. On the information that was available there were a number of questions around the methodology, such as:
- are all new rail trips assumed to have transferred from car?;
- are all new rail trips from Laurencekirk new to rail, i.e. no transfer (via Park and Ride) from existing stations?;
- are any entirely new trips generated?;
- has any change in destination choice occurred?; and
- is there an assumed loss of revenue to other modes (e.g. bus)?
8.15 All of these questions are important in understanding how the forecasts were derived and why the forecasts underestimated the outturn passenger numbers.
8.16 An important conclusion of this task is therefore that the key underlying cause and effect assumptions and analysis must be recorded and laid out in the STAG report if a latter evaluation process is to understand why outcomes may have diverged from forecast.
8.17 On the second element of this task, using the outturn data to recalculate the BCR shows that the Benefit Cost Ratio increases significantly due to the higher than forecast benefits generated by the scheme. It rises from the original estimate of 1.5, to between 2.5 and 4.4 depending on the scenario considered. For example, the 4.4 BCR figure covers the currently used 60-year appraisal period, no timetable disbenefit to those using services which travelled through Laurencekirk station and revised population growth projections.
8.18 On the WEBs analysis, there was no evidence to suggest that there were agglomeration benefits from the station reopening. In addition, while some respondents to the survey suggested that the reopening of the station had encouraged them to enter the workforce, due to a better commute, or move job, the impacts are likely to be minimal given the number of people providing a positive response. Nevertheless, there were positive responses to the labour supply questions which could mean greater impacts for larger schemes which may not be being captured in conventional transport appraisal methodologies.
8.19 Overall there is no clear evidence from the data examined that the reopening of the station at Laurencekirk has had a significant and measurable wider economic and social impact. While there have been some positive impacts across a number of the metrics considered, for example the housing and labour markets, the impacts in Laurencekirk post station reopening do not appear to be significantly different from those witnessed in other areas considered, e.g. Aberdeenshire and Scotland as a whole.
8.20 It must be noted however that the impacts of the station may not have firmly bedded in and it may take a longer period for these to materialise. In addition, some of the economic and social data is not yet readily available, e.g. census data. It may require some time before the data becomes available for analysis and firm conclusions can be reached. Until then the findings of this section on wider impacts should be seen as preliminary.
Recommendations for Rail Evaluation Guidance
8.21 Chapter 7 set out a number of recommendations to be considered for the ongoing development of Transport Scotland’s Rail Evaluation Guidance. These covered many of the issues raised as part of this study, including developing SMART objectives during the appraisal process to aid evaluation and not evaluating WEBs impacts of small station re-openings.
8.22 However, the main focus of the comments centred around the importance of data collection, and in particular the need to gather information and data in advance of the outcome evaluation, rather than as part of it. This includes information that will not be readily available from official published sources (this particularly applies to local impacts of small station re-openings) and also information to fully understand the counterfactual that does not rely on reliable and accurate memories of survey respondents. Overall, the identification and planning of data collection needs to become an integral of the overall appraisal process if outcome evaluations of rail projects are to become valuable and lessons learned are to become reliable and improve future investment.