Evaluation of Laurencekirk Railway Station
1. Executive Summary
1.1 In 2004 Aberdeenshire Council commissioned work to appraise transport options which could help meet the identified transport problems / constraints and deliver opportunities for the town of Laurencekirk and the surrounding area. The work was carried out in line with the Scottish Government / Transport Scotland’s Scottish Transport Appraisal Guidance (STAG). The findings of the STAG appraisal pointed towards the re-opening of Laurencekirk Railway Station as a possible option which could deliver a value for money solution in line with the stated transport planning objectives, which were to:
- Link rural commuters to centres of employment, educational establishments and other facilities;
- Encourage greater use of public transport by connecting the township of Laurencekirk and its surrounding area to the rail network;
- Encourage modal shift from private car to public transport by constructing a Park and Ride facility serving the new station; and
- Improve road safety by encouraging a reduction in trips made by road and through reduced road traffic.
1.2 Following the STAG Part 2 appraisal, a business case for the station was developed and the station was reopened in May 2009.
1.3 In December 2013 Transport Scotland commissioned a study to undertake an evaluation of the impact of the new station. This involved a process evaluation to assess the effectiveness of the implementation and delivery aspects of the project, and an outcome evaluation to determine whether the intended outcomes and objectives have been achieved.
As part of the evaluation, the commission required a number of specific issues to be considered. These included:
- an information gathering exercise / survey to obtain a better understanding of users of Laurencekirk station and their travel behaviour prior to the station reopening;
- analysis to understand why outturn passenger numbers and ticket revenue have exceeded forecasts;
- analysis of the outturn costs and benefits to generate a retrospective Benefit Cost Ratio for the project;
- gaining a fuller appreciation of the ‘Wider Economic Benefits’ (WEBs) of the station and also how it has impacted on the local area; and, finally
- making recommendations on the Draft Guidance on Rail Evaluation being prepared by Transport Scotland.
1.4 The process evaluation was heavily informed by information gathered from a series of interviews with the key organisations involved in the delivery and implementation of the project, as well as a number of stakeholders.
1.5 From the evidence gathered as part of the interview and consultation process, it can be concluded that the delivery of Laurencekirk railway station was a success. The project was delivered on time and on budget. Key factors that contributed to the success were:
- Careful consideration being given to who is best placed to manage risks and take overall ownership of delivery;
- A well thought out scope that is clearly understood by all parties;
- Procurement methods specifically tailored to meet the needs of the project; and
- A carefully thought out, well-designed and transparent stakeholder and community engagement process to generate participation and buy in.
1.6 To inform the outcome evaluation and develop an understanding of users’ travel behaviour an online user survey was undertaken. The recorded trips made by respondents were scaled up using the frequency of travel information provided to generate an overall figure of 52,745 recorded journeys over the previous 12 months. For context, this compares with the latest figure for the total number of exits and entries at the station in 2012-13 of 92,470.
1.7 The findings of the outcome evaluation show that the re-opening of the station at Laurencekirk has had a positive impact as measured against all the original transport planning objectives.
1.8 The re-opening has played an important role in linking the local community in the town and surrounding area to centres of employment, educational establishments and other facilities. It has also had a positive impact on those travelling for business purposes. The results of the survey show that a number of journeys would not be made if the station had not been reopened. Accessibility analysis carried out also demonstrated that facilities have become much more accessible, in terms of journey time, for those without access to a car. This has been particularly so for facilities in Aberdeen where some journey times by public transport have been reduced by up to 50 minutes.
1.9 The findings from the survey also show that the station has encouraged the use of public transport. The results reveal that the station re-opening has not only enabled people to make journeys by public transport that they would not have previously made, but it has also resulted in people making journeys that they would have previously made by car. In addition, 16% of people who travel out of the station reported that they had reduced the number of vehicles they owned as a direct result of the station reopening and less journeys being made by car. Furthermore, of those using the station, the majority travel to the station on foot, suggesting that the station reopening has also contributed to an increase in active forms of travel.
1.10 The findings of the survey also reveal that the availability of the Park and Ride facility at the station has encouraged a shift from the private car to public transport. It is clear from respondents that a large share of people using Laurencekirk station also use the Park and Ride facility and a significant percentage of these previously made or would have made the journey by driving all the way to their destination. Without the Park and Ride facility it is likely, from the evidence gathered on the distance between home and station, that a number of people would drive all the way to their destination, i.e. it is too far to walk on a daily basis.
1.11 The findings show that the station re-opening has resulted in a small reduction in car trips. They also show that a large proportion of these trips would have been made in the peak periods and therefore most congested times. The resulting overall reduction in car kilometres on the network suggests it has had a small but positive impact on road safety, particularly on the A90 into Aberdeen, although this cannot be observed and attributed wholly to Laurencekirk station.
1.12 The original STAG appraisal was carried out in 2004. To understand why outturn passengers have exceeded forecasts, and to generate a retrospective Benefit Cost Ratio, a review of the original STAG report and analysis was undertaken.
1.13 In undertaking the review of the original 2004 STAG report it became apparent that it does not contain enough detailed information, and sufficient clarity on the assumptions made, to determine the precise reasons why the forecasts were lower than outturn passenger numbers.
1.14 Because the approach was high-level and not totally consistent with the expected methodological approach, it was also not possible to simply re-run the analysis using the outturn data. In addition, the approach recommended in the current appraisal guidance is different from that which existed in 2004. Consequently the cost benefit analysis was re-done in line with current guidance.
1.15 Using the outturn data, the revised BCR ranged from 2.5 to 4.4 depending on the scenario and assumptions used. This compares with the BCR in the original report of 1.5.
1.16 Current transport appraisal guidance includes advice on techniques to capture impacts that have traditionally not been captured in conventional appraisal. This includes Wider Economic Benefits (WEBs). Analysis was therefore carried out to understand the impact of these Wider Economic Benefits from the re-opening of Laurencekirk railway station. The analysis revealed that agglomeration benefits are negligible. It also revealed that while there may have been labour supply impacts, as the new station encouraged people to move to more productive jobs or those not employed to enter the workforce, the impacts will be limited for projects similar to Laurencekirk i.e. a small station with relatively few users.
1.17 Overall there is no clear evidence from the data examined that the reopening of the station at Laurencekirk has had a significant and measurable wider economic and social impact. While there have been some positive impacts across a number of the metrics considered in, for example, the housing and labour markets, the impacts in Laurencekirk post station reopening do not appear to be significantly different from those witnessed in other areas considered, e.g. Aberdeenshire and Scotland as a whole. In addition, some of the positive trends, for example population, tend to be continuations of what was being experienced prior to the station reopening.
1.18 While there is little evidence to suggest that the reopening of the station has had wider positive social and economic impacts, it must be noted that the impacts of the station may not have firmly bedded in and it may take a longer period for these to materialise.
Recommendations for Rail Evaluation Guidance
1.19 A number of recommendations have been made for inclusion in, and development of, Transport Scotland’s Rail Evaluation Guidance. These are intended to improve the evaluations, both process and outcome, and help ensure more effective and value for money rail projects are delivered in the future.
1.20 The recommendations cover issues related to the development of SMART objectives, when to include WEBs analysis as part of the outcome evaluation and recording methodological assumptions related to the appraisal.
1.21 However, the key recommendations focus on timing and data collection. It is strongly recommended that the process evaluation is carried out early in the delivery and implementation stages of a project. This will help ensure that both experience and issues are fresh in the mind of those involved so that important lessons can be learned. Importantly, it will allow changes to be made if delivery is not going to plan, and also that key individuals involved in delivering the project are still in post and available to provide inputs to the evaluation.
1.22 The most important recommendation relates to data collection, in terms of what data is to be collected, the method of collection and the timing of collecting the data. Fully understanding the behaviour of users of rail schemes before the intervention and the details of the counterfactual are crucial to effective evaluations. The Laurencekirk outcome evaluation largely depended on the memories of users and their answers to the online survey to establish their travel behaviour before the station was reopened, and what they would have done had the station not reopened. Whilst this worked well, there are inherent risks with the accuracy and reliability of the responses. It is important that this baseline information is collected ahead of project implementation, or at the very least as quickly as possible after project opening, i.e. data should ideally be collected in advance of the outcome evaluation and not as part of it.
1.23 In addition, an outcome evaluation may not be able to rely on official published sources to inform pre and post project impacts. This is particularly so for projects that are small and / or in relatively rural areas as the socio-economic data may not be available at the required spatial level. Careful consideration of data requirements therefore needs to take place early, e.g. at appraisal stage, so that if the information is not available to compare the performance or outcomes of a scheme against the intended objectives then measures can be put in place to gather it.