3 Methodological Approach

3 Methodological Approach

Overview

This Chapter provides an overview of the approach to conducting the ‘User’ and ‘Non-User’ Surveys.  It includes details of the aims of each survey, describes how each survey was administered and provides a summary of the sample achieved.

User Survey

Survey Content 

The primary purpose of the User Survey was to gather information on the current travel patterns and behaviours of users and how these have changed since the re-opening of the line.  In order to inform the Stage 1 Evaluation, it was important that the questions selected for inclusion (and therefore the data collected) aided the assessment of the scheme against the Investment Objectives (See Table 2.1).  The latter therefore formed a key input into the question selection and design process. 

In addition, as discussed above, initial evidence suggests that there has been an increase in tourist trips to the Scottish Borders and Midlothian since the re-opening of the line.  While not explicitly included within the Investment Objectives for the scheme, this growth in visitors is an important impact and therefore in order to capture this change a series of questions specifically aimed at leisure tourists were also included in the survey.  These included questions on:

  • accommodation and trip spend;
  • the type of accommodation used;
  • the type of leisure activities undertaken;
  • the extent to which the re-opening of the Borders Railway was a factor in choosing to make the trip; and
  • what respondents would have done had they not decided to visit Midlothian / Scottish Borders.

It is recognised however that due to the time period over which the surveys were undertaken (21st November 2016 – 4th December 2016), the number of responses from visitors is likely to be low compared to that which would be achieved in peak season.  

Finally, to develop understanding around the factors which influence an individual’s choice to use the railway, questions on access to other modes (including car ownership and possession of the National Entitlement Card) as well as views on specific aspects of the service such as capacity, reliability and frequency and the extent to which these issues have influenced use were also sought.

In developing the survey, cognisance was also taken of the questions within the Borders Baseline Household Survey and the National Rail Passenger Survey (NRPS) in order to ensure consistency where possible.  

Method 

Various options were considered for collecting the user survey data and it was concluded that the most cost-effective option would be an on train, researcher administered self-completion survey.  The rationale for selecting this approach was as follows:

  • Given that the target population for the study is users of the railway, it was reasoned that at station or on-board surveys offer the most efficient solution because they provide ready access to this population and can therefore be conducted more cost effectively than less targeted approaches;
  • Given the number of responses required and the relatively short timeframe available for data collection it was reasoned that self-completion surveys rather than the more time intensive researcher led interviews would provide a more efficient solution; and
  • Given the low average number of passenger boardings / alightings per day at some stations on the route (e.g. Shawfair) it was reasoned that administering the surveys on-board the train rather than at the station would result in a more efficient use of researcher resources. 

Survey Administration

The user surveys were administered by fieldworkers on the train.  The researchers distributed the surveys on a carriage-by-carriage basis.  Passengers were encouraged to complete the survey there and then and the completed responses were collected back in by the researchers before they moved onto the next carriage.  Where completing and returning the survey on-board the train was not possible, a postage paid return envelope was provided so that participants could return the completed questionnaire in their own time.  In addition, where an individual required additional assistance in completing the survey, a telephone number and free call back service was offered so that respondents could complete the survey over the telephone. 

The user surveys were undertaken over six separate days (Monday, Tuesday, Thursday, Friday, Saturday and Sunday) during the period 21/11/16 – 04/12/16.  A Fieldwork Schedule was developed prior to conducting the surveys using passenger counts provided by ScotRail.  This covered trains departing Tweedbank northbound between the hours 0559 to 1759 and departing Brunstane southbound between the hours of 1605 to 1901.  

Achieved Response

Number of Responses

In total, 1,112 User Survey responses were received.  A breakdown of the sample characteristics is provided in Appendix A. Each respondent to the survey provided an indication of how frequently they make the trip they were making at the time of the survey (see Chapter 4, Section 2.4).  The responses to this question were then used to calculate an estimated annual return trip figure and an annual single trip figure (by multiplying the former by 2) for each respondent using the conversion factors outlined in Appendix B. Using this approach an estimated 140,000 annual single trips were captured by the sample.  This equates to over 10% of the passenger journeys recorded in the first full year of opening.  It is noted that, unless otherwise stated, all graphs in the remainder of the report are based upon the number of responses.

Non-user Survey

The second element of the primary research was a survey of non-users of the Borders Railway.  The methodological approach adopted for the delivery of this is provided below.  

Sample Identification

The characteristics of the non-user population is unknown.  As such, it was assumed that non-users are comparable to the wider population and the sample was targeted towards achieving a representative sample of this.  To do this, the catchment tiers utilised as part of the 2015 Borders Baseline study were utilised.  These are discussed below.

Catchment Tiers

Residents of the Scottish Borders and Midlothian are likely to have been affected in different ways by the line (and therefore display different behavioural responses to it) as a result of their different geographical locations.  For example, those living within walking distance of the line may display different behaviours from those living within a driving catchment of a park and ride site.  To account for this, for the Borders Baseline Study TRACC accessibility planning software was used to identify a series of potential catchments based on access to the stations.  These are shown in Figure 3.1 and defined as follows:

  • Tier 1: areas where walk-in access to the new stations is possible (<15 minutes), taking account of the walking network, including off street, footpaths and any new active travel based links to the new stations;
  • Tier 2: excluding Tier 1, areas where reasonable bus-based public transport access is possible (i.e. along bus routes serving the stations) – e.g. by bus within 15 minutes in both an AM and PM peak period; and
  • Tier 3: excluding Tiers 1 and 2, areas where only car-based access to stations is realistic (within 20 minutes), and the new stations will become the closest P&R option for accessing Edinburgh. For example, Penicuik residents would not be expected to use the Borders railway to access Edinburgh, despite being approximately 20 minutes’ drive from a station.

In order to obtain a representative sample of non-users it was important to ensure a sufficient number of responses would be drawn from each of these tiers.  The benefit of this approach was that it provided a systematic representation of the three main groups of potential users (walk in, bus-based potential and car-based) in each local authority area.  If the sample had not been targeted in this way, a good representation of non-users may not have been obtained, and the findings could be skewed by an over or under representation of some groups.

Figure 3.1: Geographical extent of Catchment Tiers

Figure 3.1: Geographical extent of Catchment Tiers

Survey Content

The primary purpose of the Non-user survey was to gather information on any perceived barriers to using the service.  At the outset of the study it was agreed that the definition of a ‘non user’ should be broadened to include ‘one-off’ users.  A new line such as the Borders Railway is likely to generate a significant number of one-off users, particularly during the first year of operation and including these within the Non-user survey would provide an opportunity to understand why they have not made greater use of the railway.

The non-user survey included questions on:

  • Main purpose of trip on the Borders Railway (one-off users only);
  • Origin / Destination Stations (one-off users only);
  • Reasons for not using the Borders Railway or not using it more frequently (including, for example, the cost of rail travel relative to bus; difficulty getting a seat or getting on the service; and the reliability of the service);
  • What improvements would encourage respondents to use the service more frequently; and
  • How often and where respondents would travel using the service if the improvements they selected were made.

Survey Method and Administration

The Non-User Survey was conducted by telephone and specifically targeted residents living within the identified tiers as discussed above.  The contact details for respondents were drawn from:

  • A database of responses to the 2015 Borders Railway Baseline Household Survey
  • a telephone database of residents of the Scottish Borders and Midlothian living within Tiers 1, 2 and 3 as defined above

For the 2015 Borders Railway Baseline Household Survey participants were asked to provide their postcode and state a) whether they anticipated using the Borders Railway in the first 12 months of operation and b) if they would be willing to take part in future research.  In total, 251 respondents said they did not intend on using the service and would be willing to take part in future research.  Of these, 171 lived within the identified walk, public transport or car catchment areas.  This cohort of respondents formed an initial contact list of potential non-users.  The Borders Baseline database was then supplemented by a purchased telephone database of residents based within the identified walk, public transport or car catchment areas. 

Achieved Response

In total, 227 responses were received to the Non-User Survey.  Overall, the number of responses received via the Borders Baseline database was relatively small, accounting for just 13% of the overall sample. In part this was because a relatively large proportion of this sample (19%) had used the Borders Railway despite saying they did not anticipate using the service in 2015.

Table 3.1 provides a breakdown of the responses by geography, with the results shown graphically in Figure 3.2.  Some 74% (n=169) of the overall sample live in Midlothian, with 26% (n=58) based within the Scottish Borders.  

The larger number of respondents from Midlothian perhaps reflects the fact that the number of users in the Scottish Borders has significantly exceeded forecast figures and therefore non-users are smaller in number and more difficult to locate.

Figure 3.2: Geographical Distribution of Respondent Home Locations

Figure 3.2: Geographical Distribution of Respondent Home Locations

In terms of the tiers, 20% (n=45) of respondents live within Tier 1 areas (where walk-in access is possible); 41% (n=93) live in Tier 2 areas (excluding Tier 1, where reasonable bus-based access is possible); and 39% (n=89) live in Tier 3 areas (excluding Tiers 1 and 2, where car based access is realistic).

Table 3.1: Absolute Number and Percentage of Total Responses by Location
Number of Responses Tier 1 Tier 2 Tier 3 All Tiers
Midlothian 38 75 56 169
Scottish Borders 7 18 33 58
Total 45 93 89 227
% of Total
Midlothian 84% 81% 63% 74%
Scottish Borders 16% 19% 37% 26%

Table 3.2 shows the 2011 population within each tier alongside the achieved sample and sample rate.  Based on the population in each location, the sample rate ranges from 0.1% to 0.5%, with the largest proportion of respondents drawn from Tier 1 Midlothian.

Table 3.2: Total Population aged 16 (Census 2011) and Sample Rate
Tier 1 Tier 2 Tier 3 All Tiers
Total Population aged >16 Midlothian 7,814 20,594 16,555 44,963
Scottish Borders 5,797 10,280 14,754 30,831
Achieved Sample Midlothian 38 75 56 169
Scottish Borders 7 18 33 58
Sample Rate Midlothian 0.5% 0.4% 0.3% 0.4%
Scottish Borders 0.1% 0.2% 0.2% 0.2%

Figure 3.3 below shows the age breakdown of respondents and the mid-2015 population estimates for the Midlothian and Scottish Borders study areas.  Overall, the sample includes a broad cross section of age categories, with the largest proportion of respondents in the 45-54 age bracket. In comparison to the total population, the sample from the Midlothian area is older with a smaller proportion of people aged between 16-44 and a larger proportion of people aged 45-74.  In contrast, the Scottish Borders sample has a larger proportion of people in the middle age categories (35-44, 45-54 and 55-59) with a smaller proportion aged 60+ and 16-24.

Figure 3.3: Age Breakdown for Study Area (NRS mid 2015) and Respondents 

Figure 3.3: Age Breakdown for Study Area (NRS mid 2015) and Respondents 

Figure 3.4 shows the distribution of respondents by employment status compared with the Census 2011 distribution for Midlothian and Scottish Borders study areas.  The Midlothian sample has a lower share of people in employment and a larger proportion of retirees and unemployed individuals than the population of the Midlothian study area as a whole. The sample from the Scottish Borders is more similar to that of the study area, albeit with a smaller proportion of students.

Figure 3.4: Employment Status Breakdown for study Area (Census 2011) and respondents

Figure 3.4: Employment Status Breakdown for study Area (Census 2011) and respondents

It is therefore concluded that the sample achieved in the non-user survey is a reasonable representation of the local population.