Methodology

This section outlines and details the methodological approach taken to deliver the NTS People’s Panel Pilot, a deliberative study consisting of a series of events with a cross section of the adult population of Scotland. As a pilot study, it is important to outline and reflect upon the methods used, and to make recommendations for how these can be refined for future studies.

Diffley Partnership were contracted to undertake the following stages of the study:

• Recruit an appropriate number of members of the public to sit on the People’s Panel, including representation from: across of the country (urban/rural/island communities); across different demographic and equalities groups; across socioeconomic backgrounds; and across transport mode use. The recruitment was not to exclude any potential members from being selected. Transport Scotland was keen to ensure that the panel provides a platform for people from a variety of backgrounds, and with different levels of experience in engagement with policy development, including those whose voices are seldom or less frequently heard.
• Support participants to take part in the sessions to ensure they are fully inclusive. This included outlining of additional resources required to participate online/by correspondence. During the contract, Diffley Partnership and Transport Scotland were to consider the viability of in person participation, should easing of restrictions around COVID-19 permit face-to-face research. Support was to include reimbursement of costs incurred and any other financial costs linked to recruitment of the panels.
• Organise, facilitate and conduct sessions, using deliberative methods to develop potential policy initiatives which will support the delivery of NTS. These sessions were to be delivered as part of the wider NTS Forum timescales which are yet to be determined by the Transport Scotland Policy Integration Team.
• Plan and deploy skilled, deliberative facilitation of discussions at each panel meeting. In preparation, agreeing the methodology and topics in advance with the Transport Policy Integration Team.
• Provide Transport Scotland with comprehensive written reports from the sessions. These were to outline the issues covered and details of the discussions with the panel.
• Arrange monthly progress meetings with Transport Scotland.
• Provide any supporting analysis/outputs.

This project was kept to realistic, agreed deadlines to correspond to the time available for the pilot. The timetable was confirmed before approaching any potential participants. Considerations included:

• Ensuring there was sufficient lead in time before each Spotlight session to prepare all materials.
• Ensuring participation was manageable for participants with no more than one Spotlight session per month.
• Considering the impact of holiday timings on participants’ availability and Transport Scotland staff availability.

Panel Recruitment

This section sets out the approach taken to recruiting the Panel.

In discussion with the contractor - Diffley Partnership – it was agreed that the recommended number of panel participants should be 20. This was based upon:

• An online method for participant recruitment.
• Sufficient numbers recruited to enable the option of breakout sessions with small sub-groups for in-depth discussion.
• Longitudinal panel approach, with all participants going through the same participation journey.

With this study, it was important to form a panel with consideration towards the following:

• Demographics – ensuring a representative membership in terms of age (16+), gender and social class (socio-economic status).
• Equalities – ensuring representation of those from a range of equalities groups including BAME status, physical disability and those with limited digital capability (for example who have limited internet access) since these may impact on travelling experiences.
• Geography – delivering a panel with a membership made up of citizens from all parts of Scotland and in line with urban/rural breakdowns in the population as a whole.
• Transport characteristics - representation of citizens who used the full range of transport modes, in line with the wider population, which should include users of all modes of travel including walking and cycling as well as car users, those who use buses and trains, those who use airports and the waters of Scotland to travel.
• Engagement of seldom-heard voices – ensuring that the People’s Panel engages those who do not regularly take part in policy research. This required designing questions during recruitment to make sure that those whose voices are not generally heard are represented. It also required screening out those who had taken part in any qualitative research in the last year and including a question during recruitment to gauge interest in the transport policy and wider issues.

Often, for large and rolling panels, participants are chosen through random selection with a demographic stratification (known as sortition); a source such as the electoral register or council tax register can be utilised, and demographic stratification might be set for variables such as age and gender. Within these stratifications a random selection can be made. This is a robust means of recruitment, often used when panels are for local or central governments’ consultation.

Given the proposed scale of the People’s Panel, instead of a sortition method, this research drew on Scotland’s largest online panel, ScotPulse, for recruitment, enabling some useful and important quantitative measures on experiences of and attitudes towards transport to be collected at the same time.

The ScotPulse panel consists of over 32,000 adults (aged 16+) from across the country and is representative of Scotland’s population against a range of demographic and geographic criteria (see Annex A). Participation in ScotPulse surveys does not require access to a laptop or Wi-Fi because all surveys can be completed via smartphones if preferred by respondents. No ScotPulse member receives more than three surveys per month.

A quantitative survey was issued via ScotPulse to 1,000 members in September 2021. Survey respondents were asked for their interest in taking part in the panel, demographic information and questions about transport (see Annexes B). This survey exercise gained both useful data and acted as a recruitment survey. After reading information on the panel and what participation would involve, 285 respondents indicated they would be interested in taking part in the deliberative sessions.

From the 285 respondents, 20 preferred participants and five standbys were identified. The process involved filtering and random selection against the following criteria:

• excluded if selected ‘I have worked/ do work within transport sectors’
• excluded if selected ‘I have taken part in a focus group or research project on transport in the past year’
• ensuring that gender, age groups over 16+ and Scottish Index of Multiple Deprivation quintiles were representative of general population
• including people with disabilities or long-term health conditions
• including parents/ carers of children of different ages
• including a range of cars in household- from 0 to 5
• including a mix of transport usage
• including individuals who selected ‘there have been times I have avoided using public transport for fear of discrimination and/ or intimidation’
• including people who did not utilise technology to book or plan travel.

20 individuals were approached to join the panel and a further five were asked to be on standby in case participants dropped out. In advance of the first Spotlight, two participants dropped out of the process due to COVID-19 and a last-minute change to their availability. One standby member was brought in at short notice resulting in 19 individuals joining the panel in time for the first spotlight session.

Attendance was high and consistent for these 19 individuals across all four sessions.

Panel Management

This section sets out how the Panel was managed throughout the study.

Key documents were prepared for participants including Information for Participants, Code of Conduct and Privacy Notice. These were sent out to all participants at three time points:

• prior to sign up, as part of ensuring informed consent
• before and after Spotlight 1

One panel member was visually impaired. All slides were sent to that participant in advance of each session so they could access in their own time and utilise screen reader technology. The team made guest speakers aware of this request, and the need to share materials in advance of the session to pass onto the panel member.

The recruitment questionnaire identified which participants were not used to utilising technology. These participants were approached in advance to ensure they had appropriate equipment and understood how to connect to the internet and use the Zoom platform. In these cases, participants were most comfortable with household members lending them devices and running through how to use Zoom with them in-person. However, were this not, contingencies were in place to supply hardware and ensure software support to overcome any forms of digital exclusion. The participants reported throughout the process that through taking part they were becoming more confident using the technology and Zoom as a result.

An important aspect of panel management was ensuring that members received a nominal payment for their time participating.

Crucially, panel management included clear and responsive communication with participants. Participants would usually contact the team by email, even to say thanks for the sessions afterwards. One of the Panel’s facilitation team was responsible for monitoring contact and responding to their communication. As they were also involved in the delivery of the session, participants gained more familiarity with them during and in between the Spotlight sessions. Building in time and processes for this aspect of panel management was key to ensure the participants felt valued and were not waiting for any information they requested.

Spotlight Delivery

This section outlines how each of the Spotlight sessions were planned.

Four Spotlights were delivered in total. All required preparation, delivery on the day and then follow up activity. The points below summarise these stages:

1. Set up and design - meeting with Transport Scotland and facilitation team to agree purpose of each session, agree broad schedule and any expert speakers to invite.
2. Design of panel materials – following set up, draft all agreed session materials.
3. Pre-event communication with members – as explained above in Panel Management, communication with participants was vital before, during and between sessions.
4. Running the events – four events followed each NTS priority. The video conferencing software ‘Zoom’ was used as a user-friendly platform. Structure of sessions consisted of part plenary and part small group work. Sessions were recorded through Zoom so that the research team could re-examine and analyse afterwards. 
5. Post session activity and ongoing evaluation – members were asked to complete brief surveys at the end of each session, giving feedback to incorporate into subsequent sessions.

Each Spotlight included short presentations to the panel, short videos, but also plenty of time for small group discussion in break-out rooms and panellist-led feedback to the wider group. Spotlight sessions were time-limited based upon a reasonable time for an online session of 2.5 hours with a break at approximately the half-way point. The research team was conscious of not adding to screen fatigue of the participants.

As part of the set up and design of the sessions, it was crucial to agree the focus of the sessions in line with the NTS priority and the 2.5 hour time frame. The research team prepared Microsoft Planner Boards for each session to map out the priority, sub-aims, related policy measures from the Delivery Plan, and compile useful sources.

Another design consideration was to ensure each session had enough variety of listening and speaking opportunities. Although expert voices, and information from Transport Scotland, were important to provide panel members with context to stimulate their input, it was essential not to turn each Spotlight into an information session directed at participants.

Videos were selected which were related to the NTS priorities and would stimulate ideas and discussion. These were short – fewer than four minutes long so that they did not dominate the session and also to accommodate internet bandwidth to ensure they played effectively for all panel members.

At the outset of the project, there was a detailed discussion on how to incorporate deliberative elements into the sessions. This resulted in the inclusion of small group discussions and polls during panel sessions. The poll could be integrated into the Zoom meeting, and provided data on the panel’s reaction to key Delivery Areas within each NTS priority area.

At the early stages of the project, a Facilitation Plan template was prepared. This was then adapted for every Spotlight session. It was a working document, and additions and changes could be made right up to the day of the session. All facilitators referred to this during the session to keep track of order, timings, actions and roles and responsibilities.

Spotlight Facilitation

This section outlines how each session was facilitated.

Regardless of how well a session is planned and designed, skilled facilitation was essential for success. Ensuring time keeping and focus, and that participants were comfortable and engaged, was achieved.

Post-Spotlight feedback and evaluation surveys were designed to understand individual participants’ views, including on the facilitation itself. The research team reflected and discussed the results which then informed follow-up conversations with Transport Scotland.

The feedback was timely and a table summarising the feedback and adjustments made was prepared for participants, so they could see the difference this was making. Annex C includes the findings from the four Spotlight sessions. All average ratings were high across the five performance areas for each Spotlight session (see Annex C). The most common response in all cases was a rating of 10, except one with a rating of nine.

The research team also looked at the textual responses in relation to ratings to pick up on any issues experienced by individual participants.

Analysis and Reporting

During the period of arranging the Spotlights, reporting consisted of:

1. Compiling bullet points from break-out sessions for expert speakers: so that they could quickly see how the panel responded to their topic.
2. Creating high-level summaries for Transport Scotland: based upon the participant feedback survey, and to inform our intra-Spotlight discussions
3. Designing summaries for participants to ensure regular feedback and engagement between their participation in Spotlights.

In the final feedback survey, participants were asked: “Do you have any recommendations on ways Transport Scotland can continue to involve members of the public in transport policy and delivery?” Comments indicated participants were supportive of public engagement and deliberation.

Participants considered inclusion of voices:

I genuinely feel this was a very worthwhile and engaging exercise with ideas from all around the country. I felt that respect to all people’s views were given by both the Diffley team and the participants and strongly believe that similar methodology would work well.

I would personally love to be involved in more of these sessions if possible. A panel with rotating members?

Speak to children.

By these panels and remember it's not just cities that we live in.

I think this panel has been a great way of doing that and we have heard from a diverse group of people across Scotland. Perhaps you could reach out to communities representative of across Scotland and hold surveys and local exhibitions where you talk to visitors (local residents and business owners) and/or meetings for a broader spread?

Participants recommended regularity of engagement:

Regular surveys and focus groups like this.

By having a yearly focus group.

Participants suggested a range of research and engagement exercises:

Continue to ask for opinions...look to other countries.

Public consultation exercises and regular surveying of cross sections of the population.

Keep targeted focus groups going as well as public consultations.

Participants expressed a desire for engagement to make a difference:

Canvass users for improvements.

Keep employing panels like this to come up with specific objectives.

More of the same, many good viewpoints were expressed.

Continue sessions like what we have been doing. They have been very insightful for us as panel members but, I'm sure we have also given you a lot of ideas that could be looked into in more detail and possibly come to fruition.

This focus group was very interesting to be a part of but already there are some travel and transport announcements that didn't align with the thoughts of the group. If the focus groups are going to be meaningful then when announcements and strategies are published they should include how the issues we bring up are going to be addressed. If we are thinking these thoughts then others will also.

The last quote is fundamental to any deliberative research through public panels. Participants will be interested in the purpose of the exercise, the connection to decision making, and whether their participation is meaningful.