Appendix B — TMfS User Satisfaction Form Feedback and Analysis

User Satisfaction Forms were returned for the following applications of TMfS:

Scottish Climate Change Programme
Glasgow City Plan II
Perth Tay Crossing
Kilbowie Park and Ride
Clyde Waterbus
Vale Corridor Park and Ride

User Feedback Assessment

The assessment below focuses on the four sections of the User Satisfaction Form identified above. The form asked users to score their application of TMfS on a scale of one to five, with one being the poorest and five the best. The results provided by each user coupled with the average score for that statement are provided in the table below.

Table B1 outlines the feedback provided for each application of TMfS:

Table B1: TMfS User Satisfaction Results

Question/ Statement	Scottish Climate Change Programme	Glasgow City Plan II	Perth Tay Crossing	Kilbowie P&R	Clyde Waterbus	Vale Corridor P&R	Average Score
How well did the model meet your needs?	N/A	4	5	4	3	3	3.8
How easy was the model to use?	N/A	4	5	4	4	4	4.2
How well did the data provided meet your needs	4	4	4	4	3	4	3.2
Was the data delivered in a format that was easy to understand?	5	N/A	5	N/A	N/A	N/A	5.0
The cost of deliverables was money well spent	N/A	N/A	4	N/A	N/A	N/A	4
Overall, how satisfied were you with the application of TMfS for your purpose?	5	4	5	4	3	4	4.2
Overall TMfS Average Score							4.1

It can be seen from Table B1 that there was generally a high level of satisfaction with TMfS. The most important ‘score’ is that provided for the overall satisfaction with the application of the modelling capability for the model user’s purpose. The average score for year two was 4.2, an improvement on the year one score of 3.8. It can be argued that the higher level of user satisfaction with TMfS is a reflection of the value generated from enhancing the modelling tools and liaising closely with the User Group.

The feedback on the use of the model and its data was weakest on the Clyde Waterbus and Vale Corridor Park and Ride applications. There was a general feeling that TMfS was not overly suitable for these applications, with each form returning a score of 3 for this question. However, this can largely be attributed to the fact that TMfS lacks data and model detail in the study areas in question. However, the fact that TMfS was used reiterates the message that it was still considered the best tool for the job.

Encouragingly, the TMfS was scored highly by the Scottish Government in the Scottish Climate Change Programme application. Building such confidence amongst the wider TMfS User Group is important in continuing to promote the wider use of the modelling capability and its data.

Tables B2-B4 outlines the level of user satisfaction with the Term Consultant for the following three categories:

Project Management and Communication
Technical Work
Deliverables

It should be noted that the forms returned for Glasgow City Plan II, Kilbowie P&R, the Clyde Waterbus and the Vale Corridor P&R were all filled in by the Term Consultant as they were the model users for these applications.

Table B2: Term Consultant Project Management and Communication User Satisfaction Results

Question/ Statement	Scottish Climate Change Programme	Glasgow City Plan II	Perth Tay Crossing	Kilbowie P&R	Clyde Waterbus	Vale Corridor P&R	Average Score
Project Manager (PM)/ User Liaison was easily accessible at all times?	5	5	5	5	5	5	5
How well did the TMfS Support Team keep you informed about progress?	5	N/A	5	N/A	N/A	N/A	5
How innovative was the PM/TMfS User Liaison in dealing with difficulties?	4	N/A	N/A	N/A	N/A	N/A	4
How proactive was the PM/TMfS User Liaison in dealing with difficulties?	5	N/A	N/A	N/A	N/A	N/A	5
How quickly were enquiries and requests dealt with?	5	N/A	5	N/A	N/A	N/A	5
How efficiently were enquiries and requests dealt with?	5	N/A	5	N/A	N/A	N/A	5
The PM/TMfS User Liaison demonstrated a ‘can do’ attitude	5	N/A	N/A	N/A	N/A	N/A	5
Overall, how willing was the TMfS Support Team to consult and offer advice?	5	N/A	N/A	N/A	N/A	N/A	5
How responsive was the TMfS Support Team to new ideas developed by the model user?	N/A	5	N/A	5	5	5	5
How willing was the TMfS Support Team to listen to the views of the model user?	N/A	5	N/A	5	5	5	5
Overall TMfS Average Score							4.9

It can be clearly seen from Table B2 that users have been highly satisfied with the project management and communication of the Term Consultant during their applications. The overall average score was 4.9, which provided an almost perfect return. It should however be noted that for the forms returned by the Term Consultant, a number of the questions were not answered as they were not deemed to be relevant.

Table B3: Term Consultant Technical Work User Satisfaction Results

Question/ Statement	Scottish Climate Change Programme	Glasgow City Plan II	Perth Tay Crossing	Kilbowie P&R	Clyde Waterbus	Vale Corridor P&R	Average Score
To what extent were the needs and requirements of the project understood and met by the TMfS Support Team?	4	N/A	5	N/A	N/A	N/A	4.5
Question/ Statement	Scottish Climate Change Programme	Glasgow City Plan II	Perth Tay Crossing	Kilbowie P&R	Clyde Waterbus	Vale Corridor P&R	Average Score
How would you rate the technical quality of the TMfS Support Team’s work?	4	N/A	5	N/A	N/A	N/A	4.5
Overall TMfS Average Score							4.5

It is difficult to draw generalisations on the quality of the Term Consultants technical work from Table B3, as only the two non-MVA users were able to fill in this section. The results we did gather indicate top quartile levels of satisfaction but a larger sample would be required to devise more general conclusions.

Table B4: Term Consultant Deliverables User Satisfaction Results

Question/ Statement	Scottish Climate Change Programme	Glasgow City Plan II	Perth Tay Crossing	Kilbowie P&R	Clyde Waterbus	Vale Corridor P&R	Average Score
Deliverables were produced on time and to specification	5	5	5	5	5	5	5
How would you rate the technical content of deliverables?	4	5	4	5	5	5	4.7
How would you rate the structure, language and style of the deliverables?	4	5	4	5	5	5	4.7
Overall TMfS Average Score							4.8

Table B4 indicates a high level of satisfaction with the quality of deliverables from the Term Consultant, with an average score across all areas of 4.8. Individual scores for each application are also all in the top quartile.

The User Satisfaction Form also seeks to ascertain an ‘overall’ level of satisfaction with the work of the TMfS Support Team. The average overall level of satisfaction was 4.2, which is again a top quartile score.

Table B5 outlines the level of user satisfaction with the role that we have played during each application:

Table B5: Transport Scotland User Satisfaction Results

Question/ Statement	Scottish Climate Change Programme	Glasgow City Plan II	Perth Tay Crossing	Kilbowie P&R	Clyde Waterbus	Vale Corridor P&R	Average Score
How efficient was TS’s procurement process?	N/A	4	3	4	4	4	3.8
How active a role did TS assume in the project?	4	N/A	1	N/A	N/A	N/A	2.5
How useful a role did TS fulfil in the project?	4	N/A	1	N/A	N/A	N/A	2.5
Overall TMfS Average Score							2.9

It can be strongly argued that Table B5 presents an unfairly low score for us, which is borne out of problems with the questionnaire. While the score for the procurement process is reasonable, the second and third questions assume that we should have an active role in each application. This is not in fact the case — our role is to initially facilitate the application and thus the questionnaire should be revised to reflect this. This problem, coupled with the low sample size, largely undermine the responses to questions two and three.

The overall average score of 4.3 for our performance is perhaps more reflective of the year one results.

The key metric to emerge from the User Satisfaction Form is the overall level of satisfaction with each application of TMfS taken as a whole. While customer feedback is valuable for each aspect of the TMfS application, it is strong overall performance that will engender confidence in TMfS and encourage future use of its modelling capability.

Table B6 outlines the overall level of satisfaction with each of the six applications:

Table B6: Overall User Satisfaction Results

Question/ Statement	Scottish Climate Change Programme	Glasgow City Plan II	Perth Tay Crossing	Kilbowie P&R	Clyde Waterbus	Vale Corridor P&R	Average Score
Overall assessment of the experience of using TMfS and interactions with TMfS Support Team/Transport Scotland	5	4	5	4	4	4	4.3

Table B6 reflects positively on the application based performance of TMfS in the last year. All of the overall satisfaction scores are in the top quartile, with the overall average increasing by 0.3 when compared against year one.

The challenge going forward is to maintain this high level of performance, whilst also trying to generate more responses from users.