APPENDIX 3: USABILITY TESTING DISCUSSION GUIDE
APPENDIX 3: USABILITY TESTING DISCUSSION GUIDE
Discussion Guide
Introduction
- General introduction to Ipsos MORI, the format of the interview, audio and visual recording, presence of observers in the viewing studio and confidentiality
Explain to User:
- We are helping our client to understand how to improve their current web site and want your comments and feedback on the things that you are going to see. The website we are using will be live, so all of the links that you click on should work.
- We are independent from our client - I did not design these pages, and will not be offended if there are some parts you do not like! Please be honest about what you think.
- We will spend some time looking over the website and I will ask you to look for information that is available. It is not a test. If you can’t find something, that is a problem with the design that we will try to solve.
- I would like you to ‘think out loud’ whenever possible, keeping a running monologue, saying anything that comes to mind – let me know what you are thinking at each stage.
General Experience (5 mins)
- How often do you use the internet? What do you use it for? Do you buy online? What do you tend to buy? What else do you use the internet for? What information do you use the internet to look for? What about for transport related things? What do you look for? What websites do you use? How do you go about finding websites? Do you use search engines? Which ones?
Initial Browsing & Task Completion (10 mins)
SOME RESPONDENTS WILL BE INVITED TO EXPLORE THE WEBSITE FREELY. THIS WILL INDICATE HOW THE SITE APPEARS TO THOSE WHO ARE IN AN INFORMATION GATHERING PHASE OTHER RESPONDENTS WILL BE ASKED TO COMPLETE SOME PARTICULAR TASKS, TO REPLICATE HOW THOSE COMING TO THE SITE WITH A PARTICULAR ACTION IN MIND WOULD BEHAVE INITIAL REACTIONS, NAVIGATION PROBLEMS WILL BE GATHERED |
You have arrived at the home page of Transport Scotland. I would like you to spend a few minutes exploring the website. Feel free to click on any part of the site which you are interested in. This is a live website, so all of the links should work. Once you have spent some time looking around the site, we will review the different pages that you visited and try to find specific information. THESE RESPONDENTS WILL THEN BE ASKED TO COMPLETE VARIOUS TASKS. I would now like to ask you I would like you to try and complete some tasks on the website, which I will give you in a moment. Feel free to explore the site as you try to complete the task. This is a live website, so all of the links should be working. Once you have completed the tasks, we will go through the different pages you have visited and talk about them in more depth.
TASK ONE – Find the environmental statement for the A90 Balmedie to Tipperty trunk road project and its completion date TASK TWO – Find a map of the M74 project, find out the cost of the project and look for details on the archaeological dig taking place TASK THREE – Find the announcement by Ministers in the second half of 2007 announcing the preferred option to replace the Forth Road Bridge TASK FOUR – Find out what stations will be located on the Glasgow Airport Rail link route and find the timetable for the project TASK FIVE – Find the details of the national concessionary travel scheme as it applies to disabled travellers TASK SIX – Find information on the review of transport projects currently being undertaken by Transport Scotland TASK SEVEN – Find the latest Scottish Transport Appraisal Guidance and information relating to the evaluation of an option as part of the post-appraisal TASK EIGHT – Find volume of traffic information on the M8 for a location of your choosing TASK NINE – Find information about the planned improvements to the Edinburgh to Glasgow railway line TASK TEN – Find Transport Scotland’s Freedom of Information Publication Scheme TASK ELEVEN – Find what jobs are published on the website TASK TWELVE – Find statistics showing FirstScotRail’s most recent performance on the Edinburgh to Glasgow railway line TASK THIRTEEN – Find information on the winter maintenance service carried out on Scotland’s trunk roads TASK FOURTEEN – Subscribe to the Forth Replacement Crossing newsletter |
Detailed Page Reviews (25 mins)
| RESPONDENTS WOULD BE SHOWN THE PAGES ON THE TRANSPORT SCOTLAND WEBSITE AND DETAILED FEEDBACK WILL BE GATHERED | HOMEPAGE What did you think of this page? What were your initial reactions? Why? What did it make you feel about the website? Why? What did you think of the design and layout of the page? What do you think of the images? What about the text? What would you expect from FOR THOSE WHO WERE FREELY BROWSING When you first left the page, you clicked [interview specific]. Why did you click there? What were you looking for? What were you expecting to find? ROAD PAGE What were your expectations of this page? What did you expect to see? Why? What do you think about the design and layout of the page? PROBE FOR PICTURES, COLOUR, TEXT, FONT, SIZE Which of the content is most memorable? Why? Is the content clear? Why / why not? How would you improve this page? Probe: design/layout/content/images How would you move on from this page? Where would you expect to go from this page? What would you expect to find? Why? RAIL PAGE What were your expectations of this page? What did you expect to see? Why? What do you think about the design and layout of the page? PROBE FOR PICTURES, COLOUR, TEXT, FONT, SIZE Which of the content is most memorable? Why? Is the content clear? Why / why not? How would you improve this page? Probe: design/layout/content/images How would you move on from this page? Where would you expect to go from this page? What would you expect to find? Why? CONCESSIONARY TRAVEL PAGE What were your expectations of this page? What did you expect to see? Why? What do you think about the design and layout of the page? PROBE FOR PICTURES, COLOUR, TEXT, FONT, SIZE Which of the content is most memorable? Why? Is the content clear? Why / why not? How would you improve this page? Probe: design/layout/content/images How would you move on from this page? Where would you expect to go from this page? What would you expect to find? Why? PROJECTS PAGE What were your expectations of this page? What did you expect to see? Why? Were your expectations met? What do you think about the design and layout of the page? PROBE FOR PICTURES, COLOUR, TEXT, FONT, SIZE How would you improve this page? Is it clear how to access projects? Probe for major projects and the full list. How would you move on from this page? Where would you expect to go from this page? What would you expect to find? Why? SPECIFIC PROJECT DETAIL PAGES What were your expectations of these pages? What did you expect to find? Why? Were your expectations met? What do you think about the design and layout of the page? PROBE FOR PICTURES, COLOUR, TEXT, FONT, SIZE What do you think about the depth of information provided about the projects on this page? Is it sufficient? Is it too detailed? Why? How would you improve this page? How would you access further project information? What do you think about the depth of information provided on these pages? Is it sufficient? Is it too detailed? Why? Did you notice the links on left and the right side of the page? REPORTS PAGE What were your expectations of this page? What did you expect to see? Why? Were your expectations met? What do you think about the design and layout of the page? How would you improve this page? Is it clear how to access the reports? How would you move on from this page? Where would you expect to go from this page? What would you expect to find? Why? |
Registration for the Forth Crossing Newsletter ( 5 mins)
RESPONDENTS WOULD BE ASKED TO REGISTER FOR THE NEWSLETTER THIS WILL PROVIDE INSIGHT INTO THE EASE OF SUBSCRIBING AND THE DEGREE TO WHICH VISITORS ARE HAPPY TO PROVIDE THE NECESSARY INFORMATION TO DO SO |
How easy was it to find the link to subscribe to the newsletter? Was it where you expected it to be? IF NO. Where would you have expected to find it? How easy was it to complete the subscription form? PROBE FOR UNDERSTANDING OF DIFFERENCE BETWEEN HTML AND TEXT VERSIONS What did you feel about the information you were being asked to complete? Are you unhappy about providing any of this? IF YES What information and why? Do you expect to provide this sort of information to subscribe? What would you expect to happen as a result of collecting this information? |
Competitor Review (5 mins)
RESPONDENTS WOULD BE SHOWN ONE OR TWO COMPETITOR SITES AND ASKED FOR THEIR REACTION AND COMPARISONS WITH TRANSPORT SCOTLAND THIS WILL PROVIDE INSIGHT INTO WHAT WORKS WELL ON OTHER SITES, LEARNINGS WHICH CAN BE APPLIED ON THE TRANSPORT SCOTLAND WEBSITE |
Websites will include: Please take a few minutes to browse this website. What are your initial reactions to the design, layout and content? PROBE FOR PICTURES, COLOUR, TEXT, FONT, SIZE What do you like / not like about the website? How do you think it compares with the Transport Scotland website? Why? Is there anything on this website that you would like to see on the Transport Scotland website? |
New Functionality (5 mins)
RESPONDENTS WILL BE PROVIDED WITH IDEAS / INITIAL DESIGNS FOR NEW FUNCTIONALITY ON THE TRANSPORT SCOTLAND WEBSSITE AND ASKED FOR THEIR REACTIONS TO IT WILL TEST USER REACTION TO PROPOSED IDEAS |
One way of developing the website would be to provide information in different formats such as podcasts or videos. How helpful would this be for you? Why? Would you watch or listen to them them? When, how? And what sort of information would it be helpful to have in podcasts and videos? How long / short would you like these to be? Why? Who should present them? Another way of improving the website would be to allow users to personalise the homepage based on their preferences. So if you are only interested in rail issues, you could choose to have information on rail more prominently displayed on the homepage, or if you are only interested in one project or only projects in one area of Scotland, you could prioritise these. How useful would you find this functionality? Or do you think this is unnecessary for this website? ASK IF USEFUL How would you choose to personalise your homepage? What would you have on it? What about a map detailing where all the Transport Scotland projects are taking place? This map could also illustrate the precise location of upcoming projects. Do you think you would refer to this? Why? |
ENSURE THESE HAVE BEEN COVERED IN DISCUSSION ABOVE. IF NOT, CONFIRM DURING THE SUMMARY.
Navigation
- Do they feel lost at any point?
- Is the process logical/understandable?
- Does the order in which information is presented seem logical?
- What is good/bad about the navigation?
- Do users recognise calls to action throughout the process?
- Is the structure/hierarchy clear?
- Do users notice all buttons?
Terminology and Iconography
- Are the icons/labelling clear/meaningful? Why? Why not?
- Are the colours clear?
- Is the terminology understandable/does it makes sense? – How can it be improved?
- Are they clear sign posts for users? Do they miss any signposting? Why?
Summary
- What was thought of the overall process – length, ease, clarity
- What recommendations if any, for improvements?
- What final suggestions for the designers?
THANK PARTICIPANT AND CLOSE