Road Accident Data Collection Form Design Research Project

7 Summary and recommendations

Summary

The aim of this project was to review the currently used versions of the STATS19 form and produce a new form that incorporates recommendations for improvement that could be potentially rolled out nationally to help improve the quality of the personal injury road accident data collected in Scotland.

This project has designed a revised STATS19 form based on:

An evidence review of literature relating to the design of STATS19 and other forms, including those for non-specialists
A consultation with Police and other users of STATS19
An expert workshop to determine the form design
A vignette study to compare the completeness and accuracy of data using existing forms and the revised form and to collate feedback

Following the results of the vignette study, further enhancements were made to the form to improve the completeness and accuracy of the data.

Literature review

The STATS19 data collection system was created in 1979, and since then, the design, content and appearance of the illustrative form have changed many times as part of quinquennial reviews. The key limitations and inconsistencies of STATS19 have been identified in previous research which suggested that improvements could be made to the form design, as well as standardisation and training on how STATS19 data should be collected and recorded.

Literature about form design for non-specialists suggests that any form should be tailored around the user and form purpose. Some designs are quicker for users to complete whereas others tend to lead to a better quality of data. Most studies found that participants preferred tick lists or free text boxes to drop down lists and other more interactive input modes but agreed that drop down lists lead to fewer data entry errors. The literature also agreed that participants find forms where the label is above or to the left of the answer box with right alignment the most comfortable as well as the quickest to use.

Consultation

Telephone interviews were undertaken with eighteen stakeholders from Police Scotland, Local Authorities, and Transport Scotland. All eight of the legacy Scottish Police force areas, believed to be using different STATS19 recording systems were represented in the consultation (seven legacy forces were represented by Police contacts, while the eighth was covered at Local Authority level).

Generally the types of form used, the processes followed and user perceptions varied. It became clear throughout the interviews that no consistent approach is used, despite widespread agreement that consistency is desirable. One legacy force uses a PDA to report data; the other respondents write notes in their notebook and complete a STATS19 form later at the Police station.

Local Authority representatives described one of the challenges that they face is related to errors about location data provided by Police Officers via the STATS19 forms. This data was viewed as critical for them to reach their road safety goals, particularly in terms of identifying problematic contributory factors or high frequency accident areas.

The consultation revealed that while Police Officers did not identify any specific areas of improvement, they did raise a number of small, usability issues. These mostly related to user friendliness and adding information or options to facilitate more accurate data input. None of the participants identified specific redundant or less useful variables, even when they suggested shortening the form to make it more user-friendly.

Form design and vignettes

A revised STATS19 form was developed using insights from the literature review and the consultation as well as expert opinion. The form was designed to ensure that it was in a usable format for the Police or other users. Based on the information about the way in which accident data was collected from the consultation, a paper-based form was not felt to be the most appropriate format, therefore the revised form was developed using Microsoft Excel.

The revised Excel form was designed to reduce or eliminate some of the accuracy issues highlighted in earlier tasks, while also incorporating any relevant best practice guidelines identified in the literature to enhance its layout and design.

Four vignettes (fictional accident case studies) were developed, refined and tested with Police Officers who had participated in the consultation. They were designed to include collision types known to cause confusion or result in inconsistencies. Qualitative feedback from users indicated that the revised form was well-received, and in addition, yielded more accurate and more consistent results than the forms that they were used to completing.

Some of the feedback on the draft revised form, as well as information obtained from the literature review, and the results of the analysis of the data received were used to make further amendments to the form. The output from this project has been the successful development of a suggested revised example of a STATS19 form, developed based on evidence from users of the form, which may lead to improved data quality.

Developing the form in Microsoft Excel was considered to be an improvement in terms of the accuracy and ease of completing, and respected Transport Scotland's requirement for a solution that did not need large scale IT resources, hardware and training. Table 4 below presents a hierarchy of forms ranging from the lowest technology to the greatest, with the advantages and disadvantages given of each. Paper forms and forms in Microsoft Word or Excel are advantageous due to users being familiar with these types of forms, but do not continually update a central STATS19 database. Web-based forms or a database, based at a central location or available via PDAs have the advantage that data input would directly feed into a central database, but this would require considerable IT infrastructure resources.

**Table 4: Types of data input form**
Form type	Advantages	Disadvantages
Paper form	Require no IT infrastructure	No error checking Handwriting errors Does not automatically complete database
Electronic form in Word	IT systems likely to have Word. Users likely to be familiar with use Can be sent electronically	Requires typing so mistakes may be made Does not automatically complete database Unable to perform logic and consistency checks
Electronic form in Excel	IT systems likely to have Excel. Users likely to be familiar with use Allows use of drop down menus Logic and consistency checks can be included Can be sent electronically	Does not automatically complete database
Electronic form in Access or web based	Automatically completes database More sophisticated logic and consistency checks can be included	Additional IT resources required May require training
Forms completed at scene of collision on PDA	Automatically completes database More sophisticated logic and consistency checks can be included Complete data at scene Automatically detect grid location of collision	Additional IT resources required May require training

Recommendations

In this section, recommendations (using the insights and information established as part of this study) are made. They are related to several key areas including further development of the type of STATS19 form used and the process for data collection, training opportunities and potential additional data that could be captured.

Type of form and process

Following the vignettes study, one Police area offered to test draft form version 2. This sort of pilot study could be carried out before a Scotland-wide distribution of the form.
Completing the STATS19 form is only part of the wider STATS19 process. The data from the form is reviewed, checked if necessary and compiled into the STATS19 database. The process used to extract data from the forms into the database varies by Police area, and the impact on this process needs to be evaluated based on the revisions to the form.
One of the Legacy Police Force areas already uses a web-based STATS19 form and another area uses PDAs to capture notes about the collision. Consideration should be given to what format the STATS19 form should take in the future to ensure that these divisions are not using a less sophisticated format than is available to them.
Any further developments should also consider existing software to collect collision data, for example, CRASH (Collision, Recording And Sharing software), MaapCloud (TRL's collision software), or the system already in use in the Ayrshire Police area. These systems would require additional IT resources and training which would need to be considered.

Training

The consultation showed that the current practice is for new Police Officers to be trained in recording road collision data at the scene of a collision with an experienced Police officer. Further training on why accurate completion of the form is important and how the data are used by other stakeholders may improve the completeness and accuracy of the form. In addition, although there is engagement between those collecting the data (i.e. Police Scotland), and those subsequently making use of the data (i.e. Local Authorities and Transport Scotland statisticians), this could be enhanced to ensure that all parties are bought into the process. This could take the form of stakeholder workshops where knowledge can be shared between all appropriate parties.
Our consultation showed that experience is key to completing the STATS19 data and most Police Officers write details of the collisions in their notebooks for transcription onto a form at the Police station. It was suggested that an aide-memoire could be produced which gave a summary of the information an officer needs to record at a collision. This would be useful for less experienced officers and when any new data are required.
Consultations with Police Officers showed that the recording of contributory factors did not appear to be a problem, and that reporting officers understood the system and completed these to the best of their knowledge (although there were a few suggestions for improvements). However, there is a lack of clarity amongst stakeholders about how these factors are assigned and they are often not considered to be reliable by users of the data. This mismatch between the data collectors and users of the data could be improved by training or engagement between both parties so that an understanding of the data collection process and the use of the data by others is understood by all.
The consultation exercise revealed that the guidelines for completing the STATS19 form (STATS20) are not accessible and therefore not used. In the revised form, selected elements of the guidelines have been added as part of the validation; however, it may be possible to provide a link to the relevant pages at various points in the Excel form, or to make the STATS20 more accessible in other ways. Any changes to STATS20 would need to be considered by SCRAS.

Data included

There could be other variables that it would be useful to collect data on which are not routinely collected as part of STATS19 that could be added to the form in the future for use within Scotland or across Great Britain. For example, the collection of driver experience and exposure data (i.e. length of time drivers have held a driving licence, approximate annual mileage and any previous motoring convictions) or the breath alcohol level. Any changes that would affect data for Great Britain would need to be considered by SCRAS.
Other items (not currently part of STATS19) were suggested for inclusion on the form as part of the feedback from the vignettes study. For example, insurance details and addresses of participants and description of collision. These could be added onto a revised form so that all of the information relating to a collision is stored in one location.

The research undertaken as part of this project has led to the development of a revised STATS19 form which was informed by those who use it most with a view to making it more user-friendly while also improving the quality of personal injury road accident data collected in Scotland. We believe that should Transport Scotland wish to take forward the above recommendations (in conjunction with ongoing engagement with key stakeholders), it will result in Police Scotland having a more robust, accurate and consistent approach to collection of personal injury road accident data across Scotland.