4 Measuring Performance

4 Measuring Performance

The purpose of the Performance Management Framework (PMF) is to support us in achieving our delivery priorities through a robust, transparent and repeatable process for recording, monitoring, analysing and reporting performance across a wide range of criteria.

A PMF, that links strategic and operational criteria, is fundamental to a holistic asset management approach. It enables us to assess and demonstrate the impact that different investment strategies have on the performance of our network, road user satisfaction and engineering/contract measures and targets. In particular the PMF:

• Facilitates effective communications with stakeholders by demonstrating performance against their requirements.
• Demonstrates how funding is used effectively.

4.1 Supporting Corporate Vision

In Transport Scotland we have produced an asset management policy and strategy setting out what we intend to do to manage the country’s strategic road infrastructure assets. This strategy aligns with the corporate vision and demonstrates the contribution that asset management makes towards achieving this vision.

The alignment between corporate vision, asset management objectives and performance measures in accordance with international best practice, is shown schematically in Figure 4.1.

Our asset management objectives are broad statements that describe the performance requirements of trunk road infrastructure assets in terms that all stakeholders can understand. These relate to outcomes and cover key aspects of asset performance such as user satisfaction, safety, serviceability and sustainability. Our asset management objectives are focused on the performance of the whole network rather than the performance of individual assets.

Each asset management objective is supported by a number of service delivery standards and performance measures which enable both overall achievement of the objective and individual aspects of performance to be measured.

Figure 4.1: Aligning Performance Measures with Corporate Vision

4.2 Our Performance Management Framework

Asset Management Objectives

Our asset management objectives, shown in Figure 4.2, are our high level objectives that reflect our business responsibilities, and can be aligned with our strategic objectives and priorities. Performance measures related to the objectives through our service delivery standards enable us to present a consistent and meaningful picture of changing performance over time, i.e. to present trends of improving, steady state or declining performance.

Figure 4.2:  Asset Management Objectives

Service Delivery Standards

Our service delivery standards are measures that support our asset management objectives and describe what each asset management objective actually means in terms of the service to be delivered. The service delivery standards, currently in the PMF, are embedded in the way our Operating Companies deliver their work.

Performance Measures

Central to asset management is actively engaging with road users to understand their views about the service they expect from the trunk road network. Our performance measures are focused on areas of genuine interest to road users and other stakeholders, enabling their levels of satisfaction and expectations to be recorded (see Section 9.5). As further development of the PMF we will review how the existing performance measures align with our current delivery priorities and Scottish Government’s strategic objectives.

One of the considerations in our review of the performance measures will be to maximise the use of data that is already collected, minimising the need for the collection of additional data. We already have a number of sources of relevant performance measures, including:

• Road User Satisfaction Measures – road user surveys are undertaken annually to identify priority areas for improvement and determine levels of satisfaction with the service.
• Business Key Performance Measures – asset maintenance and management business measures used by Transport Scotland.
• Service Provider Key Performance Indicators – the key performance measures compiled to monitor the performance and delivery of the Service Provider Contracts, as defined in the 4G Operating Company contract.

Performance measures have been selected that align with our asset management objectives through consultation with responsible individuals in the organisation. Each of the current performance measures is supported by a one-page specification that details its description, purpose and measurement metrics.

The transition of the term maintenance contracts from 3rd Generation (3G) to 4th Generation (4G) has meant that some originally identified performance metrics have evolved and datasets across contract generations are not always comparable. In addition successive Scottish Government Spending Reviews will define our levels of funding and allow us to develop appropriate targets. As a result our review of the PMF will look to facilitate the setting of updated measures once the direction of travel in performance scores and impact of the transition in contracts and affordability is understood. The identification and subsequent updating of targets will be through consultation with senior Transport Scotland officers and analysis of road user feedback exercises. 

Consideration will need to be given to the short, medium and long -term cost of achieving each target, within our available budgets, and the costs of meeting these targets will be periodically reviewed by the Transport Scotland board.

The link between our current asset management objectives and supporting service delivery standards and performance measures are as follows:

Monitoring Performance

In refreshing the PMF each asset management objective will be rated on an excellent, good, fair and poor scale and each supporting measure is also rated on the same scale. Appropriate weightings, determined based on the importance of the measure, will be updated, as required, to allow a rating for the asset management objective to be calculated. This will be defined for each measure and the following are provided as the overarching principles that we will adhere to.

Poor:

• Does not meet minimum standard required in contract.
• Does not meet the perceived minimum acceptable standard.
• Unable to determine performance due to insufficient or substandard information and/or data.
• Road users are, on average, dissatisfied or very dissatisfied.

Fair:

• Meets minimum standard required in contract but below targets.
• Meets the perceived minimum acceptable standard.
• Road users are, on average, neither satisfied nor dissatisfied.

Good:

• Meets recognised good practice as identified through national guidance.
• Meets the contract requirements/targets.
• Meets perceived good performance standards.
• Road users are, on average, fairly satisfied.

Excellent:

• Exceeds recognised good practice as identified through national guidance.
• Exceeds the contract requirements/targets.
• Exceeds perceived good performance standards.
• Innovates, sets and achieves challenging targets that will improve service.
• Road users are, on average, very satisfied.

As part of the update to the PMF we will be reviewing the existing measures and weightings to ensure they are still appropriate as well as determining how any new measures will be measured and the contribution they should make to the overall asset management objective.