Appendix D — Valuing TMfS

Appendix D — Valuing TMfS

Valuing TMfS — Methodologies

As part of the ongoing user engagement programme, the methodologies by which the value of TMfS is calculated have been reviewed, largely with the aim of producing a more contextualised view.

Given the multi-faceted nature of TMfS, it offers value on a number of different levels and to a number of different groups, including Transport Scotland and the User Group. Perhaps most importantly however, TMfS is a key tool in ensuring that the public interest is advanced through its contribution towards delivering the Scottish Government’s Purpose. TMfS provides a consistent, Scotland-wide analysis and appraisal framework. Such a facility allows both ourselves and other organisations to assess schemes and policies on a like-for-like basis, thus minimising risks and ensuring value for money.

In order to reflect the diverse nature of the value of TMfS, four independent but valuable methodologies have been developed, each assessing its value in a different light. These methodologies are outlined Table D1 below.

Table D1 TMfS Valuation Methodologies

Methodology	Description
M1	Transport Scotland Investment — assessment of the value obtained from each "stage" of our investment.
M2	Policy Context — the role of TMfS in supporting the policy context.
M3	Qualitative Assessment of ‘Added Value’ — discussion of the qualitative benefits of having a consistent ‘off-the-shelf’ model available for use by model applicants.
M4	Quantitative Assessment of ‘Added Value’ — quantitative valuation of having TMfS available ‘off-the-shelf’ (in terms of the cost savings from having to develop a ‘new’ model).

It should be noted that while each of these methodologies offers a legitimate means of looking at different aspects of how TMfS adds value, there is an element of overlap and they should not simply be added up to obtain an overall ‘value’.

Figure D1 presents these methodologies in a graphical format and the reader would be well served to refer to this figure throughout the remainder of this chapter.

Figure D1 Valuing TMfS

The formula ‘C * R +V’ under ‘Methodology M1’ is explained more fully below. In short however, the formula denotes that the value of our investment in TMfS is equal to:

• "the cost of developing a model from scratch (C) * requirement for a model (R) + [Client] Value (V)", which would give an approximate estimate of the value of having access to an ‘off-the-shelf’ version of TMfS for each application.

The value obtained from each of the methodologies outlined in Figure D1 is explained throughout the remainder of this chapter.

Methodology M1 - Transport Scotland Investment

We have made a substantial investment in TMfS and it would not be appropriate to carry out an assessment of its value without first considering how each step in the investment process adds value.

One method of assessing the value of our investment in TMfS is to consider the alternative scenarios when some or all of the investment in TMfS was not made. For example, a major element of our investment programme has been focused on developing liaisons with key stakeholders and the user engagement programme. What this methodology attempts to do is to compare the value of the user engagement programme to date, against a scenario where the user engagement programme does not exist.

The Components of Transport Scotland Investment

Our investment in TMfS comprises four separate components, as follows:

• Investment to create and maintain a multi-modal modelling suite with basic functionality, including the investment needed to ‘future-proof’ this
• Investment to improve and add functionality (including user consultation to identify requirements)
• Other user engagement (User Group Days, conferences etc) designed to increase user awareness of the Model’s capabilities
• Payment for Term Consultant Staff modelling support and advice

Taking each of these components in turn, it is possible to identify four scenarios that compare the difference between not investing in TMfS and investing in TMfS. The corresponding ‘Do Nothing’ versus ‘Do Something’ scenarios are as follows:

• Having no TMfS versus regularly updated (and future-proofed) ‘Basic TMfS ’
• Basic TMfS with no new functionality versus TMfS with new and improved functionality
• Limited awareness of TMfS versus increased awareness
• Non-Term Consultant Staff versus Term Consultant Staff carrying out the TMfS-based appraisals etc

Each of these trade-offs is considered in turn below.

‘No TMfS’ versus ‘Basic TMfS’

This scenario compares the benefits of having TMfS against not having such a facility. The ‘benefit’ of having the basic Model for a given application is made up of:

• Avoiding the cost of creating an alternative model for each application of TMfS (either from the beginning or by updating an existing model), which would have sufficient functionality to undertake the necessary appraisal
• Any added value which TMfS provides over and above the alternative (e.g. consistency with other TMfS-based appraisals, more robust data etc)
• The benefit to study timescales of having a model and the relevant data available ‘off the shelf’

For each application of TMfS it should be possible to identify:

• The scale of model required (i.e. broadly Local, Regional or National) and hence the approximate cost of creating such a model (£C)
• The level of requirement R for a model (e.g. 0-100%)
• A subjective estimate of the client’s value ‘V’ of having an ‘off-the-shelf’ model and data

That is, the value of having TMfS against not having it is equal to Cost x Requirement + [Client] Value (C * R + V), which would give an approximate estimate of the value of having access to an ‘off-the-shelf’ version of TMfS for each application.

Basic TMfS’ versus ‘Enhanced TMfS’

This section covers the investment in adding additional functionality (i.e. creating TMfS:07 and the subsequent Forth Model) and the cost of stakeholder consultation to discuss, agree and progress these enhancements. If these cost-benefit assessments can be made, they would determine whether a given enhancement and item of data collection was good value for money, or simply adding functionality for the sake of it. The various steps taken to ensure a positive cost/benefit include:

• Ensuring that any enhancement has a strong level of user support/demand, either from a single major application or a number of smaller applications
• Justifying (and costing) the proposed enhancements and data collection items to Transport Scotland
• Involving the TMfS Steering Group in the decisions on which enhancements to pursue

Assuming that these enhancements increase the future number of applications of TMfS, their costs will eventually be incorporated into the main Cost/Benefit assessment of TMfS described in the previous section (i.e. adding to the overall cost of creating and maintaining TMfS and offset by the estimated ‘C*R + V’ value to users). This ‘value dividend’ will come with a time-lag, associated with the time needed to complete the enhancements and disseminate its availability to potential users. This effect will be most evident during 2009, where the costs of preparing TMfS:07 and the Forth Model will be absorbed by the application of these models.

‘Limited Awareness’ versus ‘Increased Awareness’

This aspect covers the added value of increasing the awareness of TMfS, such as through the User Engagement Programme, the TMfS User Group Days and the TMfS Website. The benefits of TMfS can be partially assessed by measuring the increase in the number of users and applications to use the model over time (and the decrease in the number of mis-uses and unfounded criticisms of the model) and compare this with what would have happened if the programme to increase awareness had not taken place.

In addition, the stakeholders liaisons established add value by allowing for data to be shared across different policy areas, while also fostering awareness of TMfS. Organisations such as Scottish Water, the General Register Office for Scotland and the Transport Scotland Concessionary Fares Team have all benefited from either the sharing of data or knowledge with us. The added value obtained from these liaisons can be partly seen in the increasing number of non-transport based applications of the model. One of the key benefits of improving user awareness is that the cost of development is shared among an increasing number of applications.

Term Consultant Staff User Support

Another cost associated with TMfS is the consultancy payments for modelling and other support. The need to assess the net value of Term Consultant staff advice and/or running the model is only relevant if:

• The Term Consultant staff hourly rates are significantly higher than the rates of correspondingly skilled modellers in other consultancies; and/or
• The incumbent Term Consultant use more staff (or more senior staff) than necessary to undertake the relevant TMfS tasks

The question of ‘premium’ rates could be checked using rates information provided by the incumbent Term Consultant and other bidders for the TMfS Commission and similar tenders.

Evidence of ‘over-kill’ from the current Term Consultant could be gained from ‘Value for Money’ TMfS feedback from other consultants, though care is needed to take account of inherent bias in these responses. If a significant number of these feedback forms record ‘poor value for money’, then further investigation may be warranted to explore this additional ‘cost’.

If no evidence of ‘overpricing’ is found, then the cost of Term Consultant Staff support can be excluded from the cost side of the cost/benefit ‘see-saw’, since the cost can be assumed to be matched by a corresponding amount of ‘value’ from the professional advice/modelling/analysis provided.

Conclusion

While Methodology M1 does not provide an actual figure or figures for the value of TMfS over the previous year, it provides a framework in which we can understand how investment in it delivers value when compared against alternative ‘Do Nothing’ scenarios.

The valuation technique outlined above is applied to develop the quantitative value of TMfS in Methodology M4.

Methodology M2 - TMfS and the Policy Context

Overview

Chapter 2 identified the need for TMfS to contribute towards meeting the Scottish Government’s central Purpose. With this in mind, we can attain a better understanding of the value of TMfS in the light of the key schemes and policies in which we have assisted. The quantitative assessment of schemes and policies using TMfS allows policymakers to understand their impact on the National Indicators and, as a result, their contribution to the Government’s overall Purpose. While it could be argued that this could be done with a number of bespoke models, it is very unlikely that this could be achieved on a consistent and robust basis.

It is possible that the role of TMfS in helping to deliver the policy context, represents the single largest contributor to the overall value of it. The infrastructure schemes and policies that the model is used to assess are often significant strategic projects and it is vital that a consistent framework is used to assess their impacts. Even small errors in the assessment of such schemes can cost millions of pounds. In addition, without a modelling suite using consistent assumptions, it would not be possible to compare and rank interventions on a like-for-like basis, making the prioritisation of a long-term programme of investment extremely difficult. Without TMfS, it is almost impossible to assess consistently which schemes and policies contribute the most towards delivering the Government’s Purpose.

The role of TMfS in supporting the delivery of policy will only be further amplified by the recent publication of the development planning Common Statement.

Breaking the issue down

The Government’s Purpose is to deliver equitable and sustainable economic growth across Scotland and effective transport policy can be seen as a cornerstone of this strategy. Indeed, the National Transport Strategy and the Transport Scotland Corporate Plan 2008-2011 are focused on delivering this Purpose. It can be argued that transport facilitates economic growth on three levels, as follows:

• National — transport links large centres of population, key economic hubs and individual businesses, encouraging agglomeration and a more competitive and productive labour market
• Regional — regional transport policy delivers schemes and initiatives designed to link economic hubs within that region and facilitate access to jobs and other opportunities
• Local — local transport policy encourages a more vibrant local economy by allowing access to key services and addressing prominent local issues

In light of this, our review of the role of TMfS in contributing towards transport policy during the last year is separated into these categories. It will reflect on the qualitative value of TMfS on legacy applications, such as the STPR and Lothian Development Plan, as well as new, year two applications.

In addition, this section will also consider the role of TMfS in supporting other policy areas, such as planning and environmental policy.

National Policy

TMfS has been involved in a range of significant national projects during 2008, including:

• The Strategic Transport Projects Review
• Replacement Forth Crossing
• FETA Main Cable
• M8 Baillieston — Newhouse

The implementation of these projects is essential if the Government is to deliver sustainable economic growth. The STPR will identify transport schemes and policies integral to creating a vibrant Scotland, while the Forth Crossing is of huge economic importance to Scotland. The role of TMfS in providing robust evidence-based analysis has been an important element of these projects and has delivered notable added value. Indeed, it can be strongly argued that the STPR was highly dependent on TMfS — i.e. it was used to compare a wide range of strategic schemes and its outputs have contributed towards identifying which schemes best support the Government’s Purpose while also meeting a number of local and national transport objectives. Without TMfS, conducting such an analysis would have been extremely problematic. It would have also been very difficult to compare and rank the impact of options on a consistent basis.

Regional Policy

TMfS has also assumed a more prominent role in regional transport policy over the previous year, with involvement in a number of studies, including:

• TACTRAN P&R Study
• TACTRAN Accession™ Model Update
• South Lanarkshire links to the East
• Clyde Waterbus
• Edinburgh Park Rail Halt
• Edinburgh South Orbital Bus

Local Policy

TMfS has clearly made a significant contribution to local transport policy by assessing schemes and policies out with the remit of national policy. The use of the model on the above national and regional schemes has allowed for a consistent assessment of projects designed to enhance economic growth and improve accessibility.

Despite being largely strategic in nature, TMfS has been involved in an increasing number of local studies, either being used in its own right, or providing significant and essential input to more local models. Local schemes that TMfS has contributed to include:

• Edinburgh Park Rail Halt
• Perth Tay Crossing
• Bishopton Inter-Peak Model
• Vale Corridor P&R
• Kilbowie P&R
• Hairmyres P&R
• Glasgow City Centre Paramics

As with regional policy, TMfS has delivered value by being available as a consistent tool for application across a number of schemes. The analysis provided has contributed towards local development by providing evidence and solutions to local constraints on the transport network.

In all, it can be strongly argued that TMfS has delivered significant added value by providing a consistent tool that links each of the three tiers of transport policy. It has also ensured a consistency across projects - using the same model, based on consistent assumptions, meaning that the outputs can be compared with greater confidence.

Planning Policy

TMfS has also provided substantial inputs to other areas of policy, with perhaps the most significant contribution being in the planning sector. Effective planning policy is central to ensuring that the economic potential of new developments is maximised and done so in a sustainable fashion. TMfS has been used to consider the impact of a number of development plans on the strategic road network, including the:

• Lothian Development Plan
• Glasgow City Plan II

The role of TMfS is planning policy will continue to evolve following the publication of the development planning Common Statement.

Environmental Policy

Continuing the theme of sustainability, TMfS has played a key role in environmental policy. As well as contributing towards the environmental assessment of schemes, TMfS has been used in the following applications:

• Scottish Climate Change Programme
• Edinburgh Air Quality

It can be strongly argued that economic growth is only beneficial where it can be encouraged in a sustainable fashion and the role of TMfS in investigating such considerations has added value overall.

Conclusion

It can be seen from the above that the application of TMfS during 2008, has delivered a significant amount of value. The use of a consistent appraisal framework at national, regional and local level has provided more robust outcomes, while the existence of such a tool provides opportunities for linkages with other policy areas.

Methodology M3 - What would be done in the absence of TMfS?

A further methodology for assessing the value of TMfS is to investigate what users of TMfS would have done had the model not existed. The TMfS Model Request Form asks this very question. The answer(s) can be compared with what is being undertaken using this capability - the difference between the two (in a qualitative and/or quantitative sense) being equal to the value of TMfS. While this approach is potentially effective, it must be borne in mind that the outputs would likely be highly qualitative and subject to debate.

Table D2 shows the response to this question for each application conducted during 2008:

Table D2 What would be done in the absence of TMfS?

Application	Model User	What would be done in the absence of TMfS?
STPR	Jacobs Consultancy	Would have developed a bespoke model.
Replacement Forth Crossing	Jacobs Consultancy and Arup	Would have developed a bespoke model.
TACTRAN P&R Study	Colin Buchanan	Would have estimated the potential demand for each site, using Census travel-to-work data. Colin Buchanan noted that this approach is less reliable, as trip patterns cannot reliably be assigned to a highway network and appropriate pass-by matrices would have to be estimated. Travel costs and times would have to be calculated manually, giving rise to further possible errors. TMfS was also favoured by the client as a large amount of recent O-D has been collected in the TACTRAN area and included within the TMfS:05A.
South Lanarkshire bus-based P&R	Faber Maunsell	Exclusive use of Census data.
Edinburgh Park Rail Halt	Faber Maunsell	Would potentially have used a spreadsheet model based on PDFH.
Hairmyres P&R	Faber Maunsell	Exclusive use of Census data.
Perth Tay Crossing	Halcrow	Not specified.
Edinburgh South Orbital Bus	Halcrow	Possible use of Edinburgh VISUM Model or older TRIPS models.
South Lanarkshire Links to the East	Halcrow	Creation of a bespoke model.
Replacement Forth Crossing	Jacobs Consultancy	Would have developed a bespoke model.
Bishopton Inter-Peak Model	JMP	The PM peak sub-area matrices would have been factored accordingly.
Glasgow City Plan II	MVA Consultancy	Would have upgraded Glasgow SATURN Model or requested the use of SITM4.
Kilbowie P&R	MVA Consultancy	Desktop analysis of potential park and ride transfer.
Clyde Waterbus	MVA Consultancy	Would have requested SITM4 or undertaken desktop analysis.
Edinburgh Air Quality	MVA Consultancy	Could have potentially developed a bespoke model.
Vale Corridor P&R	MVA Consultancy	Desktop analysis of potential park and ride transfer.
Lothian Development Plan	MVA Consultancy	Could have potentially developed a bespoke model.
Scottish Climate Change Programme	MVA Consultancy	The Scottish Government would possibly have used a linear projection from Scottish car KM data. This would probably have been coupled with assumptions regarding expected growth (e.g. trend growth to 2025 and flat thereafter).
TACTRAN Accession	MVA Consultancy	Not applicable.
Geography of Poor Skills	Scottish Enterprise	Not applicable.
Glasgow City Centre Paramics	SIAS	Would possibly have used SITM4. Alternatively, CSTM3A could have been used — it was employed on the M74 Completion project and contains information on land-use planning changes from the Glasgow Clyde Valley Structure Plan.

It can be seen from Table D2 that TMfS has played a significant role in terms of providing a consistent framework that can be used across a number of transport studies in Scotland. With the exception of SITM 4, many of the alternative models identified are older than the modelling capability offered by TMfS and also have a more limited geographical scope.

The ‘off-the-shelf’ nature of TMfS has also provided added value, in terms of cost and time savings, where a model user has identified a bespoke model as the next best alternative to TMfS-based support. In addition, there are economies of scale to be obtained from using TMfS, in that the consistent application of the same model allows model users to move down the ‘learning curve’. Users become more proficient in the use of the model and thus are better able to understand how to interpret and apply the outputs. This is particularly important with large-scale applications of TMfS, such as the STPR. This ongoing application support in itself offers value added through skill sharing and the refinement of modelling knowledge, while also streamlining our operations. TMfS also provides a quantitative and more robust evidence portfolio than could be obtained from qualitative desk-based research.

As well as support for modelling applications, TMfS has also provided a return on our investment through developing synergies with other areas — for example, the data collection facility and our Scottish Water liaisons. This has provided us with economies of scope through reducing procurement costs, sharing data collection costs and encouraging quick and more informed decision making.

Ultimately, however, the real value of TMfS can once again be traced back to its unique position as a Scotland-wide appraisal framework, capable of providing robust, evidence-based analysis for policymakers.

TMfS has clearly provided added value for a number of different applications of the model throughout and beyond year two. These substantial benefits should not be disregarded when considering its overall value.

Methodology M4 - The Quantitative Value of TMfS in Year Two

This section attempts to estimate the quantitative value of TMfS modelling support during 2008, while also placing the level of investment within the context for which the modelling capability is being used.

Appendix E provides a quantitative estimate of the benefits of having TMfS available for the applications outlined in Table D2. This estimate is broadly based on the qualitative valuation technique outlined in Methodology M1 above. Clearly, any such estimate is likely to be highly subjective, but it is important to attempt to identify the ‘monetised’ benefits associated with the above model applications. Our methodology for identifying the benefits associated with TMfS on an application by application basis involves estimating the cost of building a bespoke model for each application. Having done this, we add data collection costs, which we consider to be 50% of the cost of model development. The ‘with data collection’ and ‘without data collection’ costs are then multiplied by the level of need for a model to determine the added value of TMfS support in each instance. Appendix E indicates that the added gross value from application support in 2008 is broadly in line with that of 2007.

It can also be seen from Appendix E that the overall cost of TMfS support is very small relative to the cost of the applications it is appraising. This is extremely significant in that the applications that are being scrutinised by TMfS are often substantial strategic projects where even a small error in calculating the cost-benefit ratio can have a significant adverse impact on the decision making process and can result in the wrong investment decisions being made. The value of TMfS outlined in Methodology M2 can be seen in the context of its contribution towards an informed and efficient decision making process, involving national transport projects, policies and programmes that can make an important contribution to Scotland’s future economic performance.

The above analysis clearly demonstrates that the value of TMfS-based support, relative to the total costs of a scheme or policy is very high. Without robust and consistent evidence on which to base decisions, policymakers can make economically inefficient and extremely costly mistakes. This in itself underlines a strong business case for TMfS.