Project control review
An assessment of the current project control arrangements
Introduction
The purpose of this section is to assess the key processes and controls in place over projects managed by the Tripartite. To illustrate how these function in practice and to enable us to assess operating effectiveness, TS has asked us to focus our testing on two projects: Ardrossan Harbour and New Islay Vessel.
Ardrossan Harbour: the Ardrossan harbour services the Arran ferry route. The project aims to upgrade the harbour facilities and to improve effectiveness alongside the new vessel commissioned for the route (MV Glen Sannox). The project is a collaboration between North Ayrshire Council, the harbour owners Peel Port Group (PPG), TS, CMAL and CFL.
New Islay Vessel: the Islay ferry route runs from Kennacraig in Kintyre to Port Ellen and Port Askaig on Islay. The project aims to replace the HM Hebridean which has reached life expiry. The project therefore seeks to procure a new vessel to improve capacity on the routes between Islay and the mainland that meets the strategic aims of reducing inequalities, climate action and economic growth.
Our approach to the controls review
We undertook a detailed review of the existing controls around project management and review, financial and budget management, procurement arrangements, risk management and legal interactions for the selected projects.
We completed a series of interviews with the key senior members of the Tripartite and reviewed supporting documentation in relation to general processes and controls and to both projects, including project plans, budgets, meeting minutes, business cases and terms of reference. We focused on the controls in place at a project level, specifically those currently enacted as part of the projects’ pre-approval stage.
We compared the controls currently in place against expected project management controls, based on guidance from the Scottish Public Finance Manual and the Office of Government Commerce, and standard practice observed elsewhere. Following the interviews and the documentation review, key themes and observations were collated together with the associated risks and potential recommendations. At the time of our review both projects were at the pre-approval stage. Approval for the Outline Business Case (OBC) for the New Islay Vessel project has subsequently been granted by TS’s IDM board, and the project is moving to stage one procurement with a budget for £75.683m. For a full list of interviewees, see Appendix C. For a full list of documents reviewed, see Appendix D.
Our Point of View
Our review of current project operational controls identified two high risk findings, one medium risk finding and one low risk finding. Please see Appendix F for finding classification definitions.
- Our review highlighted multiple instances of good control structures and foundations for future controls once projects have commenced works. For example:
- Documented structure of oversight groups / committees, e.g. Ardrossan’s structure of project manager, Steering Group and Ministerial Task Force.
- A focus on lessons learnt from current and previous projects.
- For Islay there is a well documented project plan, feasibility studies, decisions, actions and action tracking and a risk register.
- When fully approved, projects have timelines and budgets to enable project management such as reviews, KPIs, milestones and escalation.
- However, our findings noted that key foundational elements of a well established operational control environment have not yet been thoroughly established as summarised on the following page. One root cause of the findings is that the projects are not viewed as properly started at the pre-approval stage, although key activities are required to gain approval and costs have been incurred. Good practice is to establish these elements early in a project’s lifecycle to ensure a robust framework and environment in which to operate when the project’s main works are underway, as per the Project Delivery Standards (7.2) in the Scottish Public Finance Manual.
- Following Ardrossan, a programme of improvements has been implemented, which we evidenced with the more recent New Islay Vessel project, which has a project plan with milestones covering the pre-approval stage, documented review with decisions, action tracking and risk awareness. Whilst there is not a budget in place for costs incurred at the pre-approval stage (£560k), spend has remained relatively low compared to the budget for the project once underway of £76.763m.
An assessment of the current operational control arrangements
Summary of Key Findings and Recommendations
We have identified a number of opportunities to develop and embed improved controls across the projects associated with the Tripartite, which could be implemented should the decision be taken to continue with the “as-is” structure.
These findings are set out below and in more detail on the following pages, together with an analysis of the associated risks and recommendations for improvement. We have assessed the risks for each finding to enable TS to prioritise actions. Please see Appendix F for finding classification definitions.
Reference 5.1
Risk – High
Finding – Gaps in early stage project and budget management
Neither Islay nor Ardrossan have a clear audit trail over changes to the pre-approval budgeted figure. Furthermore, Ardrossan has no pre-approval project plan, and so key activities and timelines are not set out. Without a project plan, there is no means to review project progress against key activities and timelines, which may lead to difficulty in assessing required next steps to meet project requirements.
Key Recommendations
Implement overall and detailed project planning and budgeting for all stages of a project, including pre-approval. This should include: timelines, milestones, KPIs and formalised project and budget review documentation.
Reference 5.2
Risk – High
Finding – Roles, responsibilities and interaction
Whilst there is some documentation to support the organisations’ broad roles, the roles and detailed responsibilities of the various organisations related to projects are not well defined, documented at a granular level, and formally agreed. Furthermore, how the organisations interact across these roles, especially where there are interdependent responsibilities between organisations, has not been well established.
Key Recommendations
Formalise and agree the roles and responsibilities of each organisation involved in the project in line with a properly accredited and recognised Project Management Framework. This should include the interaction of these groups and their interdependent responsibilities.
Legally formalise the arrangements between each organisation involved in the project, especially where the project involves non-Tripartite members.
Reference 5.3
Risk – Medium
Findings - Escalation and authority
There is no established definition of when a risk, issue, decision or action requires escalation to a more senior group. This is due to a lack of documented Delegation of Authority to the oversight groups regarding decisions that impact project timelines or budgets. Without well defined limits on authority, there is no prescribed mechanism for escalation, and so timelines and budget impacting decisions may be taken at an inappropriate level.
Key Recommendations
Formalise and document the authority of each oversight group with respect to decisions and actions over risks and issues, including the requirement to escalate to a more senior group.
Consider setting up PMO function to ensure compliance and escalation routes where projects are outwith tolerance levels.
Reference 5.4
Risk – Low
Findings - Procurement alignment
TS has informally reviewed CMAL’s procurement strategy, however TS has no formal periodic mechanism to gain assurance that procurement agents and third parties procure in line with public sector obligations.
Key Recommendations
Establish assurance procedures over procurement strategy, methodology and activity to gain assurance that parties are procuring in line with public sector requirements where part or wholly funded by TS.
5.1 Gaps in early stage project and budget management
Risk Rating – High
Observation
There are gaps and inconsistencies between projects in the approach to project and budget management in the early stages of the project lifecycle.
The Ardrossan project is at the pre-approval stage (i.e. the business case has not yet been signed off) and has a total spend to date of £2.45m, which consists mainly of consulting fees and costs involved in scenario planning. However, there is no clear project plan and no shared Tripartite documentation for Ardrossan which provides an audit trail over updates to the initial pre-approval budget. As such, there is no mechanism to assess whether this spend is in line with the latest expectations, and whether activities required to achieve full approval (for example, full business case production; agreeing legal relationships and roles and responsibilities) are on track.
Clear, up-to-date project plans and budgets are foundational controls which form the basis for an effective project management framework. Without a project plan or up-to-date budget, related controls cannot be properly put in place, such as:
- Review of project progress
- Cost monitoring
- Action planning and tracking
- Key Performance Indicators (KPIs) and milestones
- Processes for the escalation of risks, issues, decisions and actions
In the case of Ardrossan, key activities such as agreeing legal interactions between involved parties have not been completed, despite initial activity commencing seven years ago. This was due to the absence of a clear, shared, mutually understood and agreed project plan at the pre-approval stage to guide next steps and decision making. As a result, the harbour owner no longer participates in the Steering Group, which has now not met for over a year, and there is a lack of clarity around next steps needed to progress.
The Islay ferries project was also at the pre-approval stage, but, demonstrating learnings from the Ardrossan project, there is a high level project plan in place with clear milestones. This has enabled project level review of progress, decision making, action tracking and risk awareness. The project has subsequently progressed through to stage 1 procurement with approval of the OBC.
However, as with Ardrossan, there is no audit trail over changes to the pre-approval budgeted figure for Islay. While costs to date remain low in comparison to Ardrossan, without a budget there remains a risk that costs increase without proper oversight at a project level.
Progress reviews for Islay are performed orally at the Working Group (project oversight committee) meetings and captured in the minutes. Though this fulfils the purpose, a formal structured document would allow for audit trail and focused discussion on the project’s risks, issues, decisions and actions, as well as any requirement to escalate to more senior stakeholders.
Risk
The absence of a project plan and budget covering activities to full approval results in:
- A lack of clarity around key milestones to be achieved
- A lack of clarity over the order actions must be taken in
- An inability to hold the project and its associates to account over delays, and so an inability to push the project forward
- A lack of understanding of potential costs to be incurred before the project works begin
- An inability to monitor and track acceptability of costs incurred
- A decreased ability to properly escalate problems or potential issues, as there is no clarity around timelines or costs
Ultimately, these could hinder the successful delivery of the project, and may lead to a failure by TS to deliver VfM and / or achieve their strategic objectives.
Recommendations
Project Management Standardisation
For all projects agree and implement a standardised project management framework and process (consideration could be given to existing frameworks such as PRINCE2), including standardised controls and documentation around project plans, budgets milestones, KPIs, reviews, roles and responsibilities, delegation of authorities and escalation routes. Input should be obtained as appropriate from all Tripartite members but final approval should sit with TS.
Training should be provided to relevant staff from each entity to promote standardisation and high quality key control activities.
Project Planning
At the outset of all projects create an overall high-level project plan that includes milestones and key actions. From this produce a more detailed project plan for the immediate steps of the project, including a pre-approval project plan.
Project Budgeting
At the outset of all projects, create, agree and gain appropriately senior approval for the budget aligned to the project plan. This should include a budget for immediate project steps, e.g. costs associated with pre-approval activities, and not only activities following on from approval, such as procurement.
Milestones
When agreeing both the overall high-level project plan and the more detailed immediate project plan, agree the key milestones that must be achieved for project success.
KPIs
When agreeing both the overall high-level project plan and the more detailed immediate project plan, agree the KPIs in order to facilitate review of the success and potential issues for the project. There may be more detailed KPIs relating to detailed sections of the project, e.g. for pre-approval, design, procurement, works etc.
Documented Project and Budget Review
Using the project plan and budget, establish formalised documented project and budget review processes that include:
- Expected progress to date vs actual progress to date
- Budget vs actuals
- Issues and risks to project success
- Decision making on actions
- Actions ownership and tracking
- Escalation of issues and risks for more senior decision making
Escalation and Authority
Please see Escalation and Authority for a recommendation to establish delegated authorities and escalation requirements
Roles and Responsibilities
Please see Roles, Responsibilities and Interactions for recommendation of more clearly documenting each party’s roles and responsibilities at each stage of the project.
5.2 Roles and responsibilities and interactions
Risk Rating – High
Observation
Projects for upgrades to ferry routes involve a complex interaction between different parties in terms of funding, vessel ownership and leasing, operation of routes, port ownership and leasing, land ownership and improvements, as well as procurement, project management, governance and budgeting. However, the roles and responsibilities of each party in relation to the projects, and each other, are not well defined and documented.
There are broad roles for oversight groups and the relationship of organisations to those oversight groups for both Ardrossan and Islay in their business cases and Terms of Reference. The roles as defined are in line with our expectations for groups of this nature. There are also high-level roles given in draft meeting minutes from the Ardrossan Steering Group dated 14th May 2020. However, neither the business cases nor the Terms of Reference assign key roles such as project management, procurement, governance, budget management etc, and the detailed responsibilities that align to these roles, nor agreement by those organisations to fulfil them. We do see this level of broad role and detailed responsibilities in Islay’s OBC concerning CMAL’s role of managing the ship building contract; however, we do not have the same for other key areas such as procurement and project management.
The lack of defined and documented roles and responsibilities impacts how the organisations interact, as one organisation’s role may impact heavily on another’s responsibility, e.g. CMAL as vessel procurer impacts on CFL as vessel operators. However, the way these entities should interact, the level of input required from CFL and level of responsibility on CMAL for operational success is not documented. This has previously resulted in delays – for example, where CFL desired additional input on scenario planning for Ardrossan.
The need to define roles, responsibilities and interaction is particularly important for projects with a more complex structure – for example, where the port is owned by a third parties. We noted a recent instance where legal arrangements had not been put in place between all parties, resulting in key parties not attending steering group meetings for an extended period of time.
Risk
Without documented roles and detailed responsibilities for each party to the project as a whole, specific responsibilities may not be clearly allocated to an organisation, and the interaction and interdependent responsibilities of organisations may not be fully understood. There is also a risk that resourcing needs may not be fully identified and understood. The result is that responsibilities may not be fulfilled, and organisations cannot give input to decisions that directly impact their operational success, and so the success of the project.
Recommendations
Roles and Responsibilities
Transport Scotland, in their capacity as sponsor body, should define and document the broad roles, and detailed responsibilities aligned to each organisation at each stage of the project. This should be aligned to a properly accredited and recognised Project Management framework such as PRINCE2 or the Office of Government Commerce, and should define the level of required input and interdependent responsibilities between the organisations and their roles, to ensure that adequate resource will be in place for delivery and oversight of key projects
Legal Agreements
For each project, as the sponsor body TS Ferries should liaise with Transport Scotland’s legal advisors to determine and define the appropriate legal agreements and ensure that these are in place for each project, such as interactions and responsibilities between each party in relation to fulfilling the project. These should consider any pertinent complexities / intricacies inherent in the nature of public sector vs. private sector contracting – for example, the different objectives and approach of privately owned infrastructure operators and Scottish Government’s delivery of lifeline ferry services.
5.3 Escalation and authority
Risk Rating – Medium
Observation
There is no documented policy or Delegation of Authority for the escalation of risks, issues, decisions or actions, and so no clearly prescribed threshold after which decisions must be escalated. Instead, the decision to escalate from the project oversight committees to a more senior forum is determined on an ‘as needed’ basis. In addition, there is no Programme Management Office to hold project sponsors to account.
A lack of clear authority and thresholds is exacerbated where there is no project plan or budget, as there are no clear indicators of when projects are deviating from acceptable timelines or costs, or when more senior interventions may be required or project reapproval may be appropriate. We see the effect of this with the Ardrossan project where timelines and costs have increased, however with clarity over acceptable spend and timely escalation to more senior decision makers, these issues may have been addressed more efficiently and effectively.
We note that routes do exist for operational issues, running from the technical sub groups via the Project Manager and Steering Group up to the Ministerial Task Force. There is also a Delegation of Authority in place for procurement for Islay (although this has not yet been used as procurement has yet to take place). What is currently absent, is clarity over when issues should be escalated.
Risk
Without a clear requirement for escalation, there is a risk of inappropriate decision making or an unapproved increase in costs or timelines, impacting delivery of the project and the achievement of overall strategic objectives.
Recommendations
Escalation and Authority
Produce a Delegation of Authority (DoA) for each level of project oversight that defines their authority to make decisions and take actions over risks and issues, especially deviations to project timelines and budget. The DoA should also stipulate when each level must escalate decisions and actions over risks and issues that will effect either timely delivery of the project or project costs, and overall success of the project.
When developing these, consideration should be given to the sponsor body framework documents and the DoA requirements in the SPFM. Providing a centralised repository (for example, a SharePoint solution) to store such guidance and proactively encouraging their use may improve awareness of the existence of these documents.
Consider setting up PMO function to ensure compliance and escalation routes where projects are outwith tolerance levels.
5.4 Procurement alignment
Risk Rating – Low
Observation
CMAL is responsible for procurement of ferries and related works on behalf of TS who wholly or partly fund the works. CMAL has their own procurement strategy and methodology, which is currently still being revised and approved.
Though TS Ferries Unit have informally reviewed CMAL’s procurement strategy, they do not have an agreed method for periodically gaining assurance that CMAL’s procurement strategy, methodology and activities fulfil TS’s obligations to meet public sector procurement requirements.
Similarly, where a project involves working with third parties and TS wholly or partly funds the third party’s works, TS Ferries Unit could fail to meet their public sector procurement obligations if the third party’s procurement activities do not meet the required standard.
Risk
While CMAL are bound to the same SPFM requirements as TS, there is a risk that TS Ferries Unit could be perceived as not having obtained sufficient assurance over the procurement activities, or that an element of non-compliance goes undetected due to a lack of ongoing / periodic assurance.
Recommendations
Procurement Methodology
TS, in their capacity as both project sponsor and organisation sponsor, should formally review the procurement strategy, methodology and activities of related parties to gain reasonable assurance that CMAL are conducting procurement that they are funding or part funding in accordance with relevant requirements. They could also consider obtaining periodic attestations from CMAL that they continue to comply with the relevant requirements of the SPFM.
< Previous | Contents | Next >