Impact evaluation – Did the audits make a difference?
Understanding the delivery of recommendations
Initial assessments conducted through Google Street View were verified with local authority interview participants to assess the delivery of recommendations. The impact of the CSA/SRAs on delivery was low, with approximately 28 of 732 community identified recommendations delivered (4%). Only four from sixteen CSA/SRAs resulted in recommendations such as drop kerbs, new footways and new crossings being built. One project also developed a dog fouling communication toolkit which became a citywide programme and is ongoing today. A fifth project contributed to a blanket 20 mph across the village, which may have been going to occur even without the supporting evidence of the audit. The spending levels were humble, with one local authority estimating a spend of £120k in one location, with some additional resurfacing works funded by the local NHS trust. A second local authority estimated £75k over three locations for minor measures.
Local authorities who had delivered measures shared their experience with co-funding measures. Two local authorities who had delivered measures recognised that it made sense to coordinate delivering the recommendations through other works being undertaken. This involved reaching out and coordinating with other teams already undertaking works, such as maintenance teams. It was also key to reach out to wider partners, such as the local NHS trust and community council as they have access budgets that could fund measures that cannot be funded through active travel budgets. A key difference in the local authorities who did deliver drop kerbs is that they treated drop kerb improvements as a material change that facilitates active travel and not as a maintenance issue:
We were quite fortunately in that there was a major resurfacing project going on through the middle of the high street just shortly after this audit. Coz we just managed to get stuffed tacked on..."
[Upgrading drop kerbs] was a full reconstruction… and they were made wider in most cases."
Several local authorities noted that egregious issues like a pothole in the footway could be treated as a complaint and thus be actioned more quickly. Some interview participants expressed that highlighting the three most urgent issues in the executive summary would be a useful improvement to the audit reports.
Understanding non-delivery of recommendations
Most local authorities found it challenging to deliver the community identified recommendations as they are small and sundry and fall across multiple services. Interview participants identified recommendations as falling across active travel, road safety, road maintenance, operations, economic regeneration, and/or community development services. The interviews demonstrated that each local authority is slightly different in structure, with different routes to how the community identified recommendations could be delivered.
Delivery of the audit recommendations would be contingent on a contact person who can distribute and coordinate these across services and follow-up on delivery. Some interview participants raised that as the CSA/SRAs have strong placemaking characters, they almost fit better with economic regeneration teams. Officers recognised the challenges of working across services but also expressed a level of disengagement with interdisciplinary working, which may be a pragmatic reality:
If I picked up a street audit that said, 'we need enforcement of A-boards' I would just skip that point entirely because it’s not relevant to my service."
It's an ongoing battle to do better joined-up thinking."
Many of the recommendations including the two most frequently identified recommendations such as footway repairs and drop kerbs were viewed by many as maintenance issues. Maintenance teams were understood to have a separate process of inspections and broadly viewed by interview participants as being non-receptive to both community audits and an inclusion perspective. One participant related that addressing the poor state of repair of the footways was also problematic because footways were in such poor repair across the area that there would be no end to the volume of requests that one repair could result in:
We can’t fund maintenance. Which is why some of them, like [recommendation] number three, I couldn’t bring that forward because it was poor footpath condition."
It’s very difficult. We can send it to maintenance, but actually having the ability to get them to do it is a different story."
Only so many times you can email Roads to say, “that drop kerb is wrong."
Fundamentally, most local authorities did not immediately consider minor walking measures a strategic priority. Many local authority partners did not see the CSA/SRAs relating to a route. From this perspective, the community identified recommendations did not qualify for active travel funding, which emphasises a journey from A to B. There was a mixed sense that there is no available funding for minor measures and that getting things done through maintenance is difficult. This sense of difficulty was tempered by self-awareness of an unconscious bias that walking is not as important as cycling (or driving):
Things like repairing a footway or so, I can’t really see us trying to bid for funding for those."
We become a bit blinded by the cycling stuff at times. Sometimes we just basically need to encourage people to walk. We always seem to forget that."
When you say a walking project, it almost doesn’t have the same credibility sometimes."
Prioritisation as a pathway to funding and delivery
Three local authorities emphasised the high demand for projects and that they constantly balance this against available revenue and capital resources. There was also a sense that sometimes doing the audits creates an unrealistic sense of community expectation, which then fuels a hostile attitude towards the local authority:
We get people coming to us with ideas for schemes all the time."
We get inundated with all these requests; I think they think we are going to action it straight away."
Sometimes you just get a bit of a clash when you go out. And the council… we’re the bad boys and we should fix this for them
All eight local authorities interviewed raised the importance of integrating the CSA/SRAs into their internal prioritisation processes. This integration could include earlier collaboration with Living Streets to assess where and when the audit may take place. Different services may have different priorities, which could hinder project delivery, but equally may offer opportunities, as raised in the discussion around co-funding. There was a sense of an invitation that local authority partners would welcome Living Streets playing a more proactive role in both making the case for prioritisation and in managing the communities' expectations in terms of how long it will take and realistically, if at all, for the recommendations to be delivered.
Most local authority interview participants identified Cycling Walking Safer Routes (CWSR) as the most appropriate funding source for the measures identified in the community recommendations. However, one local authority said they had allocated 100% of their current CWSR to match funding a large project, so they had no budget for small projects. Five out of eight local authorities shared that no maintenance budgets were available for these minor measures as road budgets are already overstretched. Differently, one local authority interview participant shared that their active travel team now routinely picks up these types of minor works projects and their team views this as a good way to spend active travel funding. This was a local authority who had delivered around 50% of the community identified recommendations in their CSA. For most interview participants, there was a sense of a lack of clear funding route for minor walking measures in scale with demand:
When we come on to why haven’t we delivered, well one is the staff resources, and two it’s the level of funding. Coz as much as I’m saying [we’re] feeling good we’ve put £10k into this borough, that doesnae provide an awful lot of drop kerbs at the cost of a drop kerb."
1.4 Improving partnership working
About half of the interview participants expressed a desire for greater partnership working with Living Streets with earlier collaboration on selecting the audit locations, on making the case for prioritisation as described above, coordinating with other existing programmes of works, and on multi-year follow-up after the audits to support collaboration across departments and increase delivery success. The CSA/SRAs were understood as a starting point of a process that can take several years. There was also a sense that Living Streets could help moderate community expectations about the time necessary and sustain interest over this period (as described above, potentially enable investment of some measure through other funding routes). Living Streets could assist local authorities in building up delivery routines and joined up working across services fundamentally taking on a project leadership role. Equally, with multi-year involvement Living Streets could then directly track whether recommendations have been delivered:
The audit is the easy part."
If these aren’t being followed up there is a risk that they might slowly fall down the pile."
Vegetation. We reported that to the area roads, I have no idea if they actually cut it back."
What it also does is provide that level of communication to the local people who have taken time out of their day voluntarily… for us to do nothing about it isn’t great."
Impact beyond delivery
The interviews with the local authority partners highlighted that the CSA/SRAs had had an impact beyond the actual delivery of the community identified recommendations. Local authority partners valued the CSA/SRA process for community engagement, the focus on inclusion, and the independent viewpoint of the audits. Participants appreciated that the audits represent the community perspective and saw how this could help constructively challenge some of their assumptions. This self-reflection was balanced by a desire for greater collaboration, as mentioned previously. For example, by more clearly articulating the recommendations as community identified and allowing space for the local authority partner to propose solutions in response and benefit from the audit process and inclusion insights:
When we read through them as engineers some of the things they were suggesting we thought, well that won’t work. But it’s ideas…"
I enjoyed the process I have to say. What I enjoyed about it was the schools were much more engaged doing it, and the community was much more engaged than they had ever been with anything we had done with them before."
That’s where an independent person is really good. I am receptive to that because yeah you sometimes you get your practical council head on and you can become a little bit stubborn."
Some interview participants shared that they valued the impact the engagement had in increasing officers’ understanding of the experience of Disabled people, older people, and children. One participant expressed a desire for a rolling programme of engagement with officers, including officers in maintenance and other services, to keep integrating this inclusive perspective into their day-to-day operations. Another participant emphasised the importance of including key funding gatekeepers in the audit walkabout so that they could directly share this experience and make the case for funding the recommendations:
It was good in that way too, just realising how difficult it can be for some residents to do what you take for granted."
[It] gave us as engineers an insight of someone in a wheelchair… [and] gave us a great insight into the daily challenges they face."
Interview participants from a local authority that had been proactive in delivering minor measures, such as drop kerbs, felt that delivering a successful project had built a reference to inspire both internal team members and wider communities.
Success breeds success. But I think when people see something positive happen they say, 'that’s really good, why don’t you come and do that in my area'"
From our team’s perspective, this project has bred probably bigger willingness to look at stuff like this more."
Measuring impact in walking data
Only one local authority participating in an interview described having several pedestrian counters across their city but stated that this did not provide consistent data coverage. They were uncertain who was managing this data. Four other local authorities related having a small number of pedestrian and cycle counters, but these were mostly sited on multi-user paths. The interview participants indicated there was a bias towards collecting cycling data on these counters, except for one participant who stated that the counter revealed that there was a surprisingly high number of pedestrians using a local section of the National Cycling Network. Two interview participants indicated they thought collecting walking data in town centres was challenging due to the potential of double counting people who are loitering rather than travelling through. In two locations, the interview participant thought that the economic development teams might be collecting town and city centre walking data as this is of material economic interest. Although interview participants described in essence a lack of use of walking data for either prioritisation or impact evaluation, the topic was met with interest as an area of emerging technological innovation.
Impact from the community partner perspective
The flash survey received a total of 62 responses from twelve different audit locations; from three locations only one person responded so the survey is not fully representative of all CSA/SRA locations. Eight respondents (13%) were based in two locations where five or more of the CSA/SRA recommendations have been delivered. Fourteen respondents (23%) had participated in the original audit. Given the timelapse of four to nine years the since the CSA/SRAs were conducted, this was viewed as a reasonable response rate especially as the audit routes are key local routes which still have the same high amenity value and relevance to daily living.
The flash survey indicated differences in experience from CSA/SRA locations which have had five or more of the community recommendations delivered and compared to those which have not. Settlements with delivery estimated higher levels of delivery with 60% of recommendations delivered compared to 30% in locations of non-delivery. In both cases these estimations are substantially higher than the evaluation estimate based on Google Street View and the local authority interviews. Respondents from settlements with delivery had lower desire for further walking minor measures with 50% responding yes, compared to 87% of respondents from settlements without delivery. Overall, this indicates that delivery of community identified measures has had an impact, but that there is still demand for more. This can be understandable form the perspective that even the two locations with the highest delivery levels still only delivered between 25% and 50% of the community identified recommendations.
There were also differences in perceived safety with 63% of survey respondents from settlements with delivery agreeing they feel less likely to slip or trip on the audit route compared to 30% of respondents from non-delivery settlements. Similarly, 50% of survey respondents from settlements with delivery agreed they feel safer from cars and other vehicles compared to 28% of non-delivery locations. As such, the flash survey indicates that the CSA/SRAs did have an impact from a community perspective when they were actioned. The inconsistency of evidence of impact from location without delivery may be a result of other local improvements incidental to the CSA/SRAs, or in one location a result of the blanket 20 mph introduced. With or without partial delivery of the community identified recommendations, the survey suggests strong on-going demand for minor walking measures with 82% of all respondents saying they would like more minor walking measures in their local community.
| All responses | Responses from non-delivery CSA/SRAs | Responses from CSA/SRAs with delivery* | |
|---|---|---|---|
| Participated in original audit process | 23% | 22% | 25% |
| Estimated percentage of recommendations delivered | 30% | 30% | 60% |
| Desire more minor measures along the audit route | 82% | 87% | 50% |
| Feel less likely to slip or trip on the audit route as a pedestrian | 34% | 30% | 63% |
| Feel safer from cars and other vehicles on the audit route as a pedestrian | 31% | 28% | 50% |
* Defined as five or more recommendations being delivered
While this survey is a flash survey and intended to generate quick insights only, the survey findings do indicate that there were greater changes in perceived safety in settlements where minor measures had been delivered. This indicates that changes in perceived safety can be measured simply, albeit with the limitation that it might not be practicable or possible to identify wider influences which may have influenced perceptions of pedestrian safety. Given that the Road Safety Framework 2030 follows a systems-thinking approach – meaning, it is not just one thing that delivers safety – this seems acceptable, as ultimately it will be all measures working together which really delivers on road safety for pedestrians.