Background

The People Promise Exemplar Programme aimed to test the assumption that delivering a set (‘bundle’) of interventions covering several elements of the NHS People Promise together would maximise efforts to improve staff experience and retention.

The programme focused on an improvement methodology approach and identified 28 potential interventions. The first cohort of the exemplar programme comprised 23 NHS trusts: 19 of these were acute trusts or acute and community trusts, the other 4 being community trusts and community and mental health trusts.

The People Promise contains 7 key themes, developed with those who work in the NHS in different healthcare roles and organisations to make the greatest difference in improving the experience of NHS staff in the workplace.

Evaluation

The end of programme evaluation comprised:

• a process evaluation covering the early implementation the programme
• analysis of leaver rates using a quasi-experimental method called ‘difference in differences’ (DID), conducted by NHS England analysts
• a review of staff survey data for exemplar organisations
• a detailed survey conducted with trust people promise managers at the end of the programme
• review as appropriate of other workforce data for exemplars sites

Quasi-experimental methods are non-randomised, pre-post studies, designed to understand the causal impact of an intervention, where it is not possible to conduct methods such as randomised controlled trials.

The end of programme evaluation used the data and information listed above to conduct a contribution analysis focusing on the range of interventions and bundling of activity.

To support a systematic and focused approach to the contribution analysis, we looked more closely at those organisations with more significant improvements in leaver rates at 16 months, based on principles of success case evaluation methodology.

Data and findings have been considered at aggregated national and organisational levels, to inform overall findings and insights, and to highlight emerging trends and patterns in the available data.

We used the DID analysis results to identify the organisations which demonstrated the most significant improvements in their leaver rates during the analysis period.

While it is important to note improvements across almost all organisations, this analysis allowed a targeted assessment of those showing the most notable differences in their leaver rates to help identify potential important success factors.

In a national programme such as this, there are significant volumes of potential data to review and inevitably there are varied results overall regarding apparent correlations between these wide data sources.

This approach has allowed us to focus in on areas for detailed review and data triangulation that are likely to help understand the factors contributing to positive change.

The contribution analysis conducted within the success case method approach has allowed us to identify some key findings and important themes based on generalised or overall correlations between data sets and the qualitative findings, within a complex programme delivered in a complex environment.

Analysis of leaver rates

A DID analysis was undertaken by NHS England analysts. The aim of this was to estimate the impact of the programme on monthly all staff leaver rate at the 23 exemplar organisations.

The impact of the programme was analysed starting from a baseline of July 2022. 20 months of data following programme start have been analysed (August 2022 to March 2024).

A regression analysis was undertaken to determine the main measurable factors that influence the leaver rate of NHS staff (including local unemployment rate, staff health and wellbeing indicators, workforce structure, and agency usage). These were used as control factors in the DID analysis.

To estimate the programme’s impact on the leaver rate of exemplar trusts, the sample was split into two groups: treatment group (exemplar organisations) and control group (non-exemplars).

The effect of the programme (‘treatment’) was estimated by comparing outcomes in the treatment group to the control group, with data from before and after programme implementation. The DID analysis revealed that when controlling for other variables, on average the all-staff leaver rate of exemplar trusts fell by 11.8% more than that of non-exemplar trusts in the 20-month period following programme start. This is equivalent to a reduction of 4,465 full-time equivalent (FTE) leavers over the 20-month period.

22 out of 23 exemplar trusts had a significant decrease in their all-staff leaver rates compared to non-exemplar trusts after 20 months of the programme. The analysis also identified the relative improvement at trust level with a range of -22% to -2% across the 22 organisations.

All 23 trusts had a greater decrease in their leaver rate than non-exemplars at 16 months. However, one organisation subsequently undertook a transfer of undertakings (protection of employment) [TUPE] transfer of a proportion of staff out of NHS employment, impacting the result at 20 months.

Staff survey results

Staff survey data provides key insights on staff engagement and offered the opportunity to triangulate People Promise theme level bundles of interventions with corresponding staff survey theme scores.

Based on analysis of the acute and acute and community trusts, average staff survey scores for staff engagement and morale themes improved more in 2023 for exemplar organisations compared with the average improvement for the other organisations in the benchmark group.

The staff engagement theme is often used to identify improvement due to the strength of the index, and its correlation with wider outcomes, as illustrated in Figure 2.

(Please note: analysis was only undertaken for the 19 acute trusts and acute and community trusts. There was not a sufficient sample size to undertake the same analysis for the 4 community trusts and community and mental health trusts).

Table 1: Comparison of exemplar staff survey scores against benchmark (acute trusts and acute and community trusts)

 We further analysed staff survey data by theme score for those organisations which had undertaken more than one intervention and had also achieved the most improved leaver rates.

Overall, each of these organisations improved its 2023 score for the relevant theme score in comparison with 2022, and in most cases the improvement was greater than the relative improvement in the benchmark score for each People Promise area highlighted.

Staff survey data was reviewed based on bundles of interventions for a wider group of organisations, comprising those 10 organisations which had most improved their leaver rates at 16 months.

The overall findings were consistent with those for the initial most-improved cohort. There were 2 People Promise themes highlighted where there were notable overall improvements across almost all the organisations that focused on these areas: ‘We work flexibly’ and ‘We are a team’.

Analysis of staff survey theme scores and sub-scores indicates a potential relationship between the People Promise intervention bundles and improvements in staff survey responses (with some exceptions that reflect programme, organisational and system complexity).

Generally, sub-scores for the ‘thinking of leaving’ theme have improved for those organisations with most significant decreases in their leaver rate, which – alongside changes in relevant theme scores – indicates the potential effectiveness for the bundle approach and the programme overall.

Return on investment

A key benefit of the programme is in staff retained in the NHS. Alongside this, we have considered some high-level indicators of potential return on investment for the programme. We have considered some key costs that NHS trusts face and the potential improvements in these since the start of the programme in 2022.

Modelling return on investment

We have hypothesised a potential return on investment for exemplar sites.

Investment

• the direct costs for the programme are £3.1 million
• this does not include additional, less tangible costs, such as trust senior leader time allocated within their existing role

Assumptions

We have based the high-level analysis in modelling of the costs of staff replacement used by a research team conducting economic analysis for a report of the International Public Policy Observatory*

• for each vacancy there would be a minimum 6 month period for the post to be filled
• agency cover would be used for the full 6 months
• the extra cost of employing agency staff is £54 per day (agency cost of £209 per day minus average salary cost of £155 per day)
• this analysis purely includes potential backfill costs via agency staff and therefore does not include additional costs of recruitment and onboarding

Based on the costs and assumptions outlined above, 442 staff members would need to have been retained in their organisation to cover the investment in the programme. The DID analysis has indicated that we have retained the equivalent of 4,465 FTE staff members, over 10 times more staff retained than would be potentially required, indicating a positive return on investment.

There may be further benefits that could be assessed through a future economic evaluation, such as for staff productivity, including the retention of more experienced staff.

*Daniels, K., Connolly, S., Woodard, R., van Stolk C., Patey, J., Fong, K., France, R., Vigurs, C., Herd, M. (2022). NHS staff wellbeing: Why investing in organisational and management practices makes business sense – A rapid evidence review and economic analysis. London: EPPI Centre, UCL Social Research Institute, University College London. IPPO NHS Staff Wellbeing report_LO131023.pdf (ioe.ac.uk)

High level analysis of agency costs shows that they are decreasing at a greater rate for exemplar organisations, compared with other NHS trusts.

The indicative analysis and high-level modelling undertaken indicates that the programme is very likely to have achieved a positive return on investment.

Key learning: selection and bundling of interventions

All organisations were given a set of 28 interventions covering the 7 People Promise areas, supported by activities led by relevant NHS England subject matter experts. A full list of the 28 interventions is provided in the Annex.

With some exceptions, most organisations focused on a range of interventions and more than one intervention in each People Promise area; this range is illustrated in Figure 3.

In general, those which undertook more interventions are represented in those with most improved leaver rates and those that undertook fewer with smaller improvements.

Table 2: Number of trusts implementing interventions by People Promise theme

Exemplar organisations told us they had identified key priorities and some rationale for their selection of interventions.

Those organisations with most improved leaver rates were acute trusts and acute and community trusts, while those with the least were generally community trusts and community and mental health trusts.

This may be related to sector-level differences in leaver rates, with those for community and mental health trusts being higher historically. But some general differences in approach, including to the intervention bundle, have also been identified between these 2 groups: for example, the community trusts and community and mental health trusts reported introducing a lower number of interventions than most acute trusts and acute and community trusts.

Organisations also undertook activities that they understood to be important based on their own needs.

Flexible working interventions were particularly likely to be implemented as part of the programme. This likely relates to the relative newness of such programmes and initiatives compared with other areas, such as equality and diversity and health and wellbeing which have been a focus for most organisations for several years.

Many organisations told us that some other interventions had been implemented prior to the programme. Where such existing initiatives were in place the programme encouraged their continuation, particularly where these had been identified as addressing specific organisational and staff needs.

We have learnt that where some interventions were already in place, there is value in aligning new and existing initiatives as a coherent programme.

There were several common themes relating to the interventions undertaken by those that achieved more significant improvements in leaver rates:

• pensions and flexible retirement
• efficient use of e-rostering supporting flexible working
• local listening sessions

A key theme was a focus on listening to staff. Those organisations which improved leaver rates the most had a common theme around a listening strategy. While an intervention itself, a listening and engagement strategy will also have helped inform decisions on implementation and communication of other People Promise interventions within the bundle.

Most organisations also focused on health and wellbeing, equality diversity and inclusion, and workforce disability and race equality standards.

Decisions regarding implementation of the interventions at local level were key, and these were often informed by data and analysis of organisational needs and priorities, reflecting a strategic and informed approach to addressing challenges; this is likely an important factor in success.

As well as the relevance of the bundle approach, which includes a range of interventions, the process, including understanding and application of existing data to inform intervention planning, is a key element in programme success.

In many instances organisations considered their existing interventions and practices, and reviewed strategies and policies as part of their self-assessment and action planning. This reflective practice and use of improvement methodology could drive organisations’ intentions to implement and sustain change in embedding the People Promise.

Key learning: success factors in programme delivery

All organisations described a range of communication, engagement and awareness raising activities undertaken in association with the programme. The feedback highlighted the importance of communication with staff and with other organisational managers and leaders, and the exemplar trusts understood this and had taken action to influence these factors.

Some trusts reported efforts to streamline communications and align branding where themes other than the People Promise were already in use.

A range of communications methods were used that emphasise integration into existing communication channels, use of digital platforms, alignment with organisational priorities, engagement through roadshows and face-to-face interactions, creative use of visuals and branding, and tailored communication for diverse workforce demographics.

Triangulation of findings highlighted the importance of communication and awareness for sustainability. Our data shows that higher awareness of the People Promise correlates with better articulated sustainability plans.

This may be due to the people promise managers ensuring that the People Promise is embedded in existing teams and, retention initiatives, and is becoming business as usual.

The people promise manager role has been crucial to engaging with leaders, teams and staff to promote the People Promise, improving awareness and embedding the programme into organisations’ wider work.

Examples included attending and talking at trust-wide meetings, promoting People Promise images and posters, attending inductions and welcome events to speak to new staff, attending events like open days and celebration days and delivering bespoke People Promise awareness sessions for different staff groups.

The importance of evaluation for identifying and sharing intervention success was highlighted. Trusts conducting internal evaluations often reported more successful interventions.

This could be due to using internal evaluation findings to understand improvement needs and learning in line with improvement methodology, as well as simply providing better opportunities to demonstrate success.

More than half of organisations reported undertaking some kind of internal evaluation for their local programmes, utilising various methods and metrics to measure their effectiveness. This included both quantitative and qualitative evaluation approaches.

Key learning: sustainability and future intentions

Half of exemplar organisations intended to continue with the People Promise role, as an extension of the current role, making the existing people promise manager role permanent or recruiting for a new role.

To help sustain the local exemplar programme and achievements, other organisations have embedded the exemplar programme in business as usual by incorporating it in their on-going people programmes or aligning it to the delivery of the Long Term Workforce Plan.

Many organisations had also identified opportunities to allocate additional funding to the programme, supporting its sustainability.

Most exemplar organisations had considered sustainability and future work, including embedding the interventions and maintaining positive impacts, beyond the immediate end of the formal programme.

Annex: schedule of all People Promise interventions

Publication reference: PRN01642