NHS England has recently published an impact story demonstrating the value of case management in complex rehabilitation in the South West. Here, Sally Plumb and Grace Sweeney discuss the importance of evaluation and how the impact story came about:

Ask anyone leading or working on a programme of improvement in health and care to identify the most difficult part of their job, and many will say ‘being able to measure and demonstrate the achievements we’ve made’.

Many programme leads would go on to argue that meaningful evaluation is critical if they are to be assured they are delivering value in all their activities.

That value is harder to evidence in relation to key aspects of improvement work like enabling others, building improvement capability, and making connections across organisations.

Evaluation efforts need to be focused and pragmatic whilst still capturing the real value that the programme adds. As traditional approaches to evidencing impact often don’t demonstrate the full value of improvement work, how do we move beyond simple metrics to give a comprehensive, compelling and truthful account of a programme’s achievements?

One example of where NHS England has worked to do this comes from complex rehabilitation in the South West.

In January 2017, Rosie Yarnall and Sally Plumb, two newly in-post case managers in complex rehabilitation services in the South West Specialised Commissioning team, approached NHS England’s Sustainable Improvement (SI) team for some help in capturing the impact of their work.

A number of patients who are admitted to acute hospital following a severe brain or spinal cord injury require specialist in-patient rehabilitation to treat and manage their complex disabilities.

Over recent years, the demand for these services in the South West had continued to grow, resulting in patient flow pressures into and out of the specialist rehabilitation units. Although the number of beds had increased, problems with flow continued, and there was little real understanding of the root cause of the problem and whether the capacity issues were real or apparent.

Rosie and Sally were appointed to unpick the issues leading to the pressures, tackle the waiting times and address patient flow problems. They did this by working with the region’s acute trusts, the rehabilitation providers and the CCGs, and could then provide robust clinical support and guidance to the South West NHS England commissioners to better understand the challenges in the systems.

Rosie and Sally were grappling with the challenges of demonstrating the meaningful impact alluded to at the start of this piece, as they tried to evidence the difference their newly created roles were making to both services, and patient pathways.

For the past 18 months Impact, Research and Evaluation (IRE) colleagues within the SI team had been working on developing, testing and refining an approach to capturing the real value of improvement work in complex health and care settings.

The resulting Impact Framework had been tested and refined with large transformational programmes within NHS England, and IRE jumped at the chance of now ‘road testing’ the approach with a more discrete front-line improvement project afforded by Rosie and Sally’s appointments.

The Impact Framework borrows from the world of developmental evaluation to go beyond traditional approaches to measurement and evaluation. It works through a step-by-step process from developing a Theory of Change to telling the evidence-based, compelling narrative of the programme impact.

Between January and November 2017, Rosie and Sally developed their ‘theory of change’; set out the data needed to demonstrate their short, medium and longer term outcomes; conducted an evidence review of data collected and produced an interim Impact Story.

Using the Impact Framework, they were able to show that key organisations in the region – including providers and CCGs, are collaborating better to ensure rehabilitation is being delivered in the right place and at the right time. This is benefitting patients by reducing their waiting times to access specialist rehabilitation, and organisations are working together to reduce delays in transfers of care and blockages in the system.

Rosie and Sally were able to evidence the impact they had made over an initial ten month period by using the ‘theory of change’ and an Impact Story in line with the Impact Framework, and their posts have now been extended until December 2018.