The charts used in MOSS are based on analysis of outcome measures agreed by experts including clinicians, statisticians and service users as part of a stakeholder group convened by NHS England.

These include term stillbirths and term neonatal deaths up to 28 days (with a third measure being added in future: Term Grade 2 or 3 Hypoxic Ischaemic Encephalopathy [HIE]).

These were chosen as they have a high potential of causation from care and service delivery issues (for example, avoidable harm, sub-optimal care); they have a low index of causation from known clinical conditions; and they are defined and recorded in a consistent and standardised way in clinical information systems.

In MOSS, the chart used to provide signals is the Maternity Outcomes Signal Chart which is based on cumulative sum analysis (CUSUM). The excess events chart provides further context.

Further information on the methodologies used to produce each chart is available below.

Maternity Outcomes Signal (CUSUM)

This cumulative sum (CUSUM) chart plots a statistic based on the cumulative variation between the number of events over time and the expected number of events based on the national reference rate.

There are 2 thresholds which can generate signals, which have been calculated based on a doubling of observed events in each trust compared to the national reference number of events (all events in England divided by all births).

If a signal is plotted above level 1 (amber dot) it means that statistically, there is a 95% level of confidence that the signal has not occurred by chance.

If a signal is plotted above level 2 (red dot) it means that there is a 99% level of confidence that the signal has not occurred by chance.

Excess events

This chart (a variable life adjusted display) shows the cumulative number of excess events over time compared to the national reference rate.

The national reference rate is calculated by taking the total number of events and dividing by the total number of births.

If the value on the chart is >0, this means that there have been more events than expected compared to the national reference rate.

If the value on the chart is <0, this means that there have been less events than expected compared to the national reference rate.

Signals on the chart correspond with those generated in the ‘Maternity Outcomes Signal (CUSUM) chart’, as explained below.

Statistical methodology

CUSUM methodology has been taken from Hawkins, D. M. and Olwell, D. H. (1998) ‘Cumulative Sum Charts and Charting for Quality Improvement’, and is applied to the data through the CUSUMdesign R package.

First, the national reference rate of events is calculated. To get the expected number of events in each trust, the national rate is applied to the average number of births in the trust for the time period.

This gives the expected number of events in the trust if it had the same rate of events as the national rate.

A ‘high’ number of events is defined as a doubling of this number and used to calculate the thresholds. The value of the CUSUM at time t is given by the formula below:

CUSUM(t) = CUSUM(t-1) + Number of events(t) – k

Where K (the reference value) is defined by the formula:

K = (μ1 – μ0) / (ln(μ1) – ln(μ0)).

Where μ0 = expected number of events and μ1 = twice the expected number of events (2*μ0).

The thresholds (H) are calculated using the R function getH() which uses Markov Chains to calculate all of the possible values the CUSUM statistic could take.

For these calculations K is rounded to a multiple of 0.01 in the first instance. If this fails to produce H due to computational complexity, K is rounded to multiples of 0.1, 0.2 and 0.5 until a value is successfully calculated.

The Average Run Length (ARL) is used to determine the sensitivity of the thresholds changing rates. The ARL applied to the local threshold is 20, equivalent to a false positive rate of 5% (95% confidence that the events or crossing of the threshold is not due to chance alone).

The ARL applied to the national threshold is 100, equivalent to a false positive rate of 1% (99% confidence that the doubling of events or crossing of the threshold is not due to chance alone).

Data sources

MOSS uses data from the Personal Demographics Service (PDS), which includes data from the birth notification process.

Birth Notifications data are submitted to the NHS SPINE by providers on the birth of a baby.

These data are used to create an NHS number for the baby, and to provide a notification of the birth to the local registries via ONS and General Register Office (GRO) ahead of the official birth registration.

These data are used internally within NHS England for analytical purposes, linkage to other data and dissemination.

The raw data fields in the PDS source data are shown below:

Publication reference: PRN02098_v