Since this blog was published, certain details have now been superseded (for example, the safeguards for pseudonymous data have been strengthened).  Please read the more recent care.data blogs for further details.

Dr Geraint Lewis, NHS England’s Chief Data Officer, explains why patients can be confident in agreeing to allow their health records to be shared:

Over the course of the next four weeks, every household in England will receive a leaflet, ‘Better Information Means Better Care’.

The leaflet explains how the NHS is upgrading its data systems and what people should do if they have any questions or concerns.

If you believed everything you read in the newspapers about this upgrade, you would think the NHS was either about to give away everyone’s confidential data free of charge or flog it to the highest bidder. Needless to say, we are doing no such thing: to do so would be unethical, illegal, and unconstitutional. But what actually is the plan?

Upgrading our Existing Data Systems

As the OECD reported recently, the NHS has some of the best health information systems in world.  Dating back to the 1980s, we have been collating information about every hospital admission, nationwide. By pulling this information together and then analysing it in de-identified formats, analysts can compare the safety of different NHS hospitals, monitor trends in different diseases and treatments, and use the data to plan new health services.

At the moment, we are missing this type of information for much of the care provided outside hospital. We do not collect it nationally from all GP practices, for example, nor from ambulance trusts or community health services. As a result, we know worryingly little about how all the different parts of the NHS are working together to provide safe, joined-up care for patients. As the Chief Inspector of Hospitals put it, the NHS is currently “flying blind” in this regard.

So the purpose of the care.data project, which has secured support from the BMA and the RCGP, is to address these shortcomings. Building on the successes of our existing hospital episode statistics (HES) system, we will bring together all of this missing information in order to obtain a more rounded and more complete picture of the care being delivered by the health service.

Protecting Privacy

As we’ve been doing for decades with hospital data, information from GP practices and other care settings will only be extracted as a series of codes, not as words and sentences. These codes will then be linked with a patient’s hospital codes using an automated system before being made available in three different data formats (see below). Each format is protected by a different suite of privacy safeguards as specified by the Information Commissioner’s code of practice on anonymisation. For simplicity, I refer to these formats as green, amber and red data, although their technical names are “anonymous or aggregated data”, “pseudonymised data”, and “personal confidential data”, respectively.

Green, Amber, and Red data

Green data are where we will publish the average values for large groups of patients or completely anonymous figures. For example, we might compare Ashford versus Bury in terms of the average time between presenting to a GP with bowel symptoms and having an operation for colon cancer. Green data are published free of charge for all to see. So before publishing green data, we take extra care not to publish information about rare conditions or any combinations of characteristics that might identify individuals from the data.

Amber data are where we remove each patient’s identifiers (their date of birth, postcode, and so on) and replace them with a meaningless pseudonym that bears no relationship to their “real world” identity. Amber data are essential for tracking how individuals interact with the different parts of the NHS and social care over time. For example, using amber data we can see how the NHS cares for cohorts of patients who are admitted repeatedly to hospital but who seldom visit their GP. In theory, a determined analyst could attempt to re-identify individuals within amber data by linking them to other data sets. For this reason, we never publish amber data. Instead, amber data are only made available under a legal contract to approved analysts for approved purposes. The contract stipulates how the data must be stored and protected, and how the data must be destroyed afterwards. Any attempt to re-identify an individual is strictly prohibited and there is a range of criminal and civil penalties for any infringements.

Over the years, many of the most innovative uses of amber hospital data have come from outside organisations, including universities, think-tanks and data analytics companies. We think it would be irresponsible not to make the maximum use of amber data for the benefit of patients. In future, we want charities and small academic units to be able to use amber data for the benefit of patients. Likewise, we think it would be wrong to exclude private companies simply on ideological grounds; instead, the test should be how the company wants to use the data to improve NHS care. And, as Polly Toynbee put it, if “it aids economic growth too, that’s to the good.”

Finally, in a few exceptional circumstances the HSCIC will make red data available where legally required to do so, for example in a public health emergency such as an epidemic. In the future, red data may also be made available to an organisation that has obtained the patient’s explicit consent or has been granted legal approval by the Secretary of State for Health or the Health Research Authority following independent advice from the Confidentiality Advisory Group (CAG).

CAG considers each application in great detail against the legal framework and recommends whether approval should be provided together with any conditions. Applicants for red data would need to demonstrate (i) that the research was in the public interest and for the benefit of the health service; and (ii) that it is not possible to use information that does not identify patients; and (iii) it is not possible to ask patients for their permission.

Patients have a choice

We want to make the most of the information that the NHS already collects. By drawing it together from all parts of the health service, not just hospitals, we will better be able to understand the causes of ill health, learn how to treat patients more efficiently, and find out what happens to patients after they leave hospital.

However, we are giving people a choice. If a patient is happy for their information to be used for these purposes then they do not need to do anything: there are no forms to complete and there is nothing to sign. But if they have any concerns, they can talk to their GP or contact the dedicated patient information line on 0300 456 3531

Ultimately, this is an opportunity for all of us to help the NHS deliver high quality care for all by making the most of the information collected about us.

• More information on the care.data programme is available here .