About this handbook

This implementation guide is a handbook, part of the Complaints survey toolkit, designed to assist health and social care organisations in implementing the survey with the aim to collect feedback from their complainants using a robust survey tool. The survey can be implemented as a postal survey, an online survey or as both.

Background

The report of the Francis Inquiry into the failings at Mid Staffordshire NHS Foundation Trust was published in February 2013 and placed heavy emphasis on the need, across the NHS, for organisations to recognise the importance of patient complaints and to create robust systems and cultures that were able to deal with and learn from them.

In November 2014 ‘My expectations for raising complaints and concerns’ was published. The report documented the findings from research, conducted by an independent research company with service users, advocacy groups and national bodies. It identified a need for measurement tools that allow health and social care services to measure the experience of those wanting to raise concerns or make a complaint.

This complaints survey will allows organisations to survey complainants in a consistent and systematic way. It provides:

• a means of recording how complainants experience the complaints system and the extent to which the organisation learns from complaints.
• a resource that NHS and social care complaints managers can use to assess effectiveness of their approach to complaints.
• a resource that senior executives can use to scrutinise complaint handling and take steps to improve complaint handling.
• a way for commissioners to hold providers to account locally for how they handle complaints.
• a way to drive improvements in complaint handling and resolution.

The toolkit was originally published in 2016 and an updated version was published in 2024. Details of changes are recorded in the change log, available as an appendix to this document.

Timeline

The survey is intended to be implemented as a rolling survey with a new sample of complainants being drawn once every two weeks. The complainants included in the sample should be those complainants who have received their outcome 4-5 weeks prior to sampling.

All complainants should be notified in their acknowledgement letter and outcome letter about the survey and should be given the information to opt out if they do not want to receive this survey. Therefore we recommend adding the following sentence in your standard acknowledgement letter as well as the outcome letter (please replace the ‘XXX’ with a telephone number):

Post-outcome survey only

Pre- and post- outcome survey

All complainants in your sample will be contacted by post or email. Please note that each complainant should be contacted through one method only, by post with the paper-based questionnaire or by email with a link to an online survey. There will be 2 mailings – an initial mailing followed by one reminder to non-responders with a gap of 2 weeks between the mailings.

Figure 1 below provides an illustration of the survey timetable for a 12 week period.

Once you have decided the start date for fieldwork (week 6 in the timetable below) you should count back 4-5 weeks and sample all complainants who have received their outcome during this two week period, this is Sample A. This is the first sample of people who should be sent a questionnaire.

After two weeks you will need to send all non-responders in Sample A the reminder mailing (see week 8 in the timetable). You will also need to draw your next new sample, Sample B. Again, do this by counting back 4-5 weeks and selecting all complainants who have received their outcome during this two week period, this is Sample B.

Two weeks later (week 10 in the timetable), you will need to send all non-responders in Sample B the reminder mailing. You will also need to draw your next new sample, Sample B.

Again, do this by counting back 4-5 weeks and selecting all complainants who have received their outcome during this two week period (weeks 5 and 6 in the timetable), this is Sample C.

This rolling cycle should be followed throughout the year.

Sampling

You will need to follow the separate document named ‘Sampling Compilation Guide’ to select your sample of complainants who will be sent a survey. If any complainant has indicated that they do not want their records used for secondary purposes (e.g. they have asked to be excluded from all surveys or they do not want their address details being used for any reason other than responding to their complaint), please ensure that these people are excluded from your sample mailing list. This should be checked using your local records.

Materials and practicalities

Questionnaire and covering letters

The questionnaire is included in this Toolkit. The front page of the questionnaire mentions the purpose of the survey and explains how to fill it in.

The front page also has space for a telephone number to be included if a helpline has been put in place. If it is felt necessary to add in two telephone number, e.g. one for questions and one for opting out calls, this can also be inserted.

Complainants (especially those in primary care) should be made aware that the anonymity of their response is not guaranteed, but instead that their responses will remain confidential to the practice manager/complaints department and will not be shared with clinicians or care providers.

For paper questionnaires

On the bottom left side of the front page of the paper questionnaire there is a textbox visible stating URN. Please insert the URN here e.g. ABC1001.

Here you have to insert the Unique Reference Number (URN)* for every record, this can either be done (i) via mail merging** the number on the front page when printing OR (ii) printing a label with the URN for each record and sticking these on the front page individually. Every record will receive one URN which they will keep until the end of the fieldwork.

*This field should be a combination of a 3-character code that is identical for every record (you can use a survey code such as ‘COM’ or if you are an NHS Trust you can use your three-character Department of Health code), followed by a series of 4-digit numbers ascending from 1001 assigned by you e.g. ABC1001. The final 4-character number should be unique for each record.

**For more information on the use of mail merge see: Use mail merge for bulk email, letters, labels, and envelopes – Microsoft Support

The paper questionnaire can be printed in black/white as well as colour. Questionnaires will be sent with the first mailing and with the reminder.

For online questionnaires

The online questionnaire should contain all the questions seen in the paper version of the questionnaire and all those invited to take part should receive an identical online questionnaire. It is also important that the online survey package you choose logs who has completed an online survey so that reminder emails can be sent to non-respondents only.

Covering letters/emails

The covering letters/emails for both mailings are available in this toolkit. You will need to add your organisation logo at the top of the letters as well as completing the missing information, such as adding organisation name, survey link (if using an online version) and signatory details. All the areas where this is required are highlighted.

Organisations are encouraged to carefully consider the subject lines used for email invitations. It is important to make sure that these are easily understood. Possible subject lines include: Your experience of making a complaint, or [Organisation Name] Complaints Survey, or Complaints Survey: Invitation.

The mailings

Please remember that each complainant should be contacted through one method only, by post or by email.

First mailing

The first mailing should be sent to all complainants who received their outcome 4-5 weeks previously. From the Toolkit you will need:

• printed questionnaires, numbered with the URN. The number must match the number on the address label, and the number on the list of complainant details. The Royal National Institute of the Blind recommends the URN be printed in size 14 font and located inside the box on the bottom left side of the front page of the questionnaire
• mailing 1 covering letter or email template – with the relevant fields completed for your own organisation, including your logo/ letterhead

From your own stocks/resources:

• envelopes (size A4 or A5) for mailing survey materials to complainants
• labels for addressing envelopes
• labels for sender address on reverse of envelopes
• FREEPOST envelopes* for return of questionnaires
• hosted online survey (if being offered as an option)

Once the first mailing has been sent, questionnaires will be returned completed. This data needs to be entered in the ‘Data Entry Template’**.

*Please see the section ‘FREEPOST address’.
**This template is included in the toolkit, as well as instructions for entering the data in the ‘data capture instructions’.

Second mailing

The second mailing should be sent approximately 10 working days after the first mailing, to all complainants who do not respond to the first mailing (except, of course, those who withdraw). Usually you will need to send reminders to around 55-75% of the original service user sample. The following items are needed:

From the toolkit:

• printed questionnaires, numbered with the URN. The number must match the number on the address label, and the number on the list of service user details
• mailing 2 covering letter or email template – with the relevant fields completed for your own organisation, including your logo/letterhead
• multi-language helpline sheet (if used in first mailing)

From your own stocks/resources:

• large envelopes for mailing questionnaires to complainants
• labels for addressing envelopes
• labels for sender address on reverse of envelopes
• FREEPOST envelopes for returning questionnaires
• hosted online survey (if being offered as an option)

To know which respondents to send reminder letters, the ‘Data Entry Template’ should be visited. When a questionnaire is returned, blank (which implies an opt out), or completed, the date and URN should be captured in the entry template, together with the survey status as listed in the table below:

The ‘outcome code (survey status)’ (column Q) of this file should be updated fortnightly throughout fieldwork.

When the reminder mailing is due, this column from the ‘Data Entry Template’ is copied in the Outcome Code (survey status) (Column B) of the ‘Mailing File’, as shown in Table 1. Then the ‘outcome’ column can be filtered to ‘blanks’, and these records can be sent the reminder. We recommend you keep this file encrypted. Mailing lists of complainants’ names and addresses should be destroyed when the mailing process is complete.

Table 1 – example mailing file

Sending out paper questionnaires and the reminder mailing

Address labels are needed for each complainant. We recommend using the mail merge feature in a word processing package to create the address labels from your database of service user names and addresses. It is essential that the unique record number (URN) is on each address label, as this has to be matched with the number on the front of the questionnaire. The label should not include any other information except the complainant’s name, address and postcode details, and the unique record number.

Address label format

Remember that you should check your own records for deaths before sending out reminders.

FREEPOST address

A FREEPOST address lets complainants return completed paper surveys at no cost to themselves. After you have paid for the licence, you will only pay for the responses you receive. The FREEPOST address can be printed on the envelopes you send out with the questionnaires. Printed envelopes must comply with Royal Mail guidelines.

Details of how to apply for a FREEPOST licence can be found at the Royal Mail website: www.royalmail.com.

Survey helpline

The covering letter/email to complainants should include a telephone number to call if they have any questions or worries about the survey, or if they would like to opt out. All staff who are likely to take calls should be properly briefed about the details of the survey, and be aware of the questions or worries they are likely to receive. You might want to set up a FREEPHONE line for this purpose.

Upon receiving a call, ask the complainant to tell you their unique record number (URN). This should be (i) on the address label of the envelope they received, and (ii) on the front of the questionnaire itself. You can then use this number to identify people who do not want to receive any further reminders to remove them from further mailings.

You will be required to remove any opt outs from complainants prior to survey mailing dates.

Accessibility

Disability or limited understanding of English could be a barrier for completing the survey and it would therefore be good practice to offer additional support or interpretation service (e.g. completing the survey via a telephone translation service for those with limited understanding of English).

Storing completed questionnaires

The data entered from completed questionnaires must be stored in a separate file to the lists of complainants’ names. If you have carried out an online survey as well as sent paper based questionnaires then the data should be combined in the data entry spreadsheet. Mailing lists of complainants’ names and addresses should be destroyed when the mailing process is complete. Paper questionnaires should be kept for six months after the fieldwork closes.

Data protection and confidentiality

When carrying out the survey, you will need to ensure that you comply with the Data Protection Act 2018, and ensure that all responses are kept confidential. If you have not already done so, please ensure that you add research to your Data Protection Act registration, as one of the purposes for processing personal data supplied by data subjects.

If you are part of an NHS Trust, you will also need to comply with the NHS Code of Practice on Confidentiality (2003), which incorporates the Caldicott principles.

It is important to make sure that any statements you make about complainant anonymity are accurate. In most cases when a survey is carried out by the organisation that the complainant complained to, it is not accurate to claim that complainant responses will be anonymous as the person within the organisation who receives the completed questionnaires is usually able to match these responses to complainant names and addresses.

Where there are likely to be fewer than 10 respondents to the survey, please note that the generalisability of the findings will be limited and should be treated with caution. Small numbers of respondents will also increase the likelihood that respondents can be identified, particularly where responses are broken down by demographic criteria.

It is essential that any survey is conducted in such a way that the complainants’ confidentiality is respected and given a high priority. Complainants’ responses must not be presented to anyone in a way that allows individuals to be identified. Additionally, the Survey Mailing file containing the complainants’ names and addresses should be stored securely (password protected) and the raw data set should not be provided to any member of staff at the organisation who does not need to view it (i.e. those who are not directly working on the project).

It is your legal responsibility to ensure that you meet any guarantees of anonymity or confidentiality made in covering letters and on the questionnaire form. Your organisation’s Caldicott Guardian, Data Protection Officer, information governance board or legal advisors should advise you on these matters*.

*For more information about the Data Protection Act 2018 see: ICO – Information Commissioner’s Office: ico.org.uk/. Further guidance can be found on the Market Research Society website: www.mrs.org.uk/standards/data_protection/

Appendix – change Log

The following changes were made to documents within the survey toolkit.

Short survey

• converted to a pdf form which allows people to answer questions within the pdf, to support online completion
• added form fields to the pdf which allow organisations to add contact details and URN prior to printing
• edited questions 6 and 7 to move the answer “I did not receive any updates” from question 6 to 7. This change is to better capture whether the complainant feels they were updated enough, as opposed to how often they were updated (ie, a person may have received no updates but still feel they were updated sufficiently).
• edited question 17 to “which of the following best describes you?”
• edited question 18 to “Is your gender identity the same as the gender you were registered at birth” – These two changes are to reflect how we capture sex and gender information elsewhere within NHS England
• all changes made in large print version and easy read versions

Long survey

• converted to a pdf form which allows people to answer questions within the pdf
• Added form fields to the pdf which allow organisations to add contact details and URN prior to printing
• edited questions 20 and 21 to move the answer “I did not receive any updates” from question 20 to 21. This change is to better capture whether the complainant feels they were updated enough, as opposed to how often they were updated (ie, a person may have received no updates but still feel they were updated enough).
• edited question 40 to “which of the following best describes you?”
• edited question 41 to “Is your gender identity the same as the gender you were registered at birth” – These two changes are to reflect how we capture sex and gender information elsewhere within NHS England
• all changes made in large print version and easy read versions

Complaints survey toolkit: implementation guide

• added explanation that the toolkit was updated in 2024, signposting to the change log for details of changes
• updated information in the Materials and Practicalities section referring to the URN textbox on the paper survey to reflect reality of survey
• removed reference to multi-language help sheet as this is not available
• updated document version to v.2
• removed reference to need for survey to be hosted by the organisation running the surgery. Organisations should seek internal information governance guidance regarding hosting and data storage
• removed reference to data from completed questionnaires being stored on different computer to list of complainants names. Organisations may be using cloud storage which makes alternative computers irrelevant
• updated references to 1998 data protection act to 2018 data protection act
• updated section relating to language line to cover other accessibility needs such as disability
• removed reference to multi-language leaflet template, as these were not available in originally published toolkit
• Updated broken links in references

Complaints survey toolkit: sampling compilation guide

• changed CCG references to ICB (integrated care boards) to reflect updated NHS system. This includes updating the link to NHS Digital for information regarding ICB codes
• change exclusion list entry from “complainants without a UK postal address” to “complainants without a UK postal or email address”. Updated sampling quality control checklist to reflect this change
• removed “date of birth”, “ethnicity”, and “gender” from sampling, as organisations may not have this information, and equality information will be captured from the survey questions
• updated links to KO41 return information

Complaints survey toolkit: data entry and data cleaning instructions

• Changed CCG references to ICB
• Removed guidance relating to removing name and address information from free text comments
• Removed guidance relating to censoring swearing from free text comments
• Inserted guidance relating to sharing sensitive information (i.e. names, swearing) found in free text comments – The thinking behind the guidance here is that, rather than censoring free text comments when recorded, teams should consider if anything needs to be removed when they are shared.
• Inserted guidance relating to monitoring free text comments for safeguarding issues
• Removed routing guidance for question 6 (short version) and question 20 (long version)
• Updated document from ‘final’ to ‘v.2’

Sample compilation template

• Changed CCG references to ICB
• Removed “date of birth”, “ethnicity”, and “gender” from sampling

Data entry spreadsheet

• Edited response codes for question 6 and 7 (short version) and 20 and 21 (long version) to match changes made to question options