Innovative methods to analyse the free text data and promote the FFT – a Patient Participation Group initiative

Trust/Practice: Romney House Surgery, Gloucestershire

What did they do?

The PPG realised that just reporting the FFT quantitative data wasn’t improving things. By looking at the free text comments they felt that they could find out what the patients really thought about the surgery, and what we could do to improve their experiences. They came across a technique called ‘card sorting’ as a useful approach for categorising the free text comments into themes and looking at trends. They tell us how this worked:

We gave a unique number to each FFT response and entered it into a spreadsheet along with information about which rating the respondent had given, and the free text verbatim. It became clear that people would often have more than one theme in a sentence, so for each theme a separate entry was made. For example, two separate entries were used for ‘The doctor was very sympathetic, but the receptionist was rude’.

We then decided to sort the entries or ‘cards’ into the ‘Four Cs’, an approach commonly used by Healthwatch to categorise their feedback. These are: Compliment, Comment, Concern and Complaint.

We did some trial runs testing the technique with several people to validate the sorting method. This showed up some confusion about the difference between a complaint and a concern. We decided that a concern was more negative, while a complaint was something more substantial, specific, and temporal in nature. For example, ‘I can see that there aren’t enough GPs’ would be a concern but, ‘The doctor was running an hour late and it is the third time it has happened!’ would be viewed as a complaint. We found that there was a high level of agreement between people doing the sorting, whether they were PPG members, patients, or surgery staff. We counted how many cards we had in each of the ‘Four Cs’ categories.

Now we had a way of turning qualitative data into quantities, but we still needed a way to report this. Pie charts and graphs were fine for one month’s data at a time, but it was hard to see trends over time. Then a colleague suggested using control charts. It was a perfect fit. We could track the number of positive and negative themes over time.

We still needed to be able to act on the information we had gathered, and to do that we did another kind of card sort. We sorted the cards into general themes, then tabulated the results in a spreadsheet. This enabled us to see where improvements could be made. For example, many of the cards were about the amount of time spent in the waiting room. Some were complaints, but some respondents said they didn’t mind, they just wanted to know what was happening. This led to the staff being more proactive in telling the patients when GPs were running late.

When analysing data like this, the PPG found it helpful to have as many pieces of FFT feedback as possible. This meant finding ways to increase the number of patients completing the FFT. This led them to increase their efforts to promote the FFT. One simple way they did this was to leave forms on all the chairs in the waiting room. Another year, at their annual ‘flu day’, the surgery’s social prescriber came in to help direct patients to the various activities including a tombola, the CCG information bus and nurses giving the vaccinations. While the nurse was doing this, she took the opportunity to help and encourage people to complete the FFT.

The PPG also recognised the importance of feeding back to patients and staff. To this end, the PPG did a presentation to staff to let them know about the benefits of handing out FFT forms to their patients. They suspected that GPs and nurses were less likely to do this in case patients said negative things, but the PPG assured them that that they weren’t collecting the data to punish the staff but to help make changes so that the patients would be happier with the service. The PPG also published the results on a blog and the waiting room notice board, including the charts and a ‘You Said, We Did’ list so that the patients could see that their feedback was valued and acted upon.

What did they learn?

Although it might sound like this took a lot of time and effort, once the spreadsheets and control charts were initially set up it took very little time at all. Everyone enjoyed doing the card sorting and being able to visualise trends with the use of control charts was encouraging to both staff and patients. We now had a method to analyse and act on the FFT data which enabled us not only to track how the surgery was doing, but also to make improvements to patient experience.