This month, the NHS published the sixth in its series of ‘Bite-sized guides to patient insight’: ‘Writing an effective questionnaire’ and this short video below forms the Institute’s commentary on it.

We welcome the release of this guidance, and, in general would support it. It contains nothing that contradicts our own best-practice in the field and is written in easy-to-understand language.

Our commentary is largely then, about where we feel the guidance might have gone a little further.

Barry’s detailed written comments are written below:

General comments

On the whole, the guidance is good. It certainly doesn’t contradict anything tCI would wish to say, and it makes some valid points – all of which are included in ‘Effective Surveys and Questionnaires’.

The main quibble is one of omission, rather than commission. Experience has shown (particularly with the health sector, where there’s a tendency to see services from the delivery point of view rather than the patient point of view) is that the biggest problems with asking questions is that those asking them have difficulty getting out of ‘their world’ and into that of the patient. This isn’t just about language and jargon (although these are part of it), but about mindset and compartmentalisation. For example, a health professional often sees things in terms of which departments, resources and staff are involved in a patient’s treatment, whereas a patient simply wants to ‘see someone who’ll make them better’, and isn’t necessarily knowledgeable about or interested in the processes by which this occurs.

This is addressed to a certain extent in the section of ‘Testing understanding and clarity’ on page 3, but my view is that an understanding of the way respondents see things shouldn’t just be left to the end (checking a drafted questionnaire with them), but should pervade the process of writing, otherwise there are many opportunities for misunderstandings (connected with the disjuncture between the way patients think and the way health professionals think): context is everything in an answer.

I usually recommend in the ‘Effective Surveys and Questionnaires’ course that, before beginning the writing of a questionnaire, consultors should do some qualitative work – sit down with samples of those people at whom the questionnaire is targeted, and find out how they see the world. This is useful in all sorts of ways, as it ties in with our views on best-practice pre-consultation (engagement with affected groups). It would be helpful to see this as an opening section. Trialling and piloting, once the questionnaire is written, is fine, but nothing can really substitute for getting into the patients’ heads from the get-go.

Detailed comments

Second page: “What do I need to know about the individual …”. This is generally fine, but sometimes it’s worth putting a demographic question next to one that it’s relevant to. So one about gender or sexuality might sit well after a question about whether the respondent had ever felt discriminated against. Then people can see the relevance of the question, rather than it simply sitting in a long list of ‘nosey-parker’ questions at the end.

Third page: “Avoid double-barrelled questions”. A good example, but the guidance trips itself up at the end. It’s good to unpick the three areas (face-to-face, home visits, telephone), but unfortunately the new questions suggested are potentially leading (more time on face-to-face and home visits, but less time on telephone consultations), and sit in contradiction to the next section on leading questions; better for all three to have a sliding scale between ‘too much time’ and ‘too little time’ (with ‘about right’ in the middle), and to keep it neutral.

Fourth page: “Closed. questions”. I’ve written above about engagement before writing a questionnaire in the general comments section, but there’s another reason why this is important – it helps a great deal with writing closed-answer questions. This engagement can provide a big help with understanding the kind of category answers people might give to a question; it’ll allow consultors to see ones they hadn’t thought of, and possibly eliminate ones that seem to be ‘low response’ (these can still be captured under ‘other’, but they may not need a separate category). This is helpful not only in terms of data-gathering, but it shows respondents, when they see the list of possible answers, that the consultor is ‘in their world’

Fifth page: “Open questions”. There’s a word missing in the first arrow-point (“you may still be able to ? numerical data from open response questions …”). In an ideal world, it would be good to spend more time explaining this: expanding on the pitfalls in being too precise about number; explaining that prompts for open questions are valuable (so the type of information needed can be seen by the respondent); making consultors aware of exactly how much in the way of resources analysing qualitative data needs.

Barry Creasy
January 2018