Consultation Data – Quality or Quantity? – Getting the right balance between qualitative and quantitative methods
Listening effectively to the public, through professionally-organised consultations is a challenge. If decision-makers are to do better than just respond through knee-jerk reactions to opinion-poll data, care is needed to select consultation methods that provide the most useful and meaningful insight into the issues.
There is no shortage of methods. Some techniques are quantitative in that they are based upon understanding how many people or groups believe something, and others are qualitative. By this we mean ways of appreciating what they mean, and the dynamics of a debate.
But what is the relationship between quantitative and qualitative methods? Is it a choice of one or another? Or can they be used in combination?
The answer is that the best forms of consultation will almost always involve the judicious use of both types of techniques. A few examples may explain:-
- A straightforward consultation to test residents’ views on a proposal to build, say, a new shopping precinct – could be done using a questionnaire. But more may be learnt if some residents are able to participate in a focus group so that hidden concerns and issues not easily highlighted in the survey can be discussed and considered.
- A public meeting called to discuss the implications of a Healthy Authority’s plans to rationalise services often attracts vociferous objectors. Only by supplementing such forms of consultation with a quantitative survey is it possible to judge how representative such views may be.
- Citizen Panels can provide Councils and others with great opportunities to test opinions at regular intervals, and are useful, for example in measuring satisfaction levels with public services. But even the best tracking surveys tell us little about why people change their minds, and report a higher or lower rating for the same services over time.
One approach is the sandwich-method. This involves using qualitative techniques to establish the agenda, to obtain a stakeholder-view of the issues, and even to help draft the questions. In the middle comes a quantitative survey, designed to test the hypotheses that have emerged from the earlier phase. Then at the end, after the numerical data has been received and analysed, its interpretation is subject to more qualitative consultation. Does the data mean what we think it does? Often those who are closest to the stakeholder base provide a different and important perspective – which needs to be considered.
Reliance upon quantitative data alone is dangerous.