Originally published 8 February 2011. This article is retained from the tCI archive. Law, policy or practice may have changed since publication.
A public sector organisation recently showed us its plan to take a sensitive and controversial decision. There had been extensive consultation and a distinct majority of respondents had a clear view of what should happen.
What it proposed was a detailed formula whereby five operational factors would be considered, with points awarded according to the assessment made of each criterion. But there was a sixth factor, called ‘Public acceptability’ – given a weighting equal to some of the other factors and higher than others. It prompts the question of whether consultation responses can be considered alongside other factors, or whether they are in some way different in kind from other types of data likely to be taken into account. The problem with this type of mechanistic procedure is obvious. If operational factors can always outscore the views of consulted stakeholders – why bother to engage them? Only when the analysis of other factors makes the answer quite close can their input make any difference. Otherwise it makes consultee opinions pretty irrelevant. So might it make more sense to let decision-makers determine the best operational solution and then consider whether consultation outputs warrant a change to that decision?
These convoluted processes matter hugely when public opinion is at odds with official preferences. But they must also reflect the nature of the evidence that has been gathered, and it is here that some significant challenges arise. Seasoned consultation specialists know that most exercises generate a mixture of qualitative and quantitative data and that decision-makers have a habit of interpreting them differently.
Numbers are easy to understand; high numbers impress, and they lend themselves to newspaper headlines. But they don’t always mean what you think they mean. A few years ago, we arrived to train local government staff at a Council that was being amalgamated with neighbours. A large banner decorated the building. 90% of residents want to keep XX Council! it screamed. What we suspect is should have said is that 90% of those who decided to respond to our survey want to keep XX Council. Anyone looking for entertaining examples of this genre should read Ben Goldacre’s best-selling book on ‘Bad Science’
Of course readers of this Topic won’t be guilty of the worst excesses, but the truth is that many decision-makers view consultation data as inherently flaky! Sample sizes are seldom sufficient, and when large enough are not necessarily representative. Public meetings are badly attended; focus groups are sometimes assembled haphazardly. And e-participation just attracts the nerds, doesn’t it! Totally unfair, of course, but it has enabled many organisations too often to disregard arguments they don’t like.
Now if this was the case when consultation exercises could command generous budgets, imagine how much more scepticism we might find in the coming months, especially if public engagement is done on a shoestring. Let’s always remember that there may not be much of a correlation between the amount of money spent and the quality of the evidence that is produced.
What we need is a concerted effort to improve the credibility of consultation data. We must be scrupulous in gathering it and honest in presenting it. Above all, we must focus on some of the new technologies, like the use of Social Media which may be too easy for traditionally-minded decision-makers to dismiss.
When times are tough, understanding what stakeholders are saying is really important and requires rigorous analysis and sensitive interpretation. The Coalition Government has been experimenting with ‘crowdsourcing’ techniques to gather large numbers of ideas, but if it is to make intelligent use of all this data, it now needs to analyse thousands of submissions, and convince us it makes a difference. That’s the hard part and maybe we haven’t given it enough attention so far. Time for this to change!
Trigger points
- How is consultation data presented and considered by decision-makers in your organisation?
- Who is responsible for analysis and interpretation? Should you be using independent people to guarantee objectivity?
This is the 167th Tuesday Topic; a full list of subjects covered is available for Institute members and is a valuable resource covering so many aspects of consultation and engagement