Originally published 1 November 2006. This article is retained from the tCI archive. Law, policy or practice may have changed since publication.
Those with long memories may remember something called The Pepsi Challenge. Effectively this was a blind test to see whether consumers preferred the taste of traditional Coca-Cola or the fiercely competitive upstart - Pepsi Cola.
So damaging was this to Coca Cola that on the strength of overwhelming evidence that Pepsi was preferred, they abandoned their secret but ageing formula and launched New Coke as a direct rival to the Pepsi product. What happened then is now part of marketing folklore. Coca Cola’s customers didn’t like the new taste and forced the Company to reintroduce it as Classic Cola!
The point of this story is that Coca Cola discovered the difference between short-term and long-term evaluation. The Pepsi Challenge was a sip test. This means you took a small sip of each drink and pronounced upon it; unsurprisingly people opted for the sweeter-tasting drink. But if you gave the same customer a dozen cans to drink over a weekend, the results were very different, and the traditional Coca Cola was preferred. Yet they gambled the whole Company on the short-term assessment. Apply this to consultation. We all stress the need for some form of evaluation, and in about 30% of documented strategies, there is provision for such an assessment right at the end of the process. So far so good - but relying upon instant evaluation is inadequate. If the ultimate purpose of consultation is to influence decisions or policies, then how can one assess the effectiveness of the exercise without waiting to measure the extent to which its influence was exerted? With a delay of many months between the end of a consultation period and its eventual outcome, any immediate evaluation is likely to be premature.
So let’s keep it simple. Without undertaking any elaborate research or even discussion with umpteen stakeholders ….let’s take an arbitrary period of twelve months and ask some searching questions at that point. Okay, that’s a bit long for a local Council seeking best-value style views on waste collection or street lighting; it’s also a bit short for major Government policy-making maybe with concomitant legislative implications. Even so, it’s a reasonable and comprehensible benchmark.
On the anniversary of the closing date of a consultation, it is quite legitimate to ask what happened ….or, more precisely, what has been seen to have happened. Consultors sometimes neglect their obligation to communicate with those whom they consulted, and provide feedback, both in terms of output and outcome. That feedback has to be meaningful – not just Thank you so much; we’ve taken your views into account …….
At that twelve-month point, anyone who is interested has the right to expect a fully documented audit trail, from consultation closing date to policy or decision announcement. An effective consultation is one where the causal link between the contributions of stakeholders and subsequent deliberation is openly visible.
On occasions, these are views which are eventually rejected. Fine, but let’s see the process and the papers that went from one office to the other - only the most extreme cases will be protected by exemptions to the Freedom of Information Act - and let’s see how the consultor explains its decisions to those who were consulted. Every public body is supposedly committed to online publication of such information, so there are no grounds for reticence!
We know that consultation fatigue is real and that stakeholders will only participate when they are confident that their views will be taken seriously. How will they reach this happy state if, twelve months later, there is still little evidence of real feedback and real influence? If enough people ask the question, the evidence may start to appear.
Trigger points
- Ensure that an evaluation phase is built into your consultation process; the Institute’s paper Was It Worth It? provides an overview of different evaluation criteria.
- Surf the internet; start looking at high-profile consultations that closed one year ago in your own or other organisations. What visible evidence is there of what happened?
- Adopt a standard point (we think 12 months is fine) for an evaluation and publish the resulting retrospectives.
This is the 71st Tuesday Topic; a full list of subjects covered is available for Institute members and is a valuable resource covering so many aspects of consultation and engagement