How Reliable Is Your Consultation Survey Evidence?

The Department for Culture, Media and Sport (DCMS) has just published a quality report that lays bare the strengths and limits of its own survey evidence. Most local consultation surveys never offer that same disclosure. If your evidence base cannot withstand the same scrutiny, your consultation’s defensibility is already in question.

What does the quality report actually disclose?

The Community and Engagement Survey (CES), DCMS’s new combined replacement for the Participation Survey and the Community Life Survey, published its first quality report on 2 July 2026. The report does something most local consultation surveys never attempt. It names its own weaknesses in public, before anyone has to ask.

It discloses that the individual response rate for the predecessor Participation Survey fell from 25% to 24% between 2023/24 and 2024/25, and that the response rate for the Community Life Survey fell from 22% to 20% over the same period.

It states plainly that people with no fixed abode, and those living in communal establishments such as care homes, are excluded from the sample. It acknowledges that self-reporting produces inaccurate recall rather than a precise account of events, particularly for anything a respondent is asked to remember from up to twelve months earlier.

None of this is buried in a footnote. It sits in a section headed strengths and limitations, published for anyone to read, alongside an explanation of how sampling error and confidence intervals affect the headline figures, and a description of the independent quality assurance checks carried out before each release.

The report also sets out how DCMS corrects errors. Major mistakes are reissued as soon as practical, and any unresolved issue is flagged to users rather than left quiet until the next release. That kind of visible governance is rare outside official statistics.

Why does evidence quality matter for consultation defensibility?

The Gunning principles, the legal test for fair public consultation in the UK, derived from R v London Borough of Brent ex parte Gunning (1985) 84 LGR 168, require that consultees receive sufficient information to permit intelligent consideration and response, and that their response is conscientiously considered before a decision is made.

The Supreme Court confirmed and applied this test in R (Moseley) v London Borough of Haringey [2014] UKSC 56, and it extends to the evidence a public body relies on, not only to the proposal itself.

Courts examine whether decision makers understood what their evidence could and could not tell them, not simply whether a survey was carried out. A survey with an undisclosed sample, an unstated response rate, or no acknowledgement of who it excluded gives decision makers no real way to judge how much weight it deserves, however confident the headline percentages sound.

If that survey later forms part of a consultation’s evidence base, a challenge can argue the authority never understood, let alone disclosed, its own evidence’s limits. That argument is harder to answer than it should be, because most local surveys give decision makers nothing to point to in reply.

Where do most survey-based consultations fall short?

Resident satisfaction surveys and local engagement surveys rarely publish anything close to the CES’s level of disclosure. Response rates go unreported. Sampling methods are described in a single line, if at all, and confidence intervals rarely feature in the committee report that follows.

Whether the survey reached seldom-heard groups, including those with protected characteristics under the Public Sector Equality Duty (PSED) at section 149 of the Equality Act 2010, the duty on public bodies to have due regard to equality impacts when making decisions, is often not addressed at all. Incentives, reminder letters, and paper alternatives, all used by the CES to widen its reach, are frequently absent from smaller local exercises.

This matters because the CES itself excludes groups whose circumstances often overlap with those the PSED is designed to protect: people with no fixed abode, and residents of communal establishments. A local survey with a similar blind spot, left undisclosed, creates the same equalities risk that the CES report is careful to name and explain rather than quietly absorb.

The CES’s honesty about its own limits is not a weakness in the survey. It is what allows a decision maker, and a court, to judge fairly how much the evidence can bear, and what it cannot.

What should practitioners take from this?

The CES quality report is not a consultation document, but it works as a model of what disclosure looks like when an evidence base is expected to survive scrutiny. Practitioners commissioning survey evidence for a live consultation should be able to state the response rate, the sampling method, and who the survey did not reach, alongside the results themselves.

That discipline costs little to apply. It means looking at the numbers behind the headline figures, and including those numbers, and their limitations, in the report that goes to committee or cabinet.

It also means checking survey coverage against PSED duties before relying on the results, rather than after a challenge has already been lodged. A survey that has not reached seldom-heard groups cannot fairly stand in for their views, whatever the response rate among those it did reach. Done well, this is a modest amount of extra work that does a great deal to strengthen an evidence base long before anyone has reason to test it in a courtroom or a scrutiny committee.

How tCI Can Help

Organisation Wide Learning Hub Access
Equip your entire team with professional consultation skills through one platform. Self paced courses, live virtual classrooms, practical toolkits and expert resources that build a shared baseline of competence across your organisation. Trusted by councils, NHS bodies and regulators nationwide.

Bespoke Training Workshops
Training that works with your real projects, not hypothetical scenarios. Sector tailored sessions help teams apply good practice to live challenges: sharpening consultation documents, building defensible codebooks, strengthening equality analyses. Half day or full day workshops for health, local government, planning and public service teams.

Coaching for Complex or High Risk Consultations
Expert guidance when the stakes are highest. One to one and small group coaching for senior officers navigating legally exposed or politically contentious decisions. Strengthen your judgement on proportionality, evidence standards and challenge management. Essential for organisations that may face judicial review risk or major service changes.

Whether you’re preparing for a high stakes service change, building long term consultation capability, or need confidence that your evidence approach will stand up to scrutiny, we can help.

Contact tCI: hello@consultationinstitute.org