Why response numbers don’t prove a consultation is fair

A consultation that attracts thousands of responses can still fail the legal test for fairness if the people who replied don’t reflect the population the decision affects. Here is why representativeness, not response volume, is what decides whether an exercise can withstand challenge.

What does “seldom-heard” mean in a consultation context?

Seldom-heard groups are people who are consistently under-represented in consultation and decision-making, not because they are uninterested, but because the process itself creates barriers to taking part.

The term matters because of where it puts the responsibility. As the Scottish evidence review by Iriss notes, the shift from “hard to reach” to “seldom-heard” moves attention away from blaming communities and towards an institution’s duty to make participation accessible.

In practice, seldom-heard groups vary by topic and locality, but typically include disabled people, people on low incomes, renters, younger residents, carers, and minority ethnic communities. None of these groups is inherently disengaged.

They are groups for whom the standard consultation format, whether that is a survey link, a public meeting, or a written document, presents a barrier that more confident or better-resourced respondents do not face.

This is not a side issue for consultation design. It goes to the heart of whether a decision-maker can later demonstrate that they understood who would be affected by their decision and took reasonable steps to hear from them.

Why doesn’t a high response rate prove a consultation was fair?

A high response rate lowers the risk that a consultation result is skewed, but it does not prove the exercise was fair. The question that matters in law and in methodology is whether the people who took part reflect the population the decision will affect, not how many people responded.

This connects directly to the Gunning principles, the legal test for fair public consultation in the UK. Conscientious consideration of responses, one of the core requirements, is not satisfied by a large number on a spreadsheet. It depends on whose views were actually considered.

The methodological evidence backs this up. Pew Research Center’s work on survey representativeness found that the real threat to data quality is non-response bias, the gap between the views of people who took part and those who did not, rather than the response rate itself.

A survey with 10% participation can be less biased than one with 50% participation, if the smaller group better reflects the population as a whole.

There is also a consistent pattern in who chooses to take part. Pew’s analysis of low response rates found that people who respond to surveys are more likely than non-respondents to be already engaged in civic and voluntary activity. Left unmanaged, this means a consultation will tend to over-represent people who already have a voice and under-represent those who do not.

This is the failure that should concern any practitioner reporting consultation results. An organisation that publicises “10,000 responses” but cannot say whether those responses came from across the affected population, including its seldom-heard groups, has not demonstrated a fair consultation. It has demonstrated that 10,000 people were able to respond, which is a different and much weaker claim.

What should a defensible consultation report show?

The National Standards for Community Engagement, the UK’s main framework for good practice in this area, set out Inclusion as a core principle. They define good engagement as involving the people affected by a decision, particularly those usually excluded, rather than treating headline participation figures as a quality marker in themselves.

For a consultation report to be defensible, it should answer four questions:

1. Who is affected by the decision? Residents, service users, carers, businesses, or specific demographic groups.
2. Who actually took part? A breakdown by relevant characteristics such as age, disability, ethnicity, geography or deprivation.
3. Who is missing? If thousands responded but almost none were under 25, renters, or from minority ethnic communities, the exercise carries a real vulnerability regardless of total numbers.
4. What corrective action was taken? Targeted recruitment, community intermediaries, translated materials, accessible formats, or oversampling of under-represented groups.

For organisations subject to the Public Sector Equality Duty (PSED), the legal requirement under section 149 of the Equality Act 2010 to have due regard to the impact of decisions on people sharing protected characteristics, this framework is close to the evidence a court would expect if due regard had genuinely been paid. A report that cannot answer who was missing, and what was done about it, leaves that duty unevidenced.

Building this population coverage into a consultation’s design and reporting from the outset is far more defensible than trying to reconstruct it after a challenge has been raised. The four questions above work best as part of the original brief, not as a retrospective audit once concerns have already been raised.

How tCI Can Help

Advice and Guidance
A tCI faculty member will work alongside you to support the development of your decisions and engagement approach. We provide independent, constructive advice at critical stages, helping you strengthen stakeholder mapping, test communication strategies, and plan robust post-decision engagement. Our role is to act as a critical friend, offering practical recommendations grounded in consultation law and good practice that build confidence in your process.

Risk Assessment
Early identification of legal, political or reputational risks in your engagement approach. Using tCI’s five-risk methodology, we spot gaps before challenge arises, helping you strengthen stakeholder communication and demonstrate procedural fairness from the outset.

Executive Briefings
Concise updates for senior leaders on consultation law, engagement duties and post-decision risks. Helps boards and leadership teams make confident, defensible decisions when under pressure, with clear guidance on what good engagement looks like after difficult choices are made.

Whether you’re preparing for a high stakes service change or building defensible evidence for complex decisions, we can help.

Contact tCI: hello@consultationinstitute.org