Article reader Listen + reading controls
Article reader
Preparing the reader…
Reading settings
Evaluator disagreement is information¶
Google Research has published work asking how many human raters an artificial intelligence benchmark needs. The question matters because many model evaluations rely on people to judge qualities that cannot be reduced to exact matching: helpfulness, factuality, relevance, safety, style, or the quality of an explanation.
Adding raters can improve reliability. But disagreement is not always noise that disappears when the sample grows. Sometimes it is the finding.
A rubric is a model of judgment¶
Every evaluation rubric encodes a view of what matters. Two qualified people may disagree because an instruction is ambiguous, because the answer contains a tradeoff, or because their disciplines apply different standards. A cybersecurity specialist may reward caution that an operator experiences as obstruction. A clinician may notice a dangerous omission that a general reviewer misses. A program manager may value actionability while a researcher values completeness.
Artificial intelligence (AI) evaluation becomes misleading when it compresses those differences into one average without explaining them.
Inter-rater reliability is useful for determining whether a measurement process is consistent. Low agreement can reveal poor training or unclear criteria. It can also reveal that the construct itself is contested or multidimensional. More raters may stabilize the average while preserving the underlying conflict.
The composition of the panel matters¶
The question “How many?” should follow “Which perspectives?” A large group drawn from one population can agree strongly and still miss the people most affected by a system. Expertise, lived experience, organizational role, and operating context influence which failures become visible.
This is especially important in defense and critical systems. A model output may be judged for technical correctness, security, mission relevance, legal sufficiency, cognitive burden, and recoverability. Those dimensions belong to different communities. Treating one group as a proxy for all of them creates a false consensus.
Page's work on diversity and collective problem solving helps explain why varied perspectives can improve performance on complex problems. Diversity is not a substitute for expertise, and expertise is not a substitute for perspective. Evaluation design needs both.
Preserve the disagreement trail¶
A mature evaluation process can separate several cases:
- Rater error: someone missed evidence or misapplied the rubric.
- Rubric ambiguity: the criterion needs clarification.
- Task ambiguity: more than one response is reasonably defensible.
- Value conflict: stakeholders prioritize different outcomes.
- Context dependence: the judgment changes with user, setting, or consequence.
Each case requires a different response. Only the first is solved primarily through quality control. The others may require revised system behavior, multiple metrics, subgroup reporting, or an explicit management decision about tradeoffs.
Organizations should keep examples with high disagreement and review them with the people who understand the work. These cases make excellent scenarios for governance, training, and design. They reveal where the system's apparently simple task crosses an organizational boundary.
Do not outsource the hard judgment to the aggregate¶
Automated judges are likely to make rating cheaper, but they do not remove the issue. A model judge has its own implicit preferences and blind spots. It should be validated against the relevant human communities, especially for consequential or contested criteria.
The practical objective is not perfect agreement. It is a measurement process that distinguishes uncertainty from conflict and gives decision-makers enough context to act responsibly.
When evaluators disagree, the tempting response is to gather more votes and publish the mean. Sometimes that is appropriate. Sometimes the organization needs to stop and ask what the disagreement is teaching it about the task, the users, and the values hidden inside the score.
Sources and research trail¶
- Google Research, “Building Better AI Benchmarks: How Many Raters Are Enough?” (March 31, 2026).
- Hallgren, “Computing Inter-Rater Reliability for Observational Data” (2012).
- Page, “Groups of Diverse Problem Solvers Can Outperform Groups of High-Ability Problem Solvers” (2004).
- Suresh and Guttag, “A Framework for Understanding Sources of Harm throughout the Machine Learning Life Cycle” (2021).
- National Institute of Standards and Technology, Artificial Intelligence Risk Management Framework (AI RMF 1.0) (2023).