Article reader Listen + reading controls
Article reader
Preparing the reader…
Reading settings
A benchmark needs a theory of use¶
The National Institute of Standards and Technology (NIST) has released draft guidance on best practices for automated benchmark evaluations. The subject sounds technical, but it reaches directly into strategy and procurement. Organizations routinely use benchmark results to choose models, justify investment, and communicate readiness.
A benchmark can support those decisions. It can also lend numerical confidence to a question it was never designed to answer.
The score arrives after a chain of choices¶
An artificial intelligence (AI) benchmark begins long before a model produces an answer. Someone chooses the tasks, examples, scoring rule, reference answers, sampling method, model settings, and treatment of ambiguity. Those choices encode a view of what competence looks like.
The resulting score is therefore not a property of the model alone. It is an observation produced by a measurement system. Change the prompt, tool access, judge, population, or operating conditions and the result may change. That does not make benchmarking useless. It means the result needs a scope.
Messick's work on validity in assessment remains relevant: validity concerns the interpretation and use of evidence, not simply the construction of a test. A team should be able to explain what inference a score supports and what inference it does not.
Automated judging creates a second system to evaluate¶
Large-scale AI evaluations increasingly use models to grade model outputs. This makes testing faster and can support richer tasks than exact-match scoring. It also creates another source of error.
An automated judge may prefer a style, miss a subtle factual defect, reward confident language, or disagree with a domain expert about what matters. If the judge and tested model share training patterns, their errors may be correlated. A precise result can conceal uncertain ground truth.
The remedy is not to reject automated evaluation. It is to validate the evaluator for the intended task. Compare automated judgments with qualified human review. Examine disagreements. Report uncertainty. Keep a sample of difficult and consequential cases for deeper analysis. Treat evaluation code, prompts, and judge versions as controlled components.
Bring the benchmark closer to the work¶
Generic benchmarks are useful for orientation. A deployment decision needs additional evidence drawn from the organization's own operating environment.
A practical evaluation portfolio can include:
- broad public benchmarks for general capability;
- task-specific datasets that represent local work;
- adversarial and edge cases supplied by practitioners;
- end-to-end workflow tests, including tool use and human review;
- and field monitoring for conditions that the predeployment tests miss.
The portfolio should begin with a decision. If the question is whether an assistant can help analysts synthesize evidence, the evaluation should measure source fidelity, omission, uncertainty, and the review effort imposed on the analyst—not just whether the output resembles an ideal paragraph.
It should also preserve failures as organizational knowledge. A benchmark report that ends with a leaderboard throws away the most useful material: which tasks failed, why experts disagreed, what workaround users attempted, and which design change is now required.
NIST's draft arrives at a useful moment. As automated evaluation becomes easier, organizations will be tempted to produce more scores. The better ambition is to produce stronger evidence.
Before accepting any benchmark result, ask one question: what decision would be different if this score moved? If the answer is unclear, the benchmark does not yet have a theory of use.
Sources and research trail¶
- National Institute of Standards and Technology, “Towards Best Practices for Automated Benchmark Evaluations” (January 30, 2026).
- National Institute of Standards and Technology, Artificial Intelligence Risk Management Framework (AI RMF 1.0) (2023).
- Messick, “Validity” (1987).
- Liang and colleagues, “Holistic Evaluation of Language Models” (2023).
- Raji and colleagues, “Closing the AI Accountability Gap” (2020).