Article reader Listen + reading controls
Article reader
Preparing the reader…
Reading settings
A benchmark score needs an uncertainty model¶
The National Institute of Standards and Technology (NIST) has published a report on expanding artificial intelligence evaluation with statistical models. Its central implication is easy to state and surprisingly easy to neglect: an evaluation result is an estimate.
Organizations often present model scores to one decimal place while leaving the population, sampling assumptions, and uncertainty largely invisible. That precision can exceed what the evidence supports.
A test set is a sample of possible work¶
Artificial intelligence (AI) systems may encounter an enormous variety of prompts, documents, users, tools, and operating conditions. Any evaluation observes only a selection. Statistical reasoning asks what larger population that selection represents and how confidently performance might generalize.
If a benchmark contains many near-duplicate questions, its apparent sample size can be misleading. If examples come from public sources that may appear in training data, the score may overstate performance on genuinely new work. If easy cases dominate, an average can conceal weakness where consequences are greatest.
The organizational version of the problem is familiar. A program office reports that a system is “92 percent accurate,” but users need to know whether the remaining 8 percent is random or concentrated in a particular document type, language, user group, or mission condition. A single score cannot answer.
Measurement should follow the decision¶
A procurement team comparing general-purpose models needs different evidence from an operator deciding whether to rely on one recommendation. An engineering team monitoring a new version needs sensitivity to change. An assurance team needs coverage of hazards and failure severity.
The evaluation design should therefore begin with the inference the organization wants to make. Then it can specify the relevant task population, sampling plan, metrics, uncertainty, and threshold. This is more disciplined than selecting an available benchmark and deciding afterward what the score means.
NIST's AI Risk Management Framework reinforces this context-first approach. Measurement is connected to intended purposes, affected people, risk tolerance, and ongoing management. The metric is part of an argument about use.
Make uncertainty actionable¶
Uncertainty should not be relegated to a technical appendix. It can change management decisions.
A useful evaluation report can show:
- confidence or credible intervals around aggregate estimates;
- performance across meaningful subgroups and conditions;
- sensitivity to prompt, judge, and model settings;
- the number and nature of disputed reference answers;
- severity-weighted failure analysis;
- and conditions under which the result should not be generalized.
This reporting helps teams decide where human review is necessary, where additional data would be valuable, and which operational conditions require a fallback. It also makes model comparisons more honest. A small difference between two scores may not be decision-relevant when uncertainty and deployment cost are considered.
There is a cultural benefit as well. Explicit uncertainty gives practitioners permission to discuss what is not known. High-reliability organizations do not treat uncertainty as embarrassment; they treat it as information for attention and preparedness. Weick and Sutcliffe describe this as a preoccupation with failure: weak signals and small discrepancies become opportunities to learn before they combine into larger harm.
AI evaluation is becoming more automated and more visible. It must also become more statistically and organizationally mature. A score is useful when people understand what produced it, what range of performance remains plausible, and which decision it is strong enough to support.
The goal is not to make every evaluation complicated. It is to stop treating an estimate as a fact without saying what it estimates.
Sources and research trail¶
- National Institute of Standards and Technology, “New Report: Expanding the AI Evaluation Toolbox with Statistical Models” (February 19, 2026).
- National Institute of Standards and Technology, Artificial Intelligence Risk Management Framework (AI RMF 1.0) (2023).
- National Academies of Sciences, Engineering, and Medicine, Reproducibility and Replicability in Science (2019).
- Liang and colleagues, “Holistic Evaluation of Language Models” (2023).
- Weick and Sutcliffe, Managing the Unexpected (2015).