Article reader Listen + reading controls
Article reader
Preparing the reader…
Reading settings
Scaling Trustworthy AI in Government Requires an Operating System¶
Originally published in 2024; substantially revised in 2026 to deepen the analysis and incorporate additional sources.
Government agencies do not lack AI ideas. They lack repeatable mechanisms for turning a promising use case into a capability that can be evaluated, authorized, adopted, monitored, and improved.
The usual response is to scale the technology: add compute, models, data pipelines, or platform capacity. Those investments matter. But when every program defines its own risk process, evidence package, human-oversight model, security interpretation, and approval path, the organization scales experimentation while preserving the bottlenecks that prevent adoption.
Trustworthy AI scales when the enterprise standardizes the work around the model—not merely access to the model.
Pilot success hides enterprise friction¶
A pilot can succeed through concentrated expertise and executive attention. A small team manually cleans data, resolves policy questions, monitors performance, briefs stakeholders, and keeps users engaged. Those invisible services often disappear when the capability moves into a portfolio.
Scaling reveals questions the demonstration deferred:
- Who owns the data after the pilot team leaves?
- What evidence is required for approval?
- Which risks receive independent review?
- How are users trained and supported?
- What happens when model behavior changes?
- Which incidents require escalation?
- Who funds monitoring and sustainment?
- Can another team reuse the technical and governance assets?
If each program answers independently, governance becomes artisanal. The organization accumulates governance debt: inconsistent decisions, duplicated reviews, unclear accountability, and evidence that cannot be reused.
Separate the common platform from the mission product¶
An enterprise AI strategy should distinguish between reusable foundations and mission-specific choices.
Shared foundations may include:
- Approved model access and routing
- Secure development and evaluation environments
- Data and knowledge connectors
- Identity, policy, logging, and secrets management
- Evaluation harnesses and scenario libraries
- Model and prompt versioning
- Monitoring and incident-management integrations
- Standard documentation and evidence artifacts
Mission products still need local design: user research, workflow integration, decision rights, performance thresholds, operating limits, and outcome measurement. Standardization should remove repeated infrastructure work without flattening meaningful differences among missions.
The objective is a paved road with governed extension points—not a central platform that assumes every use case is the same.
Build an evidence factory alongside the software factory¶
Modern software platforms automate compilation, testing, security checks, deployment, and telemetry. Trustworthy-AI platforms should also automate the production and maintenance of assurance evidence.
For each release, the system should preserve:
- Intended use and prohibited use
- Data sources, lineage, and fitness assessments
- Model, prompt, tool, and configuration versions
- Performance, robustness, security, and human-factors evaluations
- Known limitations and unresolved uncertainty
- Review decisions and accountable owners
- Monitoring thresholds, fallback behavior, and rollback criteria
This turns governance from a document-reconstruction exercise into a normal property of delivery. The CDAO Responsible AI Toolkit is valuable precisely because it translates high-level principles into lifecycle practices and artifacts (U.S. Department of Defense, 2023).
Risk tiering should change the process, not only the label¶
Not every AI use warrants the same review. A summarization aid, fraud-prioritization model, benefits determination system, and operational targeting capability create different consequences and require different evidence.
Risk classification is useful only if it changes what happens next. A tier should determine:
- Required evaluation depth
- Independence and expertise of reviewers
- Human authority and contestability requirements
- Monitoring and reporting frequency
- Documentation and transparency obligations
- Approval level and residual-risk owner
- Conditions for suspension or reauthorization
The NIST AI Risk Management Framework provides a flexible structure for governing, mapping, measuring, and managing context-specific risk (NIST, 2023). GAO’s AI Accountability Framework similarly organizes accountability around governance, data, performance, and monitoring (GAO, 2021).
These frameworks are most effective when embedded into portfolio intake, funding, architecture, acquisition, and delivery—not added after a system is built.
The scaling bottleneck is often organizational throughput¶
AI programs compete for scarce people: data stewards, security engineers, evaluators, legal counsel, human-factors specialists, acquisition professionals, and mission experts. A centralized review board can improve consistency but become a queue that separates decisions from local context.
A scalable model combines central capability with distributed responsibility:
- A central function defines standards, reusable services, minimum evidence, and escalation policy.
- Domain teams retain responsibility for mission outcomes, users, context, and operational performance.
- Independent reviewers focus on high-consequence claims rather than rechecking every routine artifact.
- Communities of practice share evaluation methods, incidents, patterns, and lessons across programs.
This is federated governance with an enterprise spine. It preserves local knowledge while making accountability and evidence portable.
Portfolio leaders need outcome and learning metrics¶
Counting AI use cases rewards proliferation. Counting deployed models rewards technical output. Neither reveals whether the agency is becoming better at producing useful, accountable capability.
More informative portfolio measures include:
- Time from validated mission problem to operational evaluation
- Percentage of evidence generated automatically
- Reuse of data products, evaluation sets, and approved platform services
- Time required to resolve a risk or policy decision
- User adoption and appropriate-reliance measures
- Operational outcomes compared with the baseline
- Incidents, overrides, and lessons incorporated into subsequent releases
- Time and cost required to retire or replace a capability
The enterprise is scaling when each new program becomes easier to evaluate and operate because prior programs improved the shared system.
The strategic takeaway¶
Secure compute and model access are necessary foundations. They are not a trustworthy-AI operating model. Government scales AI when it can repeatedly connect mission need, technical delivery, evidence, human judgment, approval, monitoring, and learning.
The platform should make responsible behavior easier. Governance should make uncertainty actionable. The portfolio should convert local experience into reusable institutional capability.
That is how AI moves from a collection of pilots to an accountable public institution. It is also the central concern of my work on trustworthy AI operations and transformation systems.
References¶
- National Institute of Standards and Technology. (2023). Artificial Intelligence Risk Management Framework (AI RMF 1.0).
- U.S. Government Accountability Office. (2021). Artificial Intelligence: An Accountability Framework for Federal Agencies and Other Entities.
- U.S. Department of Defense. (2023). CDAO releases Responsible AI Toolkit.
- Office of Management and Budget. (2024). M-24-10: Advancing Governance, Innovation, and Risk Management for Agency Use of Artificial Intelligence.