Building Trustworthy AI Systems: From Aspiration to Evidence
Originally published in 2024; substantially revised in 2026 to deepen the analysis and incorporate additional sources.
DARPA’s pursuit of AI systems that can be trusted raises a deceptively difficult question: what, precisely, are we claiming when we call an AI system trustworthy?
Trustworthiness is often presented as a list of desirable attributes—reliability, robustness, explainability, fairness, security, safety, accountability. Those attributes are important, but a list is not an assurance argument. A system can perform well on an aggregate benchmark and fail under a mission-relevant distribution shift. It can produce an explanation that sounds coherent without helping a user detect error. It can satisfy a formal control while leaving responsibility fragmented across organizations.
Trustworthy AI is not a permanent label attached to a model. It is a bounded, evidence-backed claim about how a socio-technical system behaves under specified conditions.