Article reader Listen + reading controls
Article reader
Preparing the reader…
Reading settings
Agent platforms make observability part of the product¶
OpenAI's March 11 release introduces new tools for building agents: the Responses application programming interface, built-in web and file search, computer use, an Agents software development kit, and integrated tracing.
The announcement reduces the amount of custom plumbing required to create systems that can pursue goals across tools. It also makes a deeper point visible: in an agentic system, observability is not support infrastructure. It is part of the product.
An application programming interface (API) call that returns text can often be evaluated at its boundary. An agent may search, select a source, call a tool, hand work to another component, retry after failure, and change external state. Two runs can reach similar answers through very different—and differently risky—paths.
The path matters¶
Traditional software observability focuses on logs, metrics, and traces that help engineers explain system behavior. Agent systems need all three, but the semantic layer is richer. Teams need to know which goal the agent is pursuing, which evidence it uses, why it selects a tool, what state changes, where a guardrail intervenes, and when a person takes control.
A final answer can look correct while the process violated policy. The reverse is also possible: an agent may follow a sound process and encounter an unavailable tool or ambiguous instruction. Evaluation that collapses both cases into pass or fail will teach the team too little.
Handoffs are control points¶
The Agents software development kit (SDK) includes explicit handoffs among agents. In organizational terms, that resembles a work transfer among specialists. Coordination research shows that handoffs require accountability, predictability, and shared understanding. Okhuysen and Bechky's integrative review is useful here: coordination mechanisms work by making responsibilities and interdependencies legible.
An agent handoff should therefore carry more than content. It should carry purpose, authority, provenance, uncertainty, and completion criteria. Otherwise, one agent can inherit a task without the context required to recognize that it should stop.
Build an operational record¶
Before production, teams should define a trace schema that answers:
- Who or what initiated the run?
- What instructions, model, tools, and data boundaries applied?
- Which claims come from sources and which depend on inference?
- Which actions changed external state?
- Where do retries, overrides, or policy checks occur?
- Who owns the outcome and the incident if it fails?
Those traces should feed evaluation datasets and incident reviews. Repeated corrections can reveal a weak prompt, an ambiguous policy, an unreliable data source, or a poorly designed handoff. The record becomes organizational memory for improving the system.
Do not confuse visibility with control¶
A beautiful trace viewer does not make an agent safe. People still need time, authority, and skill to act on what it shows. Logging sensitive reasoning or data can create security and privacy risks. Retaining everything indefinitely can turn observability into surveillance.
The design must be selective: enough evidence to reconstruct consequential behavior, bounded by access control and retention rules.
Agent platforms will make demonstrations easier. Production will still depend on whether teams can see, explain, interrupt, and learn from what the agents do. That is why observability belongs in the original capability design and acceptance criteria—not in the backlog after the first incident.
Sources and research trail¶
- OpenAI, “New Tools for Building Agents” (March 11, 2025).
- Okhuysen and Bechky, “Coordination in Organizations: An Integrative Perspective” (2009).
- Amershi and colleagues, “Software Engineering for Machine Learning” (2019).
- National Institute of Standards and Technology, AI Risk Management Framework 1.0 (2023).