Article reader Listen + reading controls
Article reader
Preparing the reader…
Reading settings
Function calling moves risk beyond the chat window¶
OpenAI has added function-calling support to its chat models. Developers can describe functions using structured definitions, and the model can return arguments that an application may use to call external tools or retrieve information.
This is an important improvement for building reliable integrations. It also makes a boundary explicit: the model proposes; the application decides what happens next.
Structured output is not validated intent¶
A language model can translate a user's request into structured arguments more cleanly than a developer can extract them from free-form prose. A valid JavaScript Object Notation (JSON) object, however, does not mean the user's intent was understood correctly, that the values are safe, or that the action is authorized.
Artificial intelligence (AI) applications must treat model-produced arguments as untrusted input. The application should validate types, ranges, identities, resource ownership, and policy before it calls anything.
This follows a longstanding security rule: do not confuse syntactic validity with permission. A well-formed request can still be dangerous.
Keep policy outside the model¶
The model may help interpret language, but deterministic controls should enforce authority. A system prompt that says “never issue a refund over $500” is not an adequate financial control. The external service should enforce the limit based on authenticated identity, role, transaction state, and organizational policy.
The same principle applies to data access. The model should not decide which employee records a user may retrieve. The application should pass the user's identity to an authorization layer that independently scopes the query.
The National Institute of Standards and Technology Zero Trust Architecture provides the right posture: evaluate each access request explicitly and grant the least privilege needed. The model is one signal in the request path, not a trusted policy engine.
Describe functions as contracts¶
Function schemas should carry more than parameter names. Teams need an operational contract for each function:
- what business or mission action it performs;
- whether it reads or changes external state;
- data classifications it may receive or return;
- permissions and user identity it requires;
- preconditions the application must validate;
- whether human confirmation is required;
- expected failure and retry behavior; and
- how the action is logged, reversed, or escalated.
This contract creates a shared artifact for developers, security teams, product owners, and operators. It also supports testing combinations of functions, where risk can emerge from a sequence even if every individual call is permitted.
Saltzer and Schroeder's least-privilege and complete-mediation principles remain strikingly current. Every access should be checked; no component should carry more authority than it needs. Function-calling AI does not require a new theory of access control. It requires disciplined application of the old one to a probabilistic interpreter.
Confirmation should expose the consequence¶
For consequential actions, a person should confirm a concrete proposal, not an abstract intention. Show the target, values, source of key information, cost, and expected consequence. Distinguish a draft from a committed action. When possible, give the user a reversible staging step.
The confirmation layer should not rely on the model to summarize itself accurately. The application can render the validated arguments directly and flag anything unusual.
Observe proposals as well as executions¶
Logs should capture the user request, model and configuration, proposed function and arguments, validation result, confirmation, execution outcome, and any rollback. Rejected proposals matter. They reveal misunderstanding, attempted misuse, and policies users routinely collide with.
Function calling turns natural language into a more dependable software interface. Its safety comes from preserving the boundary between interpretation and authority. Let the model propose a structured action. Let deterministic systems validate it. Let accountable people approve what matters. And make the entire path observable enough to learn when the interpretation was wrong.
Sources and research trail¶
- OpenAI, “Function Calling and Other API Updates” (June 13, 2023).
- OpenAI, Chat Completions API documentation (accessed June 2023).
- National Institute of Standards and Technology, Zero Trust Architecture (2020).
- Saltzer and Schroeder, “The Protection of Information in Computer Systems” (1975).
- National Institute of Standards and Technology, Security and Privacy Controls for Information Systems and Organizations (2020).