Article reader Listen + reading controls
Article reader
Preparing the reader…
Reading settings
Model choice creates a portfolio to govern¶
Amazon Web Services (AWS) has announced Amazon Bedrock, a managed service intended to give customers access to foundation models from several providers through a common cloud environment. The appeal is obvious: teams can experiment with different models and choose the capability that fits the application without building the underlying infrastructure themselves.
Choice reduces dependence on a single model. It also creates a portfolio that somebody has to govern.
“Best model” is not a stable category¶
Foundation models differ in capability, latency, price, context length, supported modalities, customization, data handling, and safety behavior. Those differences will keep changing. A model that performs best for one task today may be unnecessarily large for another or surpassed before the product reaches full deployment.
The selection question should therefore be framed as fit for purpose, not overall superiority.
For each use, an organization needs to know what properties matter and which evidence supports the choice. A customer-facing assistant may prioritize predictable refusal behavior and low latency. A technical summarization tool may prioritize traceability to source material. A disconnected edge application may require a smaller model that can operate locally.
Artificial intelligence (AI) strategy becomes portfolio management when these tradeoffs are made across many applications.
A common interface can hide important differences¶
A managed platform simplifies access, but it can make models appear more interchangeable than they are. The application programming interface (API) may look consistent while output behavior, licensing, training-data disclosures, and acceptable-use constraints differ substantially.
That is a familiar architectural tension. Standardization lowers integration cost; abstraction can conceal the very properties that determine risk.
The answer is not to reject the abstraction. It is to pair it with a model registry that records what the common interface does not. At minimum, the registry should include provider, version, intended uses, prohibited uses, data terms, evaluation results, known limitations, operational dependencies, and applications that consume the model.
The National Institute of Standards and Technology (NIST) Artificial Intelligence Risk Management Framework calls for inventories and documentation appropriate to the organization's context. A multi-model platform makes that inventory a practical control, not administrative housekeeping.
Selection must be reproducible¶
Teams often choose technology through informal demonstrations. A compelling output wins attention, then becomes architecture. That process is particularly unreliable for generative models because a few prompts reveal little about variation, failure, or cost at scale.
A model-selection harness should run the same representative tasks across candidates and record:
- task success and important failure modes;
- consistency across repeated runs;
- latency and estimated operating cost;
- security and privacy constraints;
- user ability to recognize and correct errors;
- behavior under adversarial or ambiguous input; and
- portability of prompts, retrieval, and surrounding code.
Domain experts should help define the test set and judge failures. Otherwise the platform team may optimize a score that has little relationship to the work.
Govern replacement from the beginning¶
The benefit of model choice appears only if applications can exercise it. Product teams should separate business rules, retrieval, evaluation, and user experience from provider-specific calls. They should also define what would trigger reconsideration: material price change, performance drift, a new risk, a provider retirement, or a better fit.
Replacement cannot be entirely automatic. A new model may pass technical tests while changing tone, uncertainty, or user trust. The transition should be treated as a product change with regression evidence and communication.
Amazon Bedrock signals a healthy evolution in the market: organizations will be able to choose among models rather than equate AI strategy with one vendor. But optionality is not a strategy by itself. It becomes strategic when the organization can explain why each model is in the portfolio, where it is used, what evidence justifies it, and how it will be replaced when the answer changes.
Sources and research trail¶
- Amazon Web Services, “Announcing New Tools for Building With Generative AI on AWS” (April 13, 2023).
- National Institute of Standards and Technology, Artificial Intelligence Risk Management Framework (AI RMF 1.0) (2023).
- Amershi et al., “Software Engineering for Machine Learning” (2019).
- Sculley et al., “Hidden Technical Debt in Machine Learning Systems” (2015).