Article reader Listen + reading controls
Article reader
Preparing the reader…
Reading settings
Do Not Buy a Data Platform; Build a Data Capability Market¶
When the Department of Defense asks industry for technologies covering collection, transport, storage, processing, monitoring, and analysis, it encounters a predictable commercial response: every vendor explains why its platform should become the center of the architecture.
The Department’s problem is not a lack of platforms. It is the cost of composing capabilities across programs, vendors, clouds, classifications, and missions. Selecting another comprehensive product may solve a local integration problem while creating a larger dependency.
The alternative is not to build everything internally. It is to create a data capability market: a governed environment in which products can compete at stable architectural seams and must demonstrate evidence against shared mission threads.
The 2024 “Unleashing Data at Speed and Scale” solutions meeting offered a venue for technology discovery. The enduring value of such outreach depends on what happens before and after the presentation: whether the government publishes a usable problem and reference environment, and whether successful technology can enter a funded transition path without becoming a bespoke integration project.
Publish Mission Problems as Executable Challenges¶
A broad statement such as “process data at scale” does not give industry enough context to optimize for mission value. A useful challenge package should include:
- a representative mission thread;
- users and decision points;
- synthetic or releasable data with known limitations;
- interface and security constraints;
- latency, availability, and resilience objectives;
- adversarial and degraded conditions;
- baseline performance and cost;
- evaluation metrics;
- and transition assumptions.
The package becomes executable when a vendor can connect a component to a reference environment and reproduce the government’s tests. This lowers the advantage of incumbents whose primary asset is tacit knowledge of undocumented integration.
Define the Seams Before Choosing the Product¶
Architecture decisions made during source selection tend to favor the winning vendor’s product boundaries. The government should define durable seams first.
Candidate seams include:
- ingestion and messaging;
- identity and policy decision;
- metadata and catalog;
- data contract and semantic services;
- transformation and orchestration;
- storage and query;
- model serving and evaluation;
- observability;
- and user-facing mission applications.
Not every seam needs a separate contract. Too much fragmentation can destroy end-to-end accountability. The point is to identify where replacement and competition are strategically valuable, then publish interfaces and conformance evidence at those boundaries.
The system integrator remains accountable for the mission thread. Component suppliers remain replaceable.
Government-Controlled Reference Environments Create Leverage¶
A reference environment can include:
- interface specifications and schemas;
- identity and access stubs;
- policy test cases;
- representative event streams;
- data-quality and provenance scenarios;
- model and analytic interfaces;
- observability conventions;
- and automated conformance tests.
Vendors can demonstrate integration without receiving sensitive operational access. Government teams can compare products on common evidence. New entrants can understand the environment without relying on an incumbent prime.
The reference environment should be versioned and informed by real mission systems. It is not a toy sandbox; it is an abstraction of the contracts that matter.
Evaluate the Marginal Cost of Integration¶
Vendor demonstrations often emphasize time to first success. The strategic measure is the cost of the next integration.
The Department should ask:
- How much effort does it take to add a new source, user community, model, or mission partner?
- How many mappings are reusable?
- Which changes require vendor professional services?
- Can another supplier reproduce the integration from documentation and tests?
- Does the platform preserve provenance across external components?
- Can data and configurations be exported in usable form?
- How does cost grow with volume, domains, and classification levels?
A platform that demonstrates quickly through proprietary connectors may accumulate integration debt faster than a smaller component built around explicit contracts.
Buy Evidence and Options¶
Early contracts should purchase learning:
- integration against the reference environment;
- performance under representative conditions;
- security and supply-chain evidence;
- operational telemetry;
- data and configuration portability;
- and documentation sufficient for independent reproduction.
Successful prototypes should create options, not automatic entitlement to an enterprise award. The government can expand scope where evidence supports it, integrate a component into an existing platform, or stop when value is insufficient.
Contract terms should preserve the right to replace components, retain government-generated data, and use evaluation results across follow-on competitions.
A Marketplace Needs Governance¶
A marketplace is not simply a catalog of approved products. It needs rules governing entry, continued participation, and composition.
Those rules can require:
- conformance to interfaces and telemetry standards;
- current security and assurance evidence;
- transparent pricing and resource consumption;
- version and change notification;
- incident reporting;
- performance against relevant mission profiles;
- and exit or migration support.
Products should carry scope. A capability approved for one data sensitivity, scale, or mission is not automatically suitable for another. Evidence should be reusable but not generalized beyond its conditions.
The CDAO’s Open DAGIR approach, announced later in 2024, similarly emphasized a multi-vendor ecosystem for data, analytics, and AI rather than a closed single-provider architecture. The architectural principle is broader than any particular initiative: competition requires technical and contractual seams at which alternatives can actually enter.
Integration Teams Are a Public Capability¶
Markets do not compose themselves. The Department needs government and government-accountable product teams that understand mission workflows, data semantics, platform architecture, cybersecurity, and acquisition.
These teams should own:
- reference architectures;
- interface and semantic governance;
- test environments;
- mission-thread evaluation;
- portfolio decisions;
- and the backlog of integration friction revealed by users.
Contractors can contribute essential expertise, but the government must retain enough knowledge to make architectural and risk decisions independently. Otherwise, the “market” is governed by whichever vendor controls integration.
The Strategic Inference¶
Broad industry outreach creates value when it increases the Department’s option set. It creates dependency when each engagement leads to another vertically integrated platform.
A data capability market changes the competition. Vendors still differentiate through performance, usability, security, and innovation. They do so inside an architecture whose critical interfaces, mission tests, evidence expectations, and transition mechanisms remain under public control.
The Department should not ask which company can own the data problem. It should ask which components improve a defined mission thread—and whether the next competitor can enter without rebuilding the enterprise.
That is how innovation becomes cumulative rather than episodic.
This essay was substantially revised in July 2026 to replace a second, duplicative meeting summary with a distinct analysis of modular acquisition and market architecture. It incorporates Open DAGIR, announced after the original post, as later supporting evidence.
For organizations building modular data and AI ecosystems around legacy mission estates, visit Xendev Labs or my broader portfolio.
References¶
- Office of the Under Secretary of Defense for Research and Engineering, Innovation & Modernization: Overview, March 2024.
- U.S. Department of Defense, “CDAO Announces New Approach to Scaling Data, Analytics and AI Capabilities,” May 30, 2024.
- “Department of Defense in Search of Technologies for Collecting, Processing, Transmitting Data,” ExecutiveGov, January 2024. This report was the historical prompt for the original post.