Article reader Listen + reading controls
Article reader
Preparing the reader…
Reading settings
The Defense Department Does Not Need More Data—It Needs Decision-Ready Data Products¶
The Office of the Under Secretary of Defense for Research and Engineering’s 2024 “Unleashing Data at Speed and Scale” outreach asked industry for technologies spanning collection, storage, processing, monitoring, analysis, and communication. The breadth reflected a real challenge: modern military decisions depend on data moving across sensors, networks, computing environments, organizations, classifications, and national boundaries.
It also exposed a recurring acquisition risk. When a problem is described as a list of technical functions, industry responds with products optimized for individual layers. The Department can acquire faster links, larger stores, stronger processors, and more sophisticated models while the mission thread remains fragmented.
The organizing object should be neither the technology nor “the data.” It should be a decision-ready data product with an accountable owner, defined consumers, observable quality, and a mission outcome.
The outreach meeting appeared within a broader series of innovation-and-modernization events; an official 2024 overview listed the April 15–16 “Unleashing Data at Speed and Scale” solutions meeting among engagements intended to connect emerging technology with defense problems. Contemporary reporting described areas from long-distance and multipath communications to edge computing, cataloging, tagging, storage, access, and advanced transmission methods.1
The right response is not to choose one layer. It is to establish the architecture through which layers compose.
Data Has Value Only Inside a Decision¶
Defense data strategies appropriately emphasize that data should be visible, accessible, understandable, linked, trustworthy, interoperable, and secure. Those are enabling qualities, not the final outcome.
A mission team needs to know:
- which decision or action the data supports;
- who makes that decision and under what authority;
- how current, complete, and precise the data must be;
- which uncertainty matters;
- what happens when the data is late, conflicting, or unavailable;
- which partners may access or act on it;
- and how the decision’s outcome feeds back into the data product.
Without that context, “high quality” is undefined. A position report accurate to one meter may be essential for one action and unnecessary for another. A globally complete dataset may arrive too late to support a tactical choice. A precise model output may be unusable if a coalition partner cannot see the underlying evidence.
Data engineering should begin from the decision contract and work backward.
A Data Product Is More Than a Pipeline¶
A pipeline moves and transforms information. A data product accepts continuing responsibility for its utility.
A mission data product should include:
- a named product owner accountable to consumers;
- a defined mission purpose and consumer community;
- authoritative sources and semantic definitions;
- machine-readable data contracts;
- provenance and transformation lineage;
- quality, latency, availability, and security objectives;
- interfaces for discovery and access;
- policy and releasability controls;
- monitoring and incident response;
- and a feedback mechanism for changing mission needs.
The product boundary can be small: a fused track, logistics-readiness view, threat indicator, targeting-support package, or weather effect. What matters is that the team owns the end-to-end claim that the product is fit for a decision.
This model counters the tendency for infrastructure teams to optimize throughput while mission users compensate for meaning and quality through spreadsheets and personal knowledge.
The Reference Architecture Should Make Composition Testable¶
Industry outreach works best when vendors can see where a capability must fit and how success will be evaluated. A reference architecture for decision-ready data can define layers without dictating a single implementation:
- Source and observation: sensors, systems of record, human reports, and external data;
- Transport and synchronization: networks, messaging, buffering, and disconnected operation;
- Policy and identity: authentication, authorization, releasability, purpose, and audit;
- Data contracts and semantics: schemas, identifiers, vocabularies, provenance, quality, and time;
- Processing and knowledge: transformation, fusion, entity resolution, feature generation, and knowledge graphs;
- Analytics and AI: models, rules, simulation, uncertainty, and evaluation;
- Decision experience: user workflow, collaboration, explanation, human authority, and action;
- Observability and learning: telemetry, outcome measurement, incidents, and improvement.
A vendor may innovate at any layer. The Department should provide conformance tests and mission-thread scenarios that show whether the component improves the system rather than merely performing well in isolation.
Edge and Cloud Are Placement Decisions¶
Debates about edge versus cloud often become architectural identities. In practice, computation and data should be placed according to decision needs.
Relevant variables include:
- latency and time sensitivity;
- communications availability and cost;
- classification and releasability;
- compute, power, storage, and cooling;
- model and data-update frequency;
- aggregation benefit;
- survivability and attack surface;
- and need for local human control.
Some functions should operate at the edge with bounded autonomy and cached context. Others benefit from theater or enterprise aggregation. The architecture should permit graceful partitioning: local operations continue when disconnected, synchronization preserves lineage when links return, and the system makes degraded state visible.
The objective is not to send all data to the cloud or all intelligence to the edge. It is to place each transformation where it best preserves the mission decision.
Knowledge Is the Bottleneck After Bandwidth¶
Improved networks expose the next constraint: meaning.
Two systems may exchange records while disagreeing about identity, status, confidence, time, or authority. Analysts and operators resolve those differences manually through context that never enters the machine-readable layer.
A knowledge infrastructure should make that context explicit:
- common and mapped vocabularies;
- persistent identifiers for entities, events, organizations, and systems;
- temporal and geospatial relationships;
- provenance and confidence;
- policy and authority;
- and links from observations to assessments, decisions, and outcomes.
Knowledge graphs can help represent these relationships, but the graph is not the objective. The objective is to preserve meaning through transformations and make competing claims inspectable.
Vector search and generative AI are useful additions when grounded in this layer. Without it, they can retrieve plausible fragments while concealing which source, version, classification, or operational assumption applies.
Data Quality Must Be Observable in Mission Terms¶
Enterprise dashboards often report completeness, schema validity, and pipeline health. Mission teams also need decision-quality indicators:
- age of information relative to the decision window;
- coverage of the relevant population or area;
- disagreement among sources;
- known collection gaps;
- identity-resolution confidence;
- model sensitivity to missing inputs;
- and the operational consequence of a quality failure.
Quality should travel with the product. A user should not receive an apparently definitive output while material uncertainty remains buried in pipeline logs.
Observability also needs to connect technical events to mission effects. If an upstream service slows, which decisions and users are affected? Which alternative source or degraded workflow should activate? Service maps should become decision-dependency maps.
Acquisition Should Reward Integration Evidence¶
Industry can respond to broad solicitations with polished descriptions of proprietary capability. The Department should ask for evidence in a representative mission context:
- implementation against published interfaces;
- exchange using defined data and policy contracts;
- operation under latency, loss, and adversarial conditions;
- provenance through transformation;
- measured impact on a decision baseline;
- observability and failure recovery;
- and a transition plan that preserves government data and architectural freedom.
Short demonstrations can test integration hypotheses before large awards. Successful components should enter modular contracts or platform marketplaces with continuing conformance requirements.
The goal is not a winner-take-all “data platform.” It is an ecosystem in which components can improve while mission products remain coherent.
The Strategic Inference¶
The Department’s data challenge is not a shortage of technologies. It is the institutional difficulty of composing them into reliable decision systems across boundaries.
Decision-ready data products provide a unit of ownership. Reference architectures provide a unit of composition. Mission threads provide a unit of evaluation. Knowledge infrastructure preserves meaning. Observability turns operation into learning.
Industry outreach becomes far more valuable when organized around those mechanisms. Otherwise, the Department risks collecting a portfolio of advanced pieces and assigning integration to the same overextended mission teams the technology was intended to help.
Data is unleashed not when it moves faster, but when a trusted path from observation to accountable decision becomes easier to build, inspect, and improve.
This essay was substantially revised in July 2026 to replace the original event summary with an evidence-based architecture and operating-model analysis.
For related work on data products, knowledge infrastructure, and mission-scale AI engineering, visit my portfolio or Xendev Labs.
References¶
- Office of the Under Secretary of Defense for Research and Engineering, Innovation & Modernization: Overview, March 2024.
- U.S. Department of Defense, “Official Explains DOD Data Strategy,” March 5, 2021.
- Chief Digital and Artificial Intelligence Office, “CJADC2.”
-
Jon Harper, “OSD organizes industry meeting on ‘unleashing data’ for CJADC2,” DefenseScoop, January 22, 2024. This report was the historical prompt for the original post. ↩