After a decade of cloud migration in defense and intelligence, the line about what can run where is being redrawn — and for classified and sensitive workloads, it’s moving back toward the perimeter, not further into commercial infrastructure.
This isn’t a reversal of cloud adoption broadly. It’s a sharper delineation of which workloads belong where. The term that’s emerged for the approach is “sovereign AI” — and understanding what it actually requires technically is useful well beyond the defense context.
What Sovereign AI Actually Means
Sovereign AI in its precise form means models trained on your data, running on your compute, without calling any external API at inference time. No data leaves your network. No inference request passes through a commercial provider’s infrastructure. The model weights live on hardware you control, in a facility you control, serving requests from systems that never touch the public internet.
That’s meaningfully different from “using a private deployment of a commercial model” or “running a fine-tuned model on cloud infrastructure.” Both of those approaches still involve dependencies on external entities — the model provider, the cloud provider, the network path between your environment and theirs. Sovereign AI eliminates those dependencies by design, because the threat model requires it.
Why It’s Mandated, Not Optional
SECRET and TOP SECRET data cannot touch commercial cloud infrastructure under current regulations — not because the commercial providers are untrustworthy, but because the classification system predates the commercial cloud and its authorization pathways haven’t caught up with the most sensitive data categories. That’s a governance reality, not a technical judgment about cloud security.
Beyond data residency requirements, there are three additional threat vectors that make sovereign AI architecturally necessary for the most sensitive use cases. Adversarial model extraction: a sophisticated adversary can probe an API repeatedly to reconstruct statistical properties of the training data, including potentially sensitive patterns that were never intended to be accessible externally. Supply chain risk in commercial model weights: a model trained by a commercial provider and downloaded for internal use still carries whatever was embedded in its training — including potentially undesirable behaviors or backdoors that are difficult to detect through standard evaluation. And CMMC, ITAR, and FedRAMP High impact level requirements, which create compliance obligations that some commercial AI deployments can’t satisfy regardless of their technical security posture.
From my 19 years at Lockheed Martin, the governance question I saw teams face repeatedly was not “is this AI tool good enough?” — it was “can this AI tool even touch this data category?” That question has to be answered before any technical evaluation begins. The answer determines the entire deployment architecture.
The Technical Reality of On-Premises Inference
Running LLaMA 3 70B on air-gapped NVIDIA H100s inside a Sensitive Compartmented Information Facility is a very different operational challenge than calling the OpenAI API. Hardware acquisition at classified scale takes 6 to 18 months through approved procurement channels. The inference stack itself — the software serving requests to the model — requires security review before it can be deployed in a classified environment, because the software supply chain is part of the attack surface. Model updates require the same review cycle that the initial deployment did, which means rapid iteration on model capabilities is essentially impossible in the most sensitive environments.
The interesting middle ground is what I’d call the classified edge: compact, deployable inference running on ruggedized hardware at forward positions or remote installations, with no external network dependencies at all. A quantized Mistral 7B model can run inference on hardware that fits in a Pelican case. That’s not a theoretical capability — it’s an operational requirement in environments where cloud connectivity is unavailable, unreliable, or tactically dangerous. Think of it as the Cloudflare Workers model applied to networks that can never touch the internet: push the intelligence as close to the decision point as physically possible, with no dependency on infrastructure that might not be reachable. The edge-first architecture pattern translates surprisingly well to the classified environment, once you strip away the commercial infrastructure assumptions.
What Transfers
The sovereign AI framework — data residency, inference independence, supply chain trust — applies in any heavily regulated industry, even when “classified” isn’t the specific label on the requirement. Healthcare AI processing PHI has data residency requirements. Legal AI handling privileged communications has confidentiality requirements that commercial cloud AI may not satisfy. Financial services AI operating under specific regulatory regimes has audit and data localization requirements that constrain what external APIs the inference pipeline can call.
The question “where does the inference happen and who can see the inputs?” is a governance question that every enterprise AI deployment needs to answer explicitly. The defense context makes the stakes and the requirements maximally clear — but the underlying framework applies wherever the data being processed carries obligations that extend beyond the organization’s own interests. That’s a much larger category of use cases than most AI practitioners currently treat it as.
→ Building something at the intersection of AI, edge computing, and behavioral science? Let’s connect.
Leave a Reply