Grounded Intelligences

Category: Enterprise AI

Deploying AI in regulated, high-stakes enterprise environments — ROI, governance, and real-world lessons.

  • Applying Lean Six Sigma to AI Projects: A Green Belt’s Framework

    As a Lean Six Sigma Green Belt who has applied DMAIC to AI and ML projects across 19 years in defense finance, the most consistent failure pattern I’ve seen is AI teams who skip the Define and Measure phases entirely and go straight to building a model. They have a dataset, they have a use case, they have a timeline — and they have no baseline, no measurable problem statement, and no control plan. The model gets deployed. Three months later, nobody can prove it’s better than what it replaced.

    DMAIC — Define, Measure, Analyze, Improve, Control — is a 60-year-old quality framework that was not designed for AI. It maps onto AI project structure almost perfectly anyway, because the failure modes it was designed to prevent are the same failure modes that kill AI deployments.

    Define and Measure: The Phases That Prevent Failure

    The Define phase requires a problem statement with a measurable Y. Not “improve efficiency in the tax processing workflow” — that’s a direction, not a target. “Reduce purchase order error rate from 31% to below 10% in the internal tax scenario classifier pipeline, measured over a 90-day post-deployment window” is a problem statement. The Y is specific, the current state is quantified, the target is defined, and the measurement window is explicit. You cannot claim success without meeting that specification. You also cannot accidentally redefine success after the fact when results are disappointing, which is the most common way AI projects survive their pilots despite not working.

    The Measure phase is where most AI projects actually fail, because if you don’t have a baseline, you can’t prove improvement — and yet most teams skip it because it delays the part that feels like progress. The protocol is to baseline the current process for 4 to 6 weeks before any automation or model is deployed, using the exact same metrics you’ll use post-deployment. The 2,132 hours per year figure from a major defense aircraft program withholds automation — which justified $360,000 in annual cost avoidance — started as a manual time study. Analysts logged their time against specific task categories for 3 weeks before any automation existed. That baseline wasn’t overhead; it was the evidence that made the business case and gave us a threshold to beat.

    Analyze and Improve: Choosing the Right Intervention

    The Analyze phase is root cause work: understanding why the problem exists before deciding what to do about it. Is the error rate high because the data is bad? Because the process is poorly designed? Because humans are making decisions under time pressure with incomplete information? Because the system produces ambiguous outputs that require judgment calls? Each of those root causes has a different correct intervention — and sometimes that intervention is a process redesign that doesn’t involve AI at all.

    This is the discipline that prevents solution-chasing. When you start with a model and work backward to the problem it solves, you’ll find a problem that matches the model’s capabilities — whether or not it’s the actual root cause of the performance gap you’re trying to close. Starting with root cause analysis in Alteryx or Python to understand the data quality and process patterns before touching a model architecture is slower in the short term and dramatically more reliable in the long term.

    The Improve phase is where the AI model enters — framed as a process change, not as a technology implementation. Design it, run a controlled pilot with a scope limited enough to measure cleanly, compare against the baseline using the metrics defined in the Measure phase, and iterate. Don’t skip the pilot. A model that performs well on a holdout test set and a model that performs well in production are not the same thing, and the difference is almost always in the human and process factors that a test set doesn’t capture.

    Control: The Phase That Makes It a Production System

    The Control phase is what separates an AI project from an AI system. A project ends at deployment. A system has ongoing monitoring, drift detection, retraining protocols, and documented ownership. Without those, you have a model that was accurate when it launched and an unknown amount of time before it’s quietly wrong in ways nobody has instrumented to detect.

    The Control plan for an AI system needs to specify: what metrics indicate the model is performing within acceptable bounds, what threshold triggers a retraining evaluation, who owns the retraining decision, and what the rollback procedure is if a retrained model performs worse than its predecessor. These aren’t optional governance niceties — they’re the difference between a system that survives its first year and one that gets quietly retired when the person who built it moves on.

    The sigma equivalent framing is useful for calibrating expectations here. A 3-sigma AI system — 99.73% accuracy — sounds impressive. It produces 2,700 errors per million decisions. If those decisions are purchase order classifications affecting a multi-hundred-million-dollar contract financial position, 2,700 errors is not an acceptable defect rate. The sigma target has to be set against the consequence of the error, not against an abstract accuracy benchmark.

    What Transfers

    Six Sigma rigor applied to AI is not methodology overhead imposed by people who don’t understand how AI development works. It’s the framework that answers the question every executive eventually asks: “How do we know this is working?” The answer to that question has to be built into the project from the Define phase — you can’t construct it after deployment from whatever data happens to be available. The 17 AI and automation projects I led at Lockheed Martin that produced defensible, sustained business value all had one thing in common: the measurement infrastructure was designed before the model was built, not after.

    The teams that resist the DMAIC discipline usually frame it as a speed concern — moving fast, not getting bogged down in process. In practice, the projects that skip Define and Measure don’t move faster. They move quickly toward a result they can’t prove, then spend months in a political argument about whether the system is working. The upfront rigor is the shortcut.

    → Building something at the intersection of AI, edge computing, and behavioral science? Let’s connect.

  • Sovereign AI in Defense: Why On-Premises Intelligence Is Making a Comeback

    After a decade of cloud migration in defense and intelligence, the line about what can run where is being redrawn — and for classified and sensitive workloads, it’s moving back toward the perimeter, not further into commercial infrastructure.

    This isn’t a reversal of cloud adoption broadly. It’s a sharper delineation of which workloads belong where. The term that’s emerged for the approach is “sovereign AI” — and understanding what it actually requires technically is useful well beyond the defense context.

    What Sovereign AI Actually Means

    Sovereign AI in its precise form means models trained on your data, running on your compute, without calling any external API at inference time. No data leaves your network. No inference request passes through a commercial provider’s infrastructure. The model weights live on hardware you control, in a facility you control, serving requests from systems that never touch the public internet.

    That’s meaningfully different from “using a private deployment of a commercial model” or “running a fine-tuned model on cloud infrastructure.” Both of those approaches still involve dependencies on external entities — the model provider, the cloud provider, the network path between your environment and theirs. Sovereign AI eliminates those dependencies by design, because the threat model requires it.

    Why It’s Mandated, Not Optional

    SECRET and TOP SECRET data cannot touch commercial cloud infrastructure under current regulations — not because the commercial providers are untrustworthy, but because the classification system predates the commercial cloud and its authorization pathways haven’t caught up with the most sensitive data categories. That’s a governance reality, not a technical judgment about cloud security.

    Beyond data residency requirements, there are three additional threat vectors that make sovereign AI architecturally necessary for the most sensitive use cases. Adversarial model extraction: a sophisticated adversary can probe an API repeatedly to reconstruct statistical properties of the training data, including potentially sensitive patterns that were never intended to be accessible externally. Supply chain risk in commercial model weights: a model trained by a commercial provider and downloaded for internal use still carries whatever was embedded in its training — including potentially undesirable behaviors or backdoors that are difficult to detect through standard evaluation. And CMMC, ITAR, and FedRAMP High impact level requirements, which create compliance obligations that some commercial AI deployments can’t satisfy regardless of their technical security posture.

    From my 19 years at Lockheed Martin, the governance question I saw teams face repeatedly was not “is this AI tool good enough?” — it was “can this AI tool even touch this data category?” That question has to be answered before any technical evaluation begins. The answer determines the entire deployment architecture.

    The Technical Reality of On-Premises Inference

    Running LLaMA 3 70B on air-gapped NVIDIA H100s inside a Sensitive Compartmented Information Facility is a very different operational challenge than calling the OpenAI API. Hardware acquisition at classified scale takes 6 to 18 months through approved procurement channels. The inference stack itself — the software serving requests to the model — requires security review before it can be deployed in a classified environment, because the software supply chain is part of the attack surface. Model updates require the same review cycle that the initial deployment did, which means rapid iteration on model capabilities is essentially impossible in the most sensitive environments.

    The interesting middle ground is what I’d call the classified edge: compact, deployable inference running on ruggedized hardware at forward positions or remote installations, with no external network dependencies at all. A quantized Mistral 7B model can run inference on hardware that fits in a Pelican case. That’s not a theoretical capability — it’s an operational requirement in environments where cloud connectivity is unavailable, unreliable, or tactically dangerous. Think of it as the Cloudflare Workers model applied to networks that can never touch the internet: push the intelligence as close to the decision point as physically possible, with no dependency on infrastructure that might not be reachable. The edge-first architecture pattern translates surprisingly well to the classified environment, once you strip away the commercial infrastructure assumptions.

    What Transfers

    The sovereign AI framework — data residency, inference independence, supply chain trust — applies in any heavily regulated industry, even when “classified” isn’t the specific label on the requirement. Healthcare AI processing PHI has data residency requirements. Legal AI handling privileged communications has confidentiality requirements that commercial cloud AI may not satisfy. Financial services AI operating under specific regulatory regimes has audit and data localization requirements that constrain what external APIs the inference pipeline can call.

    The question “where does the inference happen and who can see the inputs?” is a governance question that every enterprise AI deployment needs to answer explicitly. The defense context makes the stakes and the requirements maximally clear — but the underlying framework applies wherever the data being processed carries obligations that extend beyond the organization’s own interests. That’s a much larger category of use cases than most AI practitioners currently treat it as.

    → Building something at the intersection of AI, edge computing, and behavioral science? Let’s connect.

  • Explainability for Executives: Translating Model Outputs Into Decisions

    The meeting that clarified the explainability problem for me happened when a CFO looked at a dashboard I’d helped deploy, saw a field labeled “anomaly score: 0.87,” and asked — with complete seriousness, not as a challenge — “What does 0.87 mean? Is that good?” The room went quiet. The team had spent months on the model. They had spent approximately zero time on what the model’s outputs should mean to the person who was supposed to act on them.

    Three Levels of Explainability That Actually Matter

    Enterprise AI systems serve at least three distinct audiences, and each audience needs a different form of explanation. Conflating them — building one explanation interface and pointing everyone at it — produces a tool that serves none of them well.

    The operational level is for the analyst who runs the system daily. She needs to understand what the model did on this specific transaction or record — which inputs drove the output, whether the result is consistent with similar cases she’s reviewed, and what she should check if she wants to verify or override it. Technical detail is appropriate here. She uses the system 40 hours a week and can develop fluency with model-specific language.

    The managerial level is for the supervisor or team lead who reviews AI-flagged items and decides how to allocate analyst time. He needs to know what action to take — which flags require immediate investigation, which can wait, and how to triage a backlog of 200 alerts that all show scores above 0.7. He does not need to know how the model produces a score. He needs to know what the score implies about urgency and risk in terms he already uses to manage his team.

    The executive level is for the CFO, program director, or board member who needs to understand what the AI system means for the business — not what it does technically, but what decisions it supports, what it can’t tell you, and what it would take for you to rely on it for a consequential call. Most AI teams never build this level of explanation at all. They present the model’s performance metrics — accuracy, F1 score, precision-recall curve — to audiences who have no frame for evaluating what those numbers mean in operational terms.

    Why SHAP Values Don’t Belong in a Leadership Meeting

    SHAP values are genuinely useful for the operational level — an analyst who wants to understand why the model flagged a particular record can look at feature attributions and build intuition about the system’s behavior over time. That is a legitimate use of the tool.

    Presenting SHAP values to an executive is a category error. It’s not that executives are unsophisticated — it’s that SHAP values answer a question they’re not asking. They’re asking whether to trust the system’s recommendation on a $2M procurement decision. A bar chart showing that “vendor tenure” contributed 0.34 to the anomaly score does not answer that question. It transfers the cognitive work of interpretation to someone who shouldn’t have to do it.

    What works at the executive level is natural language that maps model outputs to business concepts they already use. “This purchase order is $142K above the 3-year average for this vendor category” is an explanation. “Anomaly score: 0.87” is an output that requires explanation. Counterfactuals work well too — “If the forecast had used Q3 actuals instead of Q2 estimates, the variance flag would not have fired” gives an executive the conditional reasoning she needs to evaluate whether the flag is meaningful or an artifact of a data lag.

    The Dashboard Redesign That Moved Adoption by 44 Points

    At Lockheed, a finance AI dashboard that had been running at 34% active adoption for three months went through an explainability redesign. The change was not to the model — the underlying system was unchanged. The change was entirely in the output layer: probability scores were replaced with natural language summaries written in the vocabulary of the finance analysts who used it. “Probability: 0.73” became “This cost pattern occurs in fewer than 8% of similar transactions — the last time this pattern appeared in this cost center, the variance was caused by a billing timing mismatch that took 11 days to resolve.”

    Two months after the redesign, active adoption was at 78%. The model hadn’t changed. The explainability layer had. Analysts who had been ignoring the dashboard because they couldn’t tell whether 0.73 warranted action were now using it because the output spoke in terms of their existing workflow and experience.

    What Transfers

    In clinical training, one of the first things you learn is to speak the client’s language rather than the DSM’s. A client describing their experience as “I get really wound up and can’t settle down” is telling you something useful. Responding with “that sounds like hyperarousal consistent with an anxiety spectrum presentation” communicates nothing to them — it’s accurate and useless. The skill is translation: taking precise clinical language and rendering it in terms the person in front of you can act on.

    AI explainability design is the same skill applied to a different context. The one-sentence test I apply before any model goes to a leadership meeting: can I explain this system’s decision in one sentence to a non-technical executive who will use that explanation to make a consequential call? If the answer is no, the model isn’t ready for that meeting — not because the model is bad, but because the explanation layer isn’t built yet. Technical completeness and operational readiness are not the same thing, and confusing them is how teams ship AI systems that sit unused at 34% adoption when they had the capacity to reach 78%.

    → Building something at the intersection of AI, edge computing, and behavioral science? Let’s connect.

  • How to Quantify AI ROI Without Misleading Your Stakeholders

    The $3M in savings slide is almost always wrong — not because the savings don’t exist, but because the methodology didn’t hold up to scrutiny when anyone actually looked.

    I’ve sat in enough enterprise finance reviews — 19 years of them at Lockheed Martin, many directly tied to AI and automation projects I led — to recognize the pattern. A team builds something that genuinely improves a process, they attach a number to the improvement, and then that number gets challenged in a budget review and can’t be defended. The project survives or doesn’t based on whether the finance organization decides to extend credibility rather than because the ROI case was actually sound.

    Two Failure Modes Before You Calculate Anything

    Underselling is the failure mode that gets less attention. ROI exists — real hours are recovered, real error rates drop — but the team has no rigorous way to measure it, so the project gets framed as qualitative improvement. Qualitative improvements don’t survive budget compression. They get cut when finance needs to find savings and the project has nothing auditable to defend itself with.

    Overselling is more visible and more damaging to the broader AI program. A team attributes all observed efficiency gains to the AI system when the gains were actually produced by a combination of the tool, a concurrent process redesign, an analyst who changed how she organized her work, and a quarter with lower transaction volume than the prior year. If you claim credit for all of it and a skeptical CFO asks how you isolated the AI contribution, you don’t have an answer. That moment — one moment — kills credibility for the next five AI proposals your organization brings forward.

    The Costs That Disappear From ROI Slides

    Every ROI methodology I’ve seen that didn’t survive scrutiny had the same structural problem: it counted only the benefits of the AI and the initial build cost, then stopped. The cost categories that routinely disappear from the slide are annotation and labeling cost for supervised systems, the retraining cadence — a model that needs quarterly retraining has a recurring engineering cost that should appear in the denominator, human review of AI outputs in any high-stakes workflow, and the cost of errors the system makes rather than only the errors it prevents.

    That last one is particularly common in automation ROI. A system that processes 10,000 transactions per month with a 0.5% error rate introduces 50 errors per month that someone has to catch and correct. If those errors are in finance or compliance workflows, the correction cost — loaded labor rate × correction time, plus any downstream rework — can meaningfully offset the efficiency gains. An honest ROI methodology puts that number in the model. A credibility-optimized ROI methodology hopes no one asks about it.

    How a Defense Finance Automation ROI Number Was Built

    The $360K annual cost avoidance figure from a major defense aircraft program withholds automation project held up to finance leadership scrutiny because it was constructed to be auditable, not to be impressive. The methodology: loaded labor rate for the analyst population multiplied by 2,132 recovered hours per year — hours that were timed, not estimated, against a documented pre-automation baseline — minus annual system maintenance cost, minus periodic retraining and validation cost for the Alteryx and Python workflows involved, minus an allocation for edge cases that still required manual intervention.

    The methodology was presented visibly, not buried in appendices. Finance leadership could see exactly which assumptions the number depended on and could stress-test any of them. When the program office asked what the number looked like if loaded labor rate was adjusted down by 15%, we had that sensitivity analysis ready. The figure that survived scrutiny was lower than the first draft — and more credible precisely because of that.

    Framing ROI by Audience

    A single ROI figure presented identically to every audience is a communication failure before it’s a methodology failure. CFOs want payback period and net present value — they’re evaluating capital allocation and need to compare this project against others competing for the same budget. Operational leaders want hours recovered per analyst per month because they’re managing capacity and need to understand what changes for their team on Monday. CTOs want the technical debt comparison — build versus buy, and what the maintenance cost trajectory looks like in years two and three when the team that built the system has moved on.

    Each of those framings uses the same underlying data. The work is disaggregating the ROI model into components that answer the specific question each audience is actually asking. Presenting a single number to all three audiences and hoping it resonates is why most AI ROI conversations stall in the room rather than producing decisions.

    What Transfers

    Financial rigor from defense FP&A is not defense-specific. The discipline of building auditable cost and benefit models — with documented baselines, explicit attribution logic, visible cost categories, and audience-specific framings — applies to any AI project that needs to survive budget scrutiny. The aerospace context made the stakes higher and the review process more formal, but the underlying methodology is the same one any enterprise AI team should be using. The first AI project that can’t defend its ROI doesn’t just lose its own budget. It makes the case against the next five.

    → Building something at the intersection of AI, edge computing, and behavioral science? Let’s connect.

  • Deploying AI in Regulated Environments: The Five Rules That Actually Matter

    After 17 AI and automation deployments in aerospace defense finance — projects where errors had audit implications, where every business case required finance leadership sign-off, and where “the model got it wrong” was not an acceptable incident report — I have a short list of rules that the standard enterprise AI guidance omits. These are not about model selection or infrastructure. They’re about the organizational and process conditions that determine whether an AI system survives first contact with production.

    Rule One: Baseline Before You Build

    You cannot prove improvement without a baseline, and most teams skip baselining because it delays the part that feels like progress. The correct protocol is to measure the current process for four to six weeks — tracking the same metrics you plan to track post-deployment — before any model or automation is introduced. This gives you a genuine before-and-after comparison that can survive scrutiny. Without it, you have a number that represents “what the system produces” with no reference point for whether that’s better, worse, or the same as what existed before.

    Rule Two: Encode Tacit Knowledge Before Automating

    Manual processes accumulate undocumented corrections. Analysts know that certain data fields mean something different in context, that certain edge cases require a judgment call not captured in any procedure, that certain values get quietly adjusted based on experience. When you automate the process without surfacing those corrections first, the automation produces outputs that disagree with what experienced humans would produce — and the team spends weeks figuring out why the automation is “wrong” when actually the automation is correct and the tacit knowledge just wasn’t encoded. The fix is process mapping sessions before build, specifically designed to surface what people actually do versus what the procedure says they do.

    Rule Three: Design for Consequential Decisions Differently

    Not all decisions in a workflow have the same consequence profile. For a major defense aircraft program withhold tracking system, it meant mapping the consequence of each automated decision and setting a dollar threshold above which human review was mandatory — not recommended, mandatory. The model could flag, explain, and rank; a person owned the decision. This design principle — automation for volume, human judgment for consequence — is not a hedge against AI capability; it’s the correct system design for any workflow where some decisions carry disproportionate accountability.

    Rule Four: Measure What the Model Gets Wrong, Not Just What It Gets Right

    Accuracy metrics that report overall performance obscure the distribution of errors. A model with 94% accuracy that fails catastrophically on a specific class of inputs may be worse than a model with 89% overall accuracy that fails more evenly. In the internal tax scenario classifier — the tax scenario processing system — we tracked error rate by category, not just overall. Some categories had error rates below 2%; others had rates above 12%. The overall accuracy number looked good; the category breakdown told us where the model couldn’t be trusted, which is the information that actually shapes how you deploy it.

    Rule Five: Write the Control Plan Before You Launch

    A deployed model without a control plan is a model with an unknown expiration date. The control plan needs to specify: which metrics indicate the model is within acceptable performance bounds, what threshold triggers a retraining evaluation, who owns the retraining decision, and what the rollback procedure is if a retrained version performs worse than its predecessor. These are not governance overhead — they’re what separates a system that maintains its accuracy over time from one that quietly degrades until someone notices the outputs are wrong.

    What Transfers

    None of these rules are specific to aerospace or defense. They apply to any organization deploying AI in a context where errors have real consequences — financial, legal, operational, or reputational. The regulated environment makes the requirements explicit; in less regulated contexts, the requirements still exist, they’re just easier to ignore until something breaks.

    → Building something at the intersection of AI, edge computing, and behavioral science? Let’s connect.

  • When AI Meets FP&A: Lessons from Automating Defense Finance

    The 2,132 hours per year figure is not an estimate. It came from a three-week manual time study conducted before any automation existed, where analysts logged their work against specific task categories while the baseline process ran unchanged. That number — and the $360,000+ in annual cost avoidance it represents — survived finance leadership scrutiny because the methodology was designed to be auditable before the first line of code was written.

    What the Problem Actually Was

    Defense program finance involves tracking a multi-hundred-million-dollar contract financial position tied to a major defense aircraft program — “withholds” being the portion of payments held back pending milestone, quality, or compliance conditions being met. Managing that position manually meant analysts reconciling data across multiple systems, building the same variance tables repeatedly, and spending a disproportionate share of their time on data assembly rather than analysis. The work wasn’t intellectually complex; it was procedurally complicated and time-consuming. That combination — low cognitive load, high time cost, clear rules — is exactly where automation performs well. The error rate in manual reconciliation was also meaningful: human error in a data assembly task with dozens of interdependent fields is not a character flaw; it’s a system design problem. Automating the assembly eliminates a category of error rather than reducing it incrementally.

    The Build: What Went In and What Didn’t

    The automation stack was Alteryx for data pipeline orchestration and Python for the more complex analytical components. Alteryx handled the extraction, transformation, and loading work: pulling from multiple source systems, applying business rules for categorization, and producing reconciled outputs. Python handled the variance detection logic — specifically, flagging variances that exceeded threshold values and generating the explanatory text that would previously require an analyst to write from scratch each cycle. What didn’t go into the automation: judgment calls. Variances above a certain dollar threshold still required an analyst to review the flag, understand the context, and decide whether escalation was warranted. Automating the detection is appropriate. Automating the decision is not, in a financial position with audit implications. The system was designed to produce recommendations, not approvals.

    What Went Wrong First

    The first version of the data pipeline produced reconciliation outputs that didn’t match the manual outputs analysts had been generating. Not because the logic was wrong — the logic was correct — but because the source data had inconsistencies that the manual process had been silently correcting through analyst judgment. Analysts knew that certain fields needed to be interpreted in context, not taken at face value, and they applied that interpretation automatically without documenting it. The automation exposed those undocumented corrections and made them explicit. The fix required going back to the analysts, documenting every correction they were making, and encoding that logic into the pipeline. This added three weeks to the build timeline and was entirely predictable if we had done more thorough process mapping upfront. The lesson isn’t that analysts were hiding things; it’s that tacit knowledge embedded in manual processes doesn’t surface until you try to replace the process with rules.

    Measuring the Outcome

    The 2,132 hours per year was calculated by comparing post-automation task logs against the pre-automation baseline, controlling for volume changes in the underlying program activity. The $360,000 figure used fully loaded labor rates — not salary alone — applied to the recovered hours, minus annual system maintenance costs and a periodic retraining allocation. The number presented to finance leadership was lower than the first draft, which made it more credible precisely because it showed the methodology rather than optimizing for the largest possible figure. Sensitivity analysis was prepared in advance: if the loaded labor rate assumption was adjusted down by 15%, the cost avoidance figure fell to $305,000. Finance could stress-test the assumption and the number still held.

    What Transfers

    Every element of this project — the time study methodology, the explicit encoding of tacit process knowledge, the threshold-based human review design, the auditable ROI methodology — transfers directly to enterprise AI work in any domain with similar characteristics: high-volume, rule-governed data tasks with meaningful error costs and financial accountability requirements. The aerospace context made the stakes and the review process more formal, but the discipline is domain-agnostic.

    → Building something at the intersection of AI, edge computing, and behavioral science? Let’s connect.

  • Deploying AI in Regulated Environments: Lessons from 17+ Enterprise Projects

    Deploying AI in Regulated Environments: Lessons from 17+ Enterprise Projects

    I’ve spent 19 years at Lockheed Martin. For the last four of those, I’ve led a 17+ project portfolio of automation, AI/ML, and dashboarding solutions in Aeronautics’ Finance & Business Operations Digital Transformation organization. Here are the patterns I’ve learned about deploying AI in environments where getting it wrong has real consequences.

    The Regulated-Environment Tax

    In defense, you can’t ship a model and iterate based on user complaints. The compliance surface is enormous — DCMA, DFAS, EVMS, FAR/DFAR, SOX-adjacent controls. Every automation I build has to survive not just technical review but audit scrutiny. This means the evaluation methodology, the guardrails, and the audit-logging are as important as the model itself.

    When I deployed an enterprise RAG pipeline for financial data classification, half the engineering effort went into the transparency artifacts — making sure every classification decision was traceable, every confidence score was logged, and every edge case had a human-review fallback.

    The Production Withholds Story

    The project that best illustrates this is a production withholds automation for a major defense program. The challenge: leadership needed visibility into a nine-figure withhold balance, but the source data was paragraph-form free text — inconsistent formatting, typos, abbreviations, no structured fields. The solution was a 356-tool Alteryx workflow combining ML/NLP text mining with structured field extraction, feeding a Tableau executive dashboard.

    The result: 2,132 labor hours saved per year, low-six-figure losses prevented, and the dashboard is now embedded in the program’s normal business rhythm. But the real lesson was that the NLP component had to be evaluated specifically for extraction accuracy against ground truth — not just general text classification metrics. In regulated environments, the eval framework is the product.

    AI at Scale

    More recently, I co-led an AI-driven tax-classification initiative — a cross-business-area AI chatbot evaluated against 415,000 tax scenarios on an internal enterprise AI platform. The target is a 31% reduction in purchase-order error rates. What made this different from a typical chatbot project was the evaluation rigor: structured behavior tests, accuracy validation against ground-truth tax determinations, and benchmark datasets that stress-test edge cases.

    What Transfers

    The patterns that work in defense AI transfer directly to any domain where trust matters: financial services, healthcare, legal tech, and especially responsible AI governance. Build the eval framework first. Log everything. Design for auditability from day one. And never ship a model without a human-review fallback for the cases you know it will get wrong.

    I’m currently exploring senior data scientist and AI/ML engineering roles where this kind of production rigor is valued. Get in touch.