I’ve spent 19 years at Lockheed Martin. For the last four of those, I’ve led a 17+ project portfolio of automation, AI/ML, and dashboarding solutions in Aeronautics’ Finance & Business Operations Digital Transformation organization. Here are the patterns I’ve learned about deploying AI in environments where getting it wrong has real consequences.
The Regulated-Environment Tax
In defense, you can’t ship a model and iterate based on user complaints. The compliance surface is enormous — DCMA, DFAS, EVMS, FAR/DFAR, SOX-adjacent controls. Every automation I build has to survive not just technical review but audit scrutiny. This means the evaluation methodology, the guardrails, and the audit-logging are as important as the model itself.
When I deployed an enterprise RAG pipeline for financial data classification, half the engineering effort went into the transparency artifacts — making sure every classification decision was traceable, every confidence score was logged, and every edge case had a human-review fallback.
The Production Withholds Story
The project that best illustrates this is a production withholds automation for a major defense program. The challenge: leadership needed visibility into a nine-figure withhold balance, but the source data was paragraph-form free text — inconsistent formatting, typos, abbreviations, no structured fields. The solution was a 356-tool Alteryx workflow combining ML/NLP text mining with structured field extraction, feeding a Tableau executive dashboard.
The result: 2,132 labor hours saved per year, low-six-figure losses prevented, and the dashboard is now embedded in the program’s normal business rhythm. But the real lesson was that the NLP component had to be evaluated specifically for extraction accuracy against ground truth — not just general text classification metrics. In regulated environments, the eval framework is the product.
AI at Scale
More recently, I co-led an AI-driven tax-classification initiative — a cross-business-area AI chatbot evaluated against 415,000 tax scenarios on an internal enterprise AI platform. The target is a 31% reduction in purchase-order error rates. What made this different from a typical chatbot project was the evaluation rigor: structured behavior tests, accuracy validation against ground-truth tax determinations, and benchmark datasets that stress-test edge cases.
What Transfers
The patterns that work in defense AI transfer directly to any domain where trust matters: financial services, healthcare, legal tech, and especially responsible AI governance. Build the eval framework first. Log everything. Design for auditability from day one. And never ship a model without a human-review fallback for the cases you know it will get wrong.
I’m currently exploring senior data scientist and AI/ML engineering roles where this kind of production rigor is valued. Get in touch.
Leave a Reply