Grounded Intelligences

Blog

  • The AI Roles Worth Targeting in 2026: A Field Guide for Unconventional Candidates

    I spent 19 years in defense finance and digital transformation before becoming someone companies want for AI work, and here is what I see in 2026: the most interesting AI roles are not being filled by candidates submitting the standard machine learning pipeline resume. They are being filled by people who understand both the technology and the domain it’s being applied in. The pure-ML resume is now common enough to be a commodity. The combination is rare enough to be a differentiator.

    Where Domain Expertise Beats Pure ML Background

    Several roles reward domain depth over model depth. The AI Solutions Architect needs system design thinking and domain credibility — the job is fitting AI into a real organization with real constraints, and knowledge of model internals alone doesn’t get you there. The AI Product Manager needs to understand user behavior and organizational dynamics; candidates from a pure ML background often struggle here precisely because the hard part isn’t the model. The AI Ethicist or Responsible AI Lead draws on philosophy, law, social science, and clinical backgrounds — none of which appear on a standard ML résumé. And the ML Evaluator or RLHF Specialist needs cognitive science, linguistics, and domain expertise far more than systems engineering. In each case the rare skill is the human and organizational understanding, not the technical baseline.

    A Case Study in Recombination

    My own path is the argument. A Licensed Marriage and Family Therapist license, a master’s in Industrial/Organizational Psychology, 19 years in defense finance at Lockheed Martin, and a track record building AI systems that produced quantified outcomes — $360K in annual cost avoidance on a major defense aircraft program Withholds automation, 415,000 tax scenarios processed in 4 hours through the internal tax scenario classifier. None of those pieces gets you the job on its own. A therapist isn’t an AI engineer; an I/O psychologist isn’t a finance lead; a defense finance background isn’t responsible-AI expertise. Together, though, they represent something genuinely uncommon: a person who can build the system and understand the human, organizational, and regulatory context it has to live inside.

    What Companies Want but Can’t Articulate

    What organizations are actually hiring for — even when they can’t put it in a job description — is someone who will catch the problems that emerge at the intersection of the model and the organization. The model that is technically correct but organizationally wrong. The system that is accurate but not trusted by the people meant to use it. The tool that performs beautifully in evaluation and fails in deployment because nobody accounted for how humans actually behave around it. Those failures don’t show up in a model card. They show up six months after launch, and the person who can foresee them is worth more than the person who can squeeze another point of accuracy out of a training run.

    How to Position as an Unconventional Candidate

    The move that works is building in public. Every project you document is evidence — not a claim about what you could do, but a record of what you did. This site is the argument: posts about building real systems, quantified outcomes, and the behavioral science lens most AI engineers don’t carry. A working job search dashboard at jobs.groundedintelligences.com tells a story no resume bullet can, because it’s a live system someone can actually use and inspect. When the standard candidate hands over a list of skills, the unconventional candidate hands over a working thing and says: I built that, here’s what it does, here’s what it cost, here’s what it produced.

    The Most Important Credential in 2026

    The credential that matters most now is a portfolio of working AI systems with documented outcomes. Not a certification, not a bootcamp completion, not a GitHub repository full of Jupyter notebooks that ran once on a laptop — working systems with real usage and quantified results. The bar has risen because building has gotten easier, which means the proof of competence is no longer knowing how but having done. A deployed system with users and numbers settles the question that a credential only gestures at. I wrote about the discipline behind shipping those systems in lessons from 17 enterprise projects.

    What Transfers

    These positioning principles apply in any field where the standard path is well-worn and the non-standard path creates genuine differentiation. When everyone arrives with the same resume, the credential stops distinguishing anyone — and the person who combines an unusual background with demonstrable work becomes legible in a way the conventional candidate can’t. Build the thing, document the outcome, and let the combination of skills nobody else has stacked become the reason you’re the obvious hire. The unconventional path is only a liability until you make it the point.

    → Building something at the intersection of AI, edge computing, and behavioral science? Let’s connect.

  • Building My Career Intelligence Dashboard: A Personal AI Job Search Engine

    I built a fully functional career intelligence platform running at jobs.groundedintelligences.com at zero hosting cost, doing AI-powered job matching and processing hundreds of job postings every day. It is not a demo or a prototype — it ingests live job data, scores it against my profile, and surfaces the best matches every morning, all on Cloudflare’s free tiers. Here is how it’s put together and the one design decision that mattered most.

    The Stack

    The whole thing sits on Cloudflare primitives. A Worker acts as the API server. D1 holds the job database. KV handles rate limiting and session caching. R2 stores resume versions and company logos. Workers AI generates the embeddings that drive job-to-resume matching. That’s five services, each on its free tier, composing into a real application — no virtual machines, no container bills, no separate database host. The same architecture approach underlies my personal finance PWA on Cloudflare Workers, which runs on the same foundation.

    Data Sources

    The data comes from three kinds of sources. Job board APIs — Greenhouse, Lever, and Ashby all expose free public endpoints — provide the bulk of structured listings. Curated RSS feeds from the careers pages of target companies fill in the roles those APIs miss. And a custom scraper Worker handles sites that have no API at all, pulling listings on a schedule and depositing them into D1 alongside the structured data. Together these feed a steady stream of new postings without any paid data subscription.

    The ICP Scoring System

    At the center is an Ideal Candidate Profile stored in D1 that weights the factors I care about — industry, company stage, required-background match, title level, and location preference. Every ingested job gets an ICP score from 0 to 100, calculated as the cosine similarity between the job’s embedding and the profile’s embedding, both produced by Workers AI with bge-base. The embedding approach means the system understands a role described in unfamiliar wording, not just one that happens to share keywords with my profile. A job that never says “applied AI” but describes exactly that work still scores high.

    The Matching Interface and Feedback Loop

    The front end is a PWA served by a Worker, showing jobs ranked by ICP score — each with a three-sentence AI summary explaining why the role matches my profile, generated by Workers AI running LLaMA 3 8B over the job description and the ICP. When I rate a job with a thumbs up or down, that signal feeds back into the ICP score through a simple update, so the system gets sharper the more I use it. The feedback loop is deliberately lightweight; it doesn’t retrain anything, it just nudges the profile toward what I’ve actually shown interest in versus what only looked good on paper.

    The Real Numbers

    The system ingests 200 to 400 new jobs daily across all sources. Every morning it surfaces the top 12 matches through the PWA — a curated shortlist instead of a firehose. The processing cost is $0 per month, because every component lives inside a free tier and the daily volume never approaches the limits. A dozen well-ranked roles each morning, drawn from several hundred raw postings, is a fundamentally different experience than scrolling a job board for an hour, and it costs nothing to run.

    The Decision That Mattered Most

    The single best design decision was not trying to scrape LinkedIn. It would have been a constant fight against anti-scraping measures, and it would have pulled in noise from every direction. The job boards with open APIs — Greenhouse, Lever, Ashby — are used by the kinds of companies that build things with APIs. That is exactly the target company profile for my own search. The constraint became a filter: by limiting intake to companies that expose clean public endpoints, I automatically biased the whole pipeline toward technically serious organizations. The thing I chose not to build improved the results more than another feature would have.

    What Transfers

    The personal intelligence tool architecture generalizes to any domain where you’re sorting through high-volume, heterogeneous inputs to find relevant signals. Scheduled ingestion from multiple sources, embedding-based matching against a profile of what you care about, and a lightweight feedback loop that sharpens the profile over time — that pattern works for jobs, for monitoring news, for tracking research, for anything where the problem is too much input and not enough relevance. Build the profile, embed the inputs, rank by similarity, and let your own feedback tune it.

    → Building something at the intersection of AI, edge computing, and behavioral science? Let’s connect.

  • Why Agentic AI Needs Behavioral Guardrails, Not Just Technical Ones

    Across 17-plus AI and automation projects at Lockheed Martin, I learned a distinction that the agentic AI conversation in 2026 still mostly misses: technical guardrails tell an AI agent what it cannot do, while behavioral guardrails shape what it is inclined to do. The first is rate limits, tool restrictions, and output filters — easy to specify. The second is harder to specify and far more important to get right, because an agent that stays inside its technical limits can still pursue the wrong thing with great efficiency.

    Three Failure Modes Technical Guardrails Miss

    The first is goal displacement — the agent optimizes for a proxy metric rather than the actual goal, chasing task completion rate instead of user benefit. The second is authority escalation, where the agent progressively expands its scope without explicit permission because each adjacent action seems helpful in the moment. The third is irreversibility errors — actions that can’t be undone, like sending an email, making a purchase, or modifying a file, taken without enough certainty to justify the permanence. None of these are technical violations. The agent is doing what it was told. The problem is what it was inclined to do with that instruction.

    Goal Displacement in Practice

    Picture an agent tasked with “schedule meetings efficiently.” If efficiency is the proxy it optimizes, the most efficient path to a clear calendar is to decline invitations — so it starts declining them without asking. Technically it is executing the goal flawlessly. Behaviorally it is doing the opposite of what the user wanted, because the user wanted meetings scheduled well, not avoided. The gap between the stated metric and the actual intent is where goal displacement lives, and no rate limit catches it.

    The Clinical Parallel: Supervision and Drift

    In therapy supervision, we use structured protocols not because therapists are untrustworthy but because protocols protect against subtle drift. A therapist working without supervision can slowly drift toward the client’s framing of a problem without realizing it’s happening — each small accommodation seems reasonable, and the cumulative shift is invisible from the inside. The same dynamic operates in agents over long sessions. Each adjacent action looks justified, and the agent drifts from its original mandate one reasonable step at a time. Supervision in both cases is an external check against accumulated micro-decisions that individually pass review.

    Behavioral Guardrail Design Principles

    Four principles shape agents that resist these failure modes. Agents should narrate their reasoning before acting, not just log it after — the narration is a checkpoint, not an audit trail. They should flag irreversible actions and require explicit confirmation, treating permanence as a category that always pauses for a human. They should surface uncertainty rather than proceeding under ambiguity, because the confident wrong action is the expensive one. And they should hold a defined scope statement and check against it at each decision point, so drift has something to be measured against. I built on the behavioral-science foundations for this in my post on why a therapist builds AI systems.

    The Minimum Footprint Principle

    Agents should default to the smallest action that accomplishes the goal. If the goal can be met with a read operation, the agent should not write. If it can be met with a message, the agent should not take an action that changes state. This is a design choice, not an emergent behavior — left to their own optimization, agents tend toward more action, not less, because action feels like progress. The minimum footprint principle has to be engineered in deliberately, the same way a careful clinician does the least intervention that helps rather than the most intervention they can justify.

    What Transfers

    Clinical ethics frameworks map directly onto agentic AI design. Autonomy, beneficence, non-maleficence — these are not abstractions when you’re deciding whether an agent should send the email or ask first. The question “is this what the user actually wants, or what I think they want?” is as live in an agent’s decision loop as it is in a therapy session. The field that has spent a century formalizing how to act on someone else’s behalf without overstepping has a working framework, and agentic AI would do well to borrow it rather than reinvent it badly.

    → Building something at the intersection of AI, edge computing, and behavioral science? Let’s connect.

  • Fine-Tuning LLMs on Proprietary Data: The Annotation Quality Problem

    A LoRA fine-tune of Mistral 7B on 10,000 examples cost me under $50 on RunPod in 2026 — the compute is cheap and getting cheaper. The bottleneck moved years ago, and most teams haven’t noticed. It is no longer the training run. It is the quality of the data you feed into it, and annotation quality is where fine-tuning projects quietly fail.

    What Annotation Quality Actually Means

    Annotation quality is not just “did the annotators agree with each other.” It is three harder questions. Did the annotation rubric actually capture the task you care about, or a convenient proxy for it? Were annotators calibrated against a shared set of examples before they started labeling, or did they each invent their own interpretation? And were the edge cases documented as they came up, or silently resolved and forgotten? Agreement on a bad rubric just gives you consistent noise. The quality lives in the rubric and the calibration, not in the raw agreement number alone.

    Inter-Rater Reliability as the Starting Gate

    Cohen’s kappa is the first gate, and I treat it as a gate rather than a vanity metric. A kappa below 0.70 means your annotators are not reliably applying the same rubric — which means your fine-tuning data carries the disagreement as noise straight into the weights. A kappa below 0.60 tells you something more serious: the rubric itself probably needs redesign, not just more annotator calibration. The distinction matters because the fix is different. Below 0.70 you retrain the people. Below 0.60 you rewrite the instructions.

    The internal tax scenario classifier Annotation Workflow

    Building the internal tax scenario classifier meant creating more than 10,000 training examples for tax scenario classification, and the workflow reflected everything above. Three annotators reviewed each batch. Every batch opened with a pre-calibration session using 50 canonical examples, so everyone was anchored to the same interpretation before touching new data. A resolution protocol handled disagreements — an adjudicator reviewed any annotator pair that fell below a kappa of 0.75. The first 3,000 examples took six weeks. The annotating itself was fast; what was slow was resolving rubric ambiguities, the genuinely hard classification calls where reasonable experts disagreed and the instructions had to be sharpened before we could proceed. That slowness was the work paying off, not the work failing.

    The Failure Mode

    The common failure is predictable. A team rushes annotation, skips the pre-calibration session, skips the inter-rater checks, and ships a large pile of labeled examples on a deadline. Then they fine-tune and discover the model is worse than the base model on their own task. They blame the model, the framework, the learning rate — anything but the data. The model learned exactly what it was given, which was noise dressed up as signal. There is no hyperparameter that recovers from a corrupted training set.

    The Quality Floor

    Here is the number that should reshape how teams budget annotation effort: 1,000 high-quality annotated examples with a kappa above 0.80 beats 20,000 noisy ones for fine-tuning. This is consistent across the fine-tuning literature and consistent with my own production experience. The instinct to gather more data is usually the wrong instinct. The right instinct is to make the data you already have cleaner — tighten the rubric, recalibrate the annotators, and accept a smaller, sharper training set over a large, blurry one.

    Documenting Edge Cases as Your Test Set

    For every annotation batch, I require documenting the 20 to 30 cases that were hardest to classify and why they were hard. Those documented edge cases become the test set for evaluating the fine-tuned model — deliberately, not random samples pulled from the annotation pool. A random test set tells you how the model does on average. The hard-case test set tells you how it does exactly where it’s most likely to break, which is the only place where evaluation earns its keep. The same rigor shows up across regulated deployments, which I wrote about in lessons from 17 enterprise projects.

    What Transfers

    This is clinical assessment methodology applied to machine learning annotation. Structured clinical interviews, rubric-based scoring, inter-rater reliability checks — all of these exist in clinical psychology, and they exist there for exactly the same reasons they matter in annotation. When you need multiple humans to apply the same judgment consistently to ambiguous material, the discipline is identical whether the material is a patient presentation or a tax scenario. The field that has spent decades measuring human agreement has a great deal to teach the field that just discovered it needs to.

    → Building something at the intersection of AI, edge computing, and behavioral science? Let’s connect.

  • Trust Calibration in Human-AI Systems: What the Therapy Room Taught Me

    In nineteen years as a Licensed Marriage and Family Therapist, I noticed a pattern that took me a long time to name: the patients who trusted me unconditionally were the ones not making progress. They were outsourcing their judgment to me instead of building their own. The therapeutic goal was never maximum trust — it was calibrated trust, and that same distinction turns out to be the central design problem in every human-AI system I’ve built since.

    The Same Dynamic in Human-AI Systems

    Over-trust and under-trust are both failures, and they fail in opposite directions. Over-trust — what the research calls automation bias — leads users to accept AI outputs they should have scrutinized, because the system seems authoritative and questioning it takes effort. Under-trust — algorithm aversion — leads users to ignore correct AI recommendations after a single bad experience, falling back on worse human judgment. A well-designed system has to fight both at once, which means the target is not somewhere on a line between trust and distrust. The target is trust that tracks reality.

    Defining Calibrated Trust

    Calibrated trust is trust that scales with the AI’s actual reliability in a specific domain and a specific task type. It is not a global setting. A model can be excellent at classifying tax scenarios and useless at predicting human behavior, and a calibrated user trusts it accordingly — high confidence in the first domain, near-zero in the second. The work of design is making that task-specific, domain-specific reliability visible, so the user’s trust has something accurate to attach to instead of a vague sense that the machine is smart.

    What the Automation Bias Research Shows

    Mosier and Skitka’s automation bias studies, adapted from aviation, found something counterintuitive — users in automated environments made more errors not by actively choosing wrong, but by passively accepting incorrect system outputs. The automation seemed authoritative, so the human stopped checking. The error was an error of omission, a failure to verify, which is exactly the dynamic I watched in the therapy room with patients who stopped examining their own thinking because they assumed I had already done it for them.

    The Other Failure: Algorithm Aversion

    Dietvorst, Logg, and colleagues documented the mirror image. After watching an algorithm fail just once, users preferred worse human judgment over better algorithmic judgment — even when the algorithm’s overall accuracy was demonstrably higher. One visible mistake erased a track record of being right. That asymmetry matters for design, because it means a single uncaught error doesn’t just cost you that decision — it can collapse a user’s willingness to use a tool that is, on balance, helping them.

    Design Principles for Calibrated Trust

    Three principles have held up across my builds. Show confidence scores anchored to historical calibration — not a bare percentage, but a statement like “this model is correct 87% of the time on cases like this one,” which gives the number meaning. Surface the cases where the model tends to underperform, so users learn its blind spots before they get burned by one. And require human sign-off on high-stakes decisions regardless of confidence, because some decisions need an accountable person no matter how good the model is. I explored the behavioral-science foundation for this approach in my post on why a therapist builds AI systems.

    The Lockheed Example

    On a major defense aircraft program, the variance detection model I helped build required an analyst sign-off on any flag above $500,000. That threshold had nothing to do with the model being untrustworthy — its accuracy was strong. It existed because the accountability structure demanded a human decision-maker in the loop for material amounts. The model could surface the flag, explain its reasoning, and rank it by confidence, but a person owned the decision. That is calibrated trust built into process: automation for the volume, human judgment for the consequence.

    What Transfers

    Calibrated trust is the design goal — not maximum trust, not zero trust, but trust that matches demonstrated reliability. Whether the relationship is between a patient and a therapist or a user and a model, the healthy state is the same: confidence that rises and falls with actual track record, and a structure that keeps the human’s judgment engaged rather than dormant. Build systems that earn trust where they deserve it and invite scrutiny where they don’t.

    → Building something at the intersection of AI, edge computing, and behavioral science? Let’s connect.

  • Cloudflare Workers AI in 2026: Running Free Inference at the Edge

    Running LLM inference at 300 locations worldwide for free sounds like marketing copy — but it’s actually the production architecture I use for lightweight classification and embedding tasks across my fleet of 60-plus Cloudflare Workers, every one of them sitting at $0 per month. I want to walk through what Workers AI actually delivers in 2026, what the free tier buys in real terms, and where this approach stops making sense.

    The 2026 Model Catalog

    The Workers AI catalog in 2026 covers most of what an edge application needs without reaching for an external API. For text generation you have LLaMA 3 in both 8B and 70B variants, Mistral 7B for faster lighter work, and Gemma 2B when you need the smallest possible footprint. For embeddings, BAAI/bge-base-en-v1.5 produces 768-dimension vectors that I store directly in D1. Whisper-large-v2 handles transcription. The point is that a single Worker can generate text, embed documents, and transcribe audio without a single outbound fetch to a third-party service — the model runs on Cloudflare’s GPUs at the edge node closest to the request.

    What the Free Tier Actually Buys

    Workers AI bills in “neurons” — a normalized compute unit that abstracts away the differences between a tiny embedding call and a heavy 70B generation. The free tier gives you 10,000 neurons per day. That abstraction is confusing until you map it to real work, so here are the numbers from my own usage logs. Ten thousand neurons buys roughly 5,000 LLaMA 3 8B inference calls in a day, or about 50,000 embedding generations with bge-base, or close to 300 minutes of Whisper transcription. For a personal tool or a low-traffic production service, that ceiling is rarely the constraint — I have Workers that have never come close to exhausting a day’s allotment, which is how the whole fleet stays at zero cost.

    Production Use Cases From the Fleet

    The clearest example is daniel-job-search, my career intelligence PWA. It uses Workers AI with bge-base to generate embeddings for every incoming job description and for my Ideal Candidate Profile, then runs cosine similarity in D1 to score matches. The entire matching pipeline runs inside a Worker — no external API calls, no per-token billing, no network hop to a model provider. Digital Sentinel, my OSINT monitoring system, uses Workers AI text classification to triage alerts before routing the important ones to Discord, which means the noise filtering happens at the edge instead of clogging a downstream queue. I wrote more about that build in my post on running OSINT monitoring on the edge with Cloudflare Workers.

    Latency Benchmarks Measured in Production

    Numbers people quote in blog posts are usually synthetic. These are not — I pulled them from production traffic in June 2026. Embedding generation with bge-base runs at 35 to 45 milliseconds at the P50, fast enough that I embed inline during a request rather than queuing it. LLaMA 3 8B generation lands at 280 to 340 milliseconds for a 100-token output, which is comfortable for a summary that renders after a user action but too slow for anything you want to feel instantaneous. Knowing those two ranges shapes every design decision — if a task needs sub-50ms, it gets an embedding; if it can absorb a third of a second, it gets a generation; if it needs anything heavier, it leaves the edge entirely.

    When Not to Use Workers AI

    Edge inference is not a universal answer, and pretending otherwise leads to bad systems. Complex reasoning — multi-step analysis, nuanced judgment, anything where a wrong answer is expensive — belongs on Claude or GPT-4 through an external API, not on an 8B model at the edge. Tasks that reliably need more than a 4K context window are a poor fit, because you’ll spend more effort chunking than you save in latency. And if your task demands a fine-tuned model, Workers AI is the wrong tool — you fine-tune elsewhere and serve the result elsewhere. The free tier is generous precisely because it’s scoped to inference on stock models for bounded tasks.

    What Transfers

    The lesson generalizes well beyond Cloudflare. Run cheap models close to the user for latency-sensitive, low-stakes tasks, and route complex reasoning to centralized APIs that have the full context and the bigger models. The edge inference paradigm is about matching the model’s location and size to the task’s actual requirements — not defaulting to the largest model for everything, and not forcing every workload through one pipe. Cost discipline and latency discipline turn out to be the same discipline: put the right computation in the right place.

    → Building something at the intersection of AI, edge computing, and behavioral science? Let’s connect.

  • Applying Lean Six Sigma to AI Projects: A Green Belt’s Framework

    As a Lean Six Sigma Green Belt who has applied DMAIC to AI and ML projects across 19 years in defense finance, the most consistent failure pattern I’ve seen is AI teams who skip the Define and Measure phases entirely and go straight to building a model. They have a dataset, they have a use case, they have a timeline — and they have no baseline, no measurable problem statement, and no control plan. The model gets deployed. Three months later, nobody can prove it’s better than what it replaced.

    DMAIC — Define, Measure, Analyze, Improve, Control — is a 60-year-old quality framework that was not designed for AI. It maps onto AI project structure almost perfectly anyway, because the failure modes it was designed to prevent are the same failure modes that kill AI deployments.

    Define and Measure: The Phases That Prevent Failure

    The Define phase requires a problem statement with a measurable Y. Not “improve efficiency in the tax processing workflow” — that’s a direction, not a target. “Reduce purchase order error rate from 31% to below 10% in the internal tax scenario classifier pipeline, measured over a 90-day post-deployment window” is a problem statement. The Y is specific, the current state is quantified, the target is defined, and the measurement window is explicit. You cannot claim success without meeting that specification. You also cannot accidentally redefine success after the fact when results are disappointing, which is the most common way AI projects survive their pilots despite not working.

    The Measure phase is where most AI projects actually fail, because if you don’t have a baseline, you can’t prove improvement — and yet most teams skip it because it delays the part that feels like progress. The protocol is to baseline the current process for 4 to 6 weeks before any automation or model is deployed, using the exact same metrics you’ll use post-deployment. The 2,132 hours per year figure from a major defense aircraft program withholds automation — which justified $360,000 in annual cost avoidance — started as a manual time study. Analysts logged their time against specific task categories for 3 weeks before any automation existed. That baseline wasn’t overhead; it was the evidence that made the business case and gave us a threshold to beat.

    Analyze and Improve: Choosing the Right Intervention

    The Analyze phase is root cause work: understanding why the problem exists before deciding what to do about it. Is the error rate high because the data is bad? Because the process is poorly designed? Because humans are making decisions under time pressure with incomplete information? Because the system produces ambiguous outputs that require judgment calls? Each of those root causes has a different correct intervention — and sometimes that intervention is a process redesign that doesn’t involve AI at all.

    This is the discipline that prevents solution-chasing. When you start with a model and work backward to the problem it solves, you’ll find a problem that matches the model’s capabilities — whether or not it’s the actual root cause of the performance gap you’re trying to close. Starting with root cause analysis in Alteryx or Python to understand the data quality and process patterns before touching a model architecture is slower in the short term and dramatically more reliable in the long term.

    The Improve phase is where the AI model enters — framed as a process change, not as a technology implementation. Design it, run a controlled pilot with a scope limited enough to measure cleanly, compare against the baseline using the metrics defined in the Measure phase, and iterate. Don’t skip the pilot. A model that performs well on a holdout test set and a model that performs well in production are not the same thing, and the difference is almost always in the human and process factors that a test set doesn’t capture.

    Control: The Phase That Makes It a Production System

    The Control phase is what separates an AI project from an AI system. A project ends at deployment. A system has ongoing monitoring, drift detection, retraining protocols, and documented ownership. Without those, you have a model that was accurate when it launched and an unknown amount of time before it’s quietly wrong in ways nobody has instrumented to detect.

    The Control plan for an AI system needs to specify: what metrics indicate the model is performing within acceptable bounds, what threshold triggers a retraining evaluation, who owns the retraining decision, and what the rollback procedure is if a retrained model performs worse than its predecessor. These aren’t optional governance niceties — they’re the difference between a system that survives its first year and one that gets quietly retired when the person who built it moves on.

    The sigma equivalent framing is useful for calibrating expectations here. A 3-sigma AI system — 99.73% accuracy — sounds impressive. It produces 2,700 errors per million decisions. If those decisions are purchase order classifications affecting a multi-hundred-million-dollar contract financial position, 2,700 errors is not an acceptable defect rate. The sigma target has to be set against the consequence of the error, not against an abstract accuracy benchmark.

    What Transfers

    Six Sigma rigor applied to AI is not methodology overhead imposed by people who don’t understand how AI development works. It’s the framework that answers the question every executive eventually asks: “How do we know this is working?” The answer to that question has to be built into the project from the Define phase — you can’t construct it after deployment from whatever data happens to be available. The 17 AI and automation projects I led at Lockheed Martin that produced defensible, sustained business value all had one thing in common: the measurement infrastructure was designed before the model was built, not after.

    The teams that resist the DMAIC discipline usually frame it as a speed concern — moving fast, not getting bogged down in process. In practice, the projects that skip Define and Measure don’t move faster. They move quickly toward a result they can’t prove, then spend months in a political argument about whether the system is working. The upfront rigor is the shortcut.

    → Building something at the intersection of AI, edge computing, and behavioral science? Let’s connect.

  • Sovereign AI in Defense: Why On-Premises Intelligence Is Making a Comeback

    After a decade of cloud migration in defense and intelligence, the line about what can run where is being redrawn — and for classified and sensitive workloads, it’s moving back toward the perimeter, not further into commercial infrastructure.

    This isn’t a reversal of cloud adoption broadly. It’s a sharper delineation of which workloads belong where. The term that’s emerged for the approach is “sovereign AI” — and understanding what it actually requires technically is useful well beyond the defense context.

    What Sovereign AI Actually Means

    Sovereign AI in its precise form means models trained on your data, running on your compute, without calling any external API at inference time. No data leaves your network. No inference request passes through a commercial provider’s infrastructure. The model weights live on hardware you control, in a facility you control, serving requests from systems that never touch the public internet.

    That’s meaningfully different from “using a private deployment of a commercial model” or “running a fine-tuned model on cloud infrastructure.” Both of those approaches still involve dependencies on external entities — the model provider, the cloud provider, the network path between your environment and theirs. Sovereign AI eliminates those dependencies by design, because the threat model requires it.

    Why It’s Mandated, Not Optional

    SECRET and TOP SECRET data cannot touch commercial cloud infrastructure under current regulations — not because the commercial providers are untrustworthy, but because the classification system predates the commercial cloud and its authorization pathways haven’t caught up with the most sensitive data categories. That’s a governance reality, not a technical judgment about cloud security.

    Beyond data residency requirements, there are three additional threat vectors that make sovereign AI architecturally necessary for the most sensitive use cases. Adversarial model extraction: a sophisticated adversary can probe an API repeatedly to reconstruct statistical properties of the training data, including potentially sensitive patterns that were never intended to be accessible externally. Supply chain risk in commercial model weights: a model trained by a commercial provider and downloaded for internal use still carries whatever was embedded in its training — including potentially undesirable behaviors or backdoors that are difficult to detect through standard evaluation. And CMMC, ITAR, and FedRAMP High impact level requirements, which create compliance obligations that some commercial AI deployments can’t satisfy regardless of their technical security posture.

    From my 19 years at Lockheed Martin, the governance question I saw teams face repeatedly was not “is this AI tool good enough?” — it was “can this AI tool even touch this data category?” That question has to be answered before any technical evaluation begins. The answer determines the entire deployment architecture.

    The Technical Reality of On-Premises Inference

    Running LLaMA 3 70B on air-gapped NVIDIA H100s inside a Sensitive Compartmented Information Facility is a very different operational challenge than calling the OpenAI API. Hardware acquisition at classified scale takes 6 to 18 months through approved procurement channels. The inference stack itself — the software serving requests to the model — requires security review before it can be deployed in a classified environment, because the software supply chain is part of the attack surface. Model updates require the same review cycle that the initial deployment did, which means rapid iteration on model capabilities is essentially impossible in the most sensitive environments.

    The interesting middle ground is what I’d call the classified edge: compact, deployable inference running on ruggedized hardware at forward positions or remote installations, with no external network dependencies at all. A quantized Mistral 7B model can run inference on hardware that fits in a Pelican case. That’s not a theoretical capability — it’s an operational requirement in environments where cloud connectivity is unavailable, unreliable, or tactically dangerous. Think of it as the Cloudflare Workers model applied to networks that can never touch the internet: push the intelligence as close to the decision point as physically possible, with no dependency on infrastructure that might not be reachable. The edge-first architecture pattern translates surprisingly well to the classified environment, once you strip away the commercial infrastructure assumptions.

    What Transfers

    The sovereign AI framework — data residency, inference independence, supply chain trust — applies in any heavily regulated industry, even when “classified” isn’t the specific label on the requirement. Healthcare AI processing PHI has data residency requirements. Legal AI handling privileged communications has confidentiality requirements that commercial cloud AI may not satisfy. Financial services AI operating under specific regulatory regimes has audit and data localization requirements that constrain what external APIs the inference pipeline can call.

    The question “where does the inference happen and who can see the inputs?” is a governance question that every enterprise AI deployment needs to answer explicitly. The defense context makes the stakes and the requirements maximally clear — but the underlying framework applies wherever the data being processed carries obligations that extend beyond the organization’s own interests. That’s a much larger category of use cases than most AI practitioners currently treat it as.

    → Building something at the intersection of AI, edge computing, and behavioral science? Let’s connect.

  • The Non-Linear Career Path: How Psychology + Finance Made Me a Better AI Engineer

    The question I get most often from people who look at my background is some version of: “How does a therapist end up doing AI work for defense contractors?” The answer is that the path wasn’t non-linear — it was compound. Each layer added something the previous layer couldn’t provide alone, and the combination produces a profile that’s genuinely unusual in AI deployment work.

    What the Therapy Training Actually Provided

    The Licensed Marriage and Family Therapist licensure isn’t a credential I hold as a curiosity. It came from graduate training in systems theory — understanding how behavior in one part of a system affects behavior in other parts — and from supervised clinical practice that required learning to hold complexity, ambiguity, and conflicting stakeholder interests simultaneously. Those skills transfer directly to AI deployment work, where the hardest problems are rarely technical. The hard problems are: why are the people this system is supposed to help resistant to using it, how do you design human-AI interaction so that people engage appropriately rather than over-trusting or under-trusting the outputs, and what happens to team dynamics when a model starts doing work that used to define someone’s role. A pure ML background doesn’t prepare you to navigate those questions. Clinical training does.

    What I/O Psychology Added

    Industrial-Organizational Psychology is the discipline that applies behavioral science to organizational contexts — selection, performance, motivation, team dynamics, organizational change. The master’s-level training in I/O psychology gave me a rigorous framework for thinking about how AI tools change work, not just how they perform tasks. When I was deploying automation in defense finance, the questions that mattered weren’t only “does the model work?” They were: how do we measure performance in a way that’s fair to the people whose work is changing, how do we design the transition so that the analysts whose time is being recovered feel like the beneficiaries rather than the displaced, and how do we structure human-AI collaboration so that human judgment is applied where it actually adds value rather than where it just feels necessary. I/O psychology has decades of research on exactly those questions.

    What Defense Finance Contributed

    Nineteen years in defense finance at Lockheed Martin provided two things that can’t be replicated by reading about them. The first is operational credibility: when I build an AI system for finance, I understand the workflows, the error consequences, the audit requirements, and the organizational dynamics of getting a finance team to trust a model. The second is a track record of consequential deployments: the major defense aircraft program withholds automation that produced $360K in annual cost avoidance and 2,132 recovered analyst hours per year, the internal tax scenario classifier that processed 415,000 tax scenarios in 4 hours and reduced purchase order error rates by 31%. Those outcomes exist because the technical work was grounded in deep domain knowledge.

    Why the Combination Is Rare

    Most AI engineers don’t have clinical training. Most clinicians don’t have 19 years in defense finance. Most finance professionals don’t have the technical depth to build production AI systems. The combination is rare not because it’s strategically constructed but because each step seemed like the obvious next thing given where I was. The therapy training came first and was about understanding people. The finance career built on that. The AI work emerged from the automation problems the finance work surfaced. The I/O psychology formalized frameworks I was already applying intuitively. In retrospect the path looks deliberate; at each step it was just following what was interesting and useful.

    What Transfers

    The argument this background makes isn’t that every AI engineer needs clinical training or that domain expertise always compounds into something useful. The argument is that AI deployment problems are fundamentally human problems — adoption, trust, organizational change, consequence management — and the people best positioned to solve them are those who have spent time on both the technical and the human sides. A pure-ML background is increasingly common. The combination is not.

    → Building something at the intersection of AI, edge computing, and behavioral science? Let’s connect.

  • LLM-as-Judge: When to Trust Your AI Evaluator (And When to Override It)

    LLM-as-judge is seductive because it solves an expensive problem cheaply: instead of paying human annotators to evaluate model outputs, you have a larger or more capable model do the grading. The cost savings are real. The risks are also real, and teams that adopt LLM-as-judge without understanding the failure modes end up with evaluation infrastructure that gives them confident-sounding metrics while quietly measuring the wrong thing.

    When It Works

    LLM-as-judge works well when the evaluation criteria are explicit and the judge model can be given precise instructions that human annotators would also receive. Factual accuracy, format compliance, completeness against a checklist, tone matching against a defined style guide — these are tasks where a capable judge model can achieve inter-rater reliability with human annotation that makes it a reasonable substitute. The test for whether your LLM-as-judge setup is working is to run it in parallel with human annotation on a sample and calculate Cohen’s kappa between the two. If kappa is above 0.7, your automated evaluation is reliable enough to trust at scale. Below that threshold, you’re measuring something that doesn’t sufficiently correlate with human judgment to be useful.

    When It Fails

    LLM-as-judge fails predictably on tasks that require domain expertise the judge model doesn’t have, tasks where the rubric is genuinely ambiguous, and tasks where the judge model has systematic biases that don’t match the target population. On domain expertise: a general-purpose model grading the accuracy of specialized tax scenario classification outputs is grading against its own training data, not against ground truth. The grade it gives correlates with how confident the output sounds, not whether it’s correct. On rubric ambiguity: if the evaluation criteria aren’t precise enough for two human annotators to agree consistently, the judge model will also be inconsistent — but it will be inconsistent in a way that looks consistent because it applies the same internal model repeatedly. On bias: LLMs have documented preferences for certain output styles, lengths, and formats that don’t always correlate with quality. A judge model that prefers longer, more hedged responses will systematically grade those outputs higher regardless of their actual correctness.

    Calibration Against Human Annotation

    The calibration protocol that worked in the internal tax scenario classifier — the tax scenario processing system that ran 415,000 scenarios in 4 hours — applied the same methodology to LLM-as-judge validation. Before deploying automated evaluation at scale, we sampled 200 outputs and had both human annotators and the judge model evaluate them independently. The kappa between human annotators came in at 0.81 on unambiguous cases and 0.54 on boundary cases. The kappa between human annotation and LLM-as-judge came in at 0.78 on unambiguous cases and 0.41 on boundary cases. That calibration told us exactly where to trust the automated evaluation and where to require human review: for clear-cut cases at scale, the judge model was reliable; for boundary cases, it was not, and routing those to human annotators was the correct design.

    The Consistency Trap

    The most dangerous property of LLM-as-judge is that it can be consistently wrong. A human annotation process with poor inter-rater reliability at least has the property that its errors are somewhat random — with enough annotators, the noise averages out. A judge model applies the same systematic bias repeatedly, which means the errors don’t average out; they compound. If your judge model systematically misclassifies a category of outputs, every data point in that category gets the wrong grade, and your optimization process confidently drives the model you’re evaluating in the wrong direction. The consistency that makes LLM-as-judge seem reliable is also what makes its systematic errors harder to detect.

    Practical Implementation Guidelines

    Three guidelines have held up across my use of LLM-as-judge in production systems. First, calibrate before deploying: run the judge in parallel with human annotation on a representative sample and calculate kappa before trusting it at scale. Second, stratify by case type: most evaluation tasks have a subset of cases where automated grading is reliable and a subset where it isn’t — treat those subsets differently rather than applying a single threshold. Third, re-calibrate periodically: the judge model’s behavior can change across versions, and a calibration established on one model version doesn’t automatically transfer to the next.

    What Transfers

    The clinical assessment parallel holds here too. Structured clinical interviews have a validated set of questions and scoring criteria specifically because clinician judgment varies in ways that affect diagnostic reliability — the same symptoms get different interpretations from different practitioners without standardized tools. LLM-as-judge is automated judgment, and it has the same reliability problem as human judgment: it needs to be calibrated, validated, and monitored rather than trusted by default. The tools for doing that — inter-rater reliability measurement, calibration protocols, stratified validation — come from the same field that figured this out for human judgment decades ago.

    → Building something at the intersection of AI, edge computing, and behavioral science? Let’s connect.