Grounded Intelligences

Category: LLM Engineering

Deep dives into LLM fine-tuning, evaluation, RAG pipelines, and production LLM systems.

  • Fine-Tuning LLMs on Proprietary Data: The Annotation Quality Problem

    A LoRA fine-tune of Mistral 7B on 10,000 examples cost me under $50 on RunPod in 2026 — the compute is cheap and getting cheaper. The bottleneck moved years ago, and most teams haven’t noticed. It is no longer the training run. It is the quality of the data you feed into it, and annotation quality is where fine-tuning projects quietly fail.

    What Annotation Quality Actually Means

    Annotation quality is not just “did the annotators agree with each other.” It is three harder questions. Did the annotation rubric actually capture the task you care about, or a convenient proxy for it? Were annotators calibrated against a shared set of examples before they started labeling, or did they each invent their own interpretation? And were the edge cases documented as they came up, or silently resolved and forgotten? Agreement on a bad rubric just gives you consistent noise. The quality lives in the rubric and the calibration, not in the raw agreement number alone.

    Inter-Rater Reliability as the Starting Gate

    Cohen’s kappa is the first gate, and I treat it as a gate rather than a vanity metric. A kappa below 0.70 means your annotators are not reliably applying the same rubric — which means your fine-tuning data carries the disagreement as noise straight into the weights. A kappa below 0.60 tells you something more serious: the rubric itself probably needs redesign, not just more annotator calibration. The distinction matters because the fix is different. Below 0.70 you retrain the people. Below 0.60 you rewrite the instructions.

    The internal tax scenario classifier Annotation Workflow

    Building the internal tax scenario classifier meant creating more than 10,000 training examples for tax scenario classification, and the workflow reflected everything above. Three annotators reviewed each batch. Every batch opened with a pre-calibration session using 50 canonical examples, so everyone was anchored to the same interpretation before touching new data. A resolution protocol handled disagreements — an adjudicator reviewed any annotator pair that fell below a kappa of 0.75. The first 3,000 examples took six weeks. The annotating itself was fast; what was slow was resolving rubric ambiguities, the genuinely hard classification calls where reasonable experts disagreed and the instructions had to be sharpened before we could proceed. That slowness was the work paying off, not the work failing.

    The Failure Mode

    The common failure is predictable. A team rushes annotation, skips the pre-calibration session, skips the inter-rater checks, and ships a large pile of labeled examples on a deadline. Then they fine-tune and discover the model is worse than the base model on their own task. They blame the model, the framework, the learning rate — anything but the data. The model learned exactly what it was given, which was noise dressed up as signal. There is no hyperparameter that recovers from a corrupted training set.

    The Quality Floor

    Here is the number that should reshape how teams budget annotation effort: 1,000 high-quality annotated examples with a kappa above 0.80 beats 20,000 noisy ones for fine-tuning. This is consistent across the fine-tuning literature and consistent with my own production experience. The instinct to gather more data is usually the wrong instinct. The right instinct is to make the data you already have cleaner — tighten the rubric, recalibrate the annotators, and accept a smaller, sharper training set over a large, blurry one.

    Documenting Edge Cases as Your Test Set

    For every annotation batch, I require documenting the 20 to 30 cases that were hardest to classify and why they were hard. Those documented edge cases become the test set for evaluating the fine-tuned model — deliberately, not random samples pulled from the annotation pool. A random test set tells you how the model does on average. The hard-case test set tells you how it does exactly where it’s most likely to break, which is the only place where evaluation earns its keep. The same rigor shows up across regulated deployments, which I wrote about in lessons from 17 enterprise projects.

    What Transfers

    This is clinical assessment methodology applied to machine learning annotation. Structured clinical interviews, rubric-based scoring, inter-rater reliability checks — all of these exist in clinical psychology, and they exist there for exactly the same reasons they matter in annotation. When you need multiple humans to apply the same judgment consistently to ambiguous material, the discipline is identical whether the material is a patient presentation or a tax scenario. The field that has spent decades measuring human agreement has a great deal to teach the field that just discovered it needs to.

    → Building something at the intersection of AI, edge computing, and behavioral science? Let’s connect.

  • LLM-as-Judge: When to Trust Your AI Evaluator (And When to Override It)

    LLM-as-judge is seductive because it solves an expensive problem cheaply: instead of paying human annotators to evaluate model outputs, you have a larger or more capable model do the grading. The cost savings are real. The risks are also real, and teams that adopt LLM-as-judge without understanding the failure modes end up with evaluation infrastructure that gives them confident-sounding metrics while quietly measuring the wrong thing.

    When It Works

    LLM-as-judge works well when the evaluation criteria are explicit and the judge model can be given precise instructions that human annotators would also receive. Factual accuracy, format compliance, completeness against a checklist, tone matching against a defined style guide — these are tasks where a capable judge model can achieve inter-rater reliability with human annotation that makes it a reasonable substitute. The test for whether your LLM-as-judge setup is working is to run it in parallel with human annotation on a sample and calculate Cohen’s kappa between the two. If kappa is above 0.7, your automated evaluation is reliable enough to trust at scale. Below that threshold, you’re measuring something that doesn’t sufficiently correlate with human judgment to be useful.

    When It Fails

    LLM-as-judge fails predictably on tasks that require domain expertise the judge model doesn’t have, tasks where the rubric is genuinely ambiguous, and tasks where the judge model has systematic biases that don’t match the target population. On domain expertise: a general-purpose model grading the accuracy of specialized tax scenario classification outputs is grading against its own training data, not against ground truth. The grade it gives correlates with how confident the output sounds, not whether it’s correct. On rubric ambiguity: if the evaluation criteria aren’t precise enough for two human annotators to agree consistently, the judge model will also be inconsistent — but it will be inconsistent in a way that looks consistent because it applies the same internal model repeatedly. On bias: LLMs have documented preferences for certain output styles, lengths, and formats that don’t always correlate with quality. A judge model that prefers longer, more hedged responses will systematically grade those outputs higher regardless of their actual correctness.

    Calibration Against Human Annotation

    The calibration protocol that worked in the internal tax scenario classifier — the tax scenario processing system that ran 415,000 scenarios in 4 hours — applied the same methodology to LLM-as-judge validation. Before deploying automated evaluation at scale, we sampled 200 outputs and had both human annotators and the judge model evaluate them independently. The kappa between human annotators came in at 0.81 on unambiguous cases and 0.54 on boundary cases. The kappa between human annotation and LLM-as-judge came in at 0.78 on unambiguous cases and 0.41 on boundary cases. That calibration told us exactly where to trust the automated evaluation and where to require human review: for clear-cut cases at scale, the judge model was reliable; for boundary cases, it was not, and routing those to human annotators was the correct design.

    The Consistency Trap

    The most dangerous property of LLM-as-judge is that it can be consistently wrong. A human annotation process with poor inter-rater reliability at least has the property that its errors are somewhat random — with enough annotators, the noise averages out. A judge model applies the same systematic bias repeatedly, which means the errors don’t average out; they compound. If your judge model systematically misclassifies a category of outputs, every data point in that category gets the wrong grade, and your optimization process confidently drives the model you’re evaluating in the wrong direction. The consistency that makes LLM-as-judge seem reliable is also what makes its systematic errors harder to detect.

    Practical Implementation Guidelines

    Three guidelines have held up across my use of LLM-as-judge in production systems. First, calibrate before deploying: run the judge in parallel with human annotation on a representative sample and calculate kappa before trusting it at scale. Second, stratify by case type: most evaluation tasks have a subset of cases where automated grading is reliable and a subset where it isn’t — treat those subsets differently rather than applying a single threshold. Third, re-calibrate periodically: the judge model’s behavior can change across versions, and a calibration established on one model version doesn’t automatically transfer to the next.

    What Transfers

    The clinical assessment parallel holds here too. Structured clinical interviews have a validated set of questions and scoring criteria specifically because clinician judgment varies in ways that affect diagnostic reliability — the same symptoms get different interpretations from different practitioners without standardized tools. LLM-as-judge is automated judgment, and it has the same reliability problem as human judgment: it needs to be calibrated, validated, and monitored rather than trusted by default. The tools for doing that — inter-rater reliability measurement, calibration protocols, stratified validation — come from the same field that figured this out for human judgment decades ago.

    → Building something at the intersection of AI, edge computing, and behavioral science? Let’s connect.

  • Building RAG Over 10,000 Research Papers: Architecture and Hard Lessons

    I built a retrieval-augmented generation system over 10,000 academic papers in I/O psychology, organizational behavior, and AI ethics — not as a demonstration project, but because I needed a research assistant that could actually find the supporting evidence buried in dense methodology sections and tell me where it came from.

    Why Academic Paper RAG Is Harder Than Product RAG

    Most RAG tutorials assume clean, well-structured source documents — FAQs, product manuals, internal wikis. Academic PDFs are none of those things. Multi-column layouts break naive text extractors. Equations render as gibberish or disappear entirely. Citations frequently span page breaks, so the reference text you want is split across two chunks. Abstracts — the obvious candidate for a summary embedding — often don’t reflect the actual contribution of the paper. A paper titled “A Meta-Analysis of TAM Adoption Predictors” might have its most useful content buried in a limitations section that no one would retrieve based on the title alone.

    The naive approach — extract all text, split into 512-token chunks with 50-token overlap, embed, and retrieve — produces results that look plausible but miss the actual evidence. For casual Q&A over product documentation, that’s acceptable. For research work where you’re trying to accurately represent the state of evidence on a topic, it’s not.

    The Chunking Strategy That Actually Works

    The architecture I settled on uses section-aware chunking rather than a sliding window. Each paper is parsed to extract named sections — Introduction, Methods, Results, Discussion, Limitations — and each section becomes its own chunk set. Every chunk carries metadata: paper title, publication year, authors, journal, and section name. That metadata travels with the chunk through retrieval and into the prompt.

    The reason section-level metadata matters is that the provenance of a claim is different depending on whether it comes from a Results section or a Discussion section. “Participants showed a 23% improvement in task completion” means something different in Results than in Discussion. Knowing which section a retrieved chunk came from changes how you should weight it in your synthesis.

    For the embedding model, I started with text-embedding-3-small and moved to text-embedding-3-large for this corpus specifically. The quality difference for dense academic prose — methodology descriptions, statistical terminology, theoretical constructs — was meaningful enough to justify the cost difference. For retrieval over casual conversational text, small would have been fine.

    Hybrid Retrieval and the Reranker You Cannot Skip

    Academic text has a property that makes pure vector retrieval underperform: exact terminology matters enormously. When a research question involves “Cohen’s kappa” or “confirmatory factor analysis” or “heteroscedasticity,” semantic similarity search will retrieve conceptually adjacent material that doesn’t actually contain the relevant methodology. BM25 keyword retrieval catches exact matches that vector search misses.

    The retrieval pipeline runs both — vector search weighted at 0.6, BM25 at 0.4 — then merges the candidate pools. The weight split was empirically tuned on a held-out evaluation set of 200 research questions where I had manually verified the correct source chunks. Getting the balance wrong in either direction cost meaningful retrieval quality.

    The reranker is not optional. Without a cross-encoder reranker applied to the merged candidate pool, the top-ranked results consistently look relevant — they match on topic — but miss the actual supporting evidence. The cross-encoder evaluates query-chunk relevance jointly rather than independently, which costs more compute but catches the cases where a chunk that seems topically relevant doesn’t actually answer the question. I tested the pipeline with and without reranking on the same 200-question evaluation set, and the difference in retrieval precision was large enough that I would not ship academic RAG without it.

    Three Hard Lessons

    The first hard lesson was the reranker, already described. The second was metadata filtering by publication year. A 2018 paper’s recommendations for model evaluation benchmarks, fairness metrics, or dataset standards may be functionally obsolete. Without a year filter built into the retrieval interface, the system would confidently surface outdated methodology as current best practice. Year-range filtering is now a first-class parameter in every query I run against this corpus.

    The third lesson was citation chains. The paper that directly answers your question is frequently not the paper that contains the data supporting its claim — it cites another paper, which may cite a third. A retrieval system that only returns the top-ranked chunks and stops there will miss this structure entirely. I added a citation-following step that, when a retrieved chunk contains a reference to a specific study, fetches that study’s chunks and includes them as secondary context. This added complexity, but it’s the step that makes the difference between a system that finds what you asked for and a system that finds the actual evidence.

    What Transfers

    The chunking-with-metadata principle applies to any long-form document RAG — legal contracts, engineering manuals, case files, regulatory filings. The insight is the same: your chunks should carry enough provenance information that downstream reasoning can assess not just what a chunk says, but where it sits in the document’s argumentative structure and how current it is. Sliding-window chunking that strips that context produces retrieval that feels right more often than it is right — which is a failure mode that’s harder to catch than one that fails loudly. If you’re working on similar document retrieval problems, the broader lessons from enterprise AI deployment apply here as well: the architecture decisions that seem like premature optimization at 100 documents become the only thing standing between you and retrieval collapse at 10,000.

    → Building something at the intersection of AI, edge computing, and behavioral science? Let’s connect.

  • LLM Evaluation at Scale: Building a Golden Dataset That Actually Works

    I have watched teams ship LLM features with no structured evaluation whatsoever — no golden dataset, no inter-rater agreement calculation, no adversarial examples — and then express genuine surprise three weeks later when production breaks in ways that dev never predicted. This is not a tooling problem or a research problem; it’s a discipline problem, and the fix is unglamorous: you need a curated, annotated evaluation set built before you ship, not assembled in a panic after your first production incident.

    What a Golden Dataset Actually Is

    A golden dataset is a curated collection of (input, expected_output) pairs that represent the real distribution of tasks your model will encounter in production — including the hard ones, the edge cases, and the inputs specifically designed to surface failure modes. It is not a sample of the easy inputs where the model already performs well. It is not a set of examples you grabbed from a tutorial. It is a structured artifact, maintained like code, that tells you whether your model is getting better or worse as you iterate. Three ingredients separate a real golden dataset from a list of test cases someone threw together on a Friday afternoon. First, it needs at least 500 examples — below that threshold, your accuracy measurements have confidence intervals wide enough to be meaningless. Second, it needs adversarial examples mixed with the easy ones in a ratio that reflects your actual production failure rate — if 15% of your production inputs are edge cases, your golden set should be at least 15% adversarial. Third, every example needs to be annotated by at least two independent annotators, with inter-rater reliability calculated using Cohen’s kappa — and your target kappa should be 0.7 or higher before you trust the labels.

    How the internal tax scenario classifier Forced the Discipline

    When I built the internal tax scenario classifier — the tax scenario processing system that ran 415,000 scenarios in 4 hours and achieved a 31% reduction in purchase order error rate — the evaluation problem was not optional. Tax classification is high-stakes and adversarially complex: the same economic transaction can have materially different tax treatments depending on jurisdiction, contract structure, and a half-dozen other variables. Getting the model wrong didn’t produce a slightly inconvenient user experience; it produced incorrect tax filings with audit exposure attached. We built an annotation corpus of 8,000+ labeled tax scenarios before we ran a single evaluation pass. The annotation workflow ran in three phases: a senior tax specialist assigned the ground-truth label for each scenario, a second specialist reviewed independently, and disagreements went to a structured resolution process where both annotators discussed the case against a written decision criteria document before a label was assigned. The kappa score in the first annotation round was 0.61 — below the 0.7 threshold — which told us the labeling criteria were ambiguous in about 20% of the cases. We spent two weeks tightening the criteria document and re-annotating the contested examples. The second pass came in at 0.74. That kappa score meant something: when the model scored 88% accuracy against that dataset, we had high confidence the 88% was real and not an artifact of easy examples and loose labels.

    The Distribution Trap

    The most common mistake teams make after building a golden dataset is constructing one that is too easy — heavy on the clear-cut cases, light on the ambiguous ones — and then celebrating when the model hits 94% accuracy. That number is meaningless if your production inputs include the 6% of scenarios where the model catastrophically fails, because that 6% might represent the highest-stakes decisions in the entire workflow. The adversarial examples you add in week three of evaluation should not be random stress tests — they should be the actual failure modes you observed when you let the model run on unlabeled production data in shadow mode. Shadow mode evaluation — running the model in parallel with the existing process without using its outputs, just observing — is how you discover what your golden set is missing. The failure modes you find in shadow mode become your adversarial examples, which makes your golden set a living artifact that improves as you learn more about where the model breaks.

    What Transfers

    Clinical assessment methodology has required structured reliability measurement for decades — the DSM field trials that established diagnostic criteria used inter-rater kappa as a primary validity metric, and a diagnostic tool with kappa below 0.8 was considered insufficiently reliable for clinical use. That standard didn’t emerge because psychologists are unusually rigorous; it emerged because the cost of diagnostic error — misdiagnosis, missed treatment, wrong intervention — made reliability non-negotiable. LLM evaluation should be held to the same logic. If the model’s output touches a consequential decision, the annotation process that produces your ground truth needs enough rigor that two independent experts agree on the right answer at least 70% of the time after accounting for chance agreement. Below that threshold, you’re not measuring model performance — you’re measuring label noise. The investment in evaluation infrastructure is front-loaded and genuinely tedious, but it’s the only thing that gives you honest signal about whether your model is improving or whether you’re just iterating on vibes.

    → Building something at the intersection of AI, edge computing, and behavioral science? Let’s connect.