Running LLM inference at 300 locations worldwide for free sounds like marketing copy — but it’s actually the production architecture I use for lightweight classification and embedding tasks across my fleet of 60-plus Cloudflare Workers, every one of them sitting at $0 per month. I want to walk through what Workers AI actually delivers in 2026, what the free tier buys in real terms, and where this approach stops making sense.
The 2026 Model Catalog
The Workers AI catalog in 2026 covers most of what an edge application needs without reaching for an external API. For text generation you have LLaMA 3 in both 8B and 70B variants, Mistral 7B for faster lighter work, and Gemma 2B when you need the smallest possible footprint. For embeddings, BAAI/bge-base-en-v1.5 produces 768-dimension vectors that I store directly in D1. Whisper-large-v2 handles transcription. The point is that a single Worker can generate text, embed documents, and transcribe audio without a single outbound fetch to a third-party service — the model runs on Cloudflare’s GPUs at the edge node closest to the request.
What the Free Tier Actually Buys
Workers AI bills in “neurons” — a normalized compute unit that abstracts away the differences between a tiny embedding call and a heavy 70B generation. The free tier gives you 10,000 neurons per day. That abstraction is confusing until you map it to real work, so here are the numbers from my own usage logs. Ten thousand neurons buys roughly 5,000 LLaMA 3 8B inference calls in a day, or about 50,000 embedding generations with bge-base, or close to 300 minutes of Whisper transcription. For a personal tool or a low-traffic production service, that ceiling is rarely the constraint — I have Workers that have never come close to exhausting a day’s allotment, which is how the whole fleet stays at zero cost.
Production Use Cases From the Fleet
The clearest example is daniel-job-search, my career intelligence PWA. It uses Workers AI with bge-base to generate embeddings for every incoming job description and for my Ideal Candidate Profile, then runs cosine similarity in D1 to score matches. The entire matching pipeline runs inside a Worker — no external API calls, no per-token billing, no network hop to a model provider. Digital Sentinel, my OSINT monitoring system, uses Workers AI text classification to triage alerts before routing the important ones to Discord, which means the noise filtering happens at the edge instead of clogging a downstream queue. I wrote more about that build in my post on running OSINT monitoring on the edge with Cloudflare Workers.
Latency Benchmarks Measured in Production
Numbers people quote in blog posts are usually synthetic. These are not — I pulled them from production traffic in June 2026. Embedding generation with bge-base runs at 35 to 45 milliseconds at the P50, fast enough that I embed inline during a request rather than queuing it. LLaMA 3 8B generation lands at 280 to 340 milliseconds for a 100-token output, which is comfortable for a summary that renders after a user action but too slow for anything you want to feel instantaneous. Knowing those two ranges shapes every design decision — if a task needs sub-50ms, it gets an embedding; if it can absorb a third of a second, it gets a generation; if it needs anything heavier, it leaves the edge entirely.
When Not to Use Workers AI
Edge inference is not a universal answer, and pretending otherwise leads to bad systems. Complex reasoning — multi-step analysis, nuanced judgment, anything where a wrong answer is expensive — belongs on Claude or GPT-4 through an external API, not on an 8B model at the edge. Tasks that reliably need more than a 4K context window are a poor fit, because you’ll spend more effort chunking than you save in latency. And if your task demands a fine-tuned model, Workers AI is the wrong tool — you fine-tune elsewhere and serve the result elsewhere. The free tier is generous precisely because it’s scoped to inference on stock models for bounded tasks.
What Transfers
The lesson generalizes well beyond Cloudflare. Run cheap models close to the user for latency-sensitive, low-stakes tasks, and route complex reasoning to centralized APIs that have the full context and the bigger models. The edge inference paradigm is about matching the model’s location and size to the task’s actual requirements — not defaulting to the largest model for everything, and not forcing every workload through one pipe. Cost discipline and latency discipline turn out to be the same discipline: put the right computation in the right place.
→ Building something at the intersection of AI, edge computing, and behavioral science? Let’s connect.
Leave a Reply