Three cards from two operators. Two themes with a shared thread underneath: the teams shipping AI at production scale are winning by shrinking the AI's role, not expanding it.
Shrink the model's job to what only the model can do
Tomasz Tunguz published a detailed breakdown of Vercel's inbound agent on October 6. The headline finding is counterintuitive: the most reliable production AI agent workflow is mostly not AI. Vercel's architecture runs 65% of its nodes as pure deterministic code and only 14% as fully agentic LLM inference. The remaining nodes handle transitions and lightweight processing.
Production AI agent workflows run 65% deterministic code; a prompt that grows past 1,000 lines is a signal to replace it with explicit rules captures the mechanism behind that ratio. The Vercel team started with a 125-line prompt written by one of their top SDRs. The prompt grew to 1,000 lines as edge cases accumulated. At that size, model behavior became inconsistent. The team replaced the accumulated routing logic with 14 explicit deterministic rules, and inference cost dropped to roughly $1,000 per year. The prompt's job now covers the 14% of interactions where language judgment is actually necessary. Tunguz:
"The model shouldn't get to decide, we just want that to occur. So now actually, our inbound is 14 rules."
This card pairs with the earlier Codifying the best human workflow before deployment is the critical bottleneck for AI inbound sales agents, not model capability (September 15): codifying the best human workflow is the prerequisite; this is what the architecture looks like once that workflow is captured.
- Tomasz Tunguz, Production AI agent workflows run 65% deterministic code; a prompt that grows past 1,000 lines is a signal to replace it with explicit rules. Production AI inbound agents run 65% deterministic code; a 1,000-line prompt is a signal to replace routing logic with explicit rules.
How Airbnb builds AI-native engineering infrastructure
Ahmad Al-Dahle appeared on Latent Space on October 2 to describe Airbnb's AI-native engineering platform from the inside. Two mechanisms stand out as transferable claims.
The first is Everest, Airbnb's internal codebase knowledge graph. An AI-queryable codebase knowledge graph compresses domain onboarding from months to weeks, enabling generalists to contribute to specialist code documents how it works: prior development context, domain invariants, and past decisions are indexed via embeddings and surfaced through LLM-based retrieval. A generalist engineer entering a specialist part of the codebase queries the graph instead of spending months in implicit knowledge transfer. The compounding effect is visible in Airbnb's own timelines. Grocery delivery, built before Everest captured those learnings, took 8-9 months. Airport pickups, built after grocery learnings were indexed, took 6 weeks. Al-Dahle:
"Because of this context graph that exists across the codebase, we're able to have generalists work across very specialist parts of the code."
The second mechanism is how Airbnb decides which AI use cases to deploy. Production AI deployments require a separate eval suite per use case, sampled from real production queries, not shared benchmarks documents the eval approach: each production AI deployment gets its own eval suite, sampled from actual production queries for that use case, with deliberate overweighting of edge cases in the support domain. This is the mechanism that tells the team which 50% of support tickets are safe to route to agents. Al-Dahle:
"We run evals specifically for each use case. So if we're using a model for search, we have a set of search queries that are sampled from production that we measure against. If we're doing customer support, we also sample all the production queries that we care most about — including edge cases."
The two Airbnb cards are paired: Everest gives agents the codebase context they need; per-use-case evals verify that the context is being applied correctly before increasing automation scope.
- Ahmad Al-Dahle, An AI-queryable codebase knowledge graph compresses domain onboarding from months to weeks, enabling generalists to contribute to specialist code. An AI-queryable codebase knowledge graph compresses specialist onboarding from months to weeks by converting tacit engineering context into on-demand retrieval.
- Ahmad Al-Dahle, Production AI deployments require a separate eval suite per use case, sampled from real production queries, not shared benchmarks. Each AI use case in production needs its own eval suite sampled from real production queries; the per-use-case eval is what determines which tickets are safe to automate.