Claim
Evaluating model quality by token-count output (tokenmaxxing) became a misleading heuristic once real agent deployments revealed that chained inference across multi-step tasks multiplies per-token costs to a scale where cost-per-outcome is the actual constraint, not raw model capability.
Mechanism
Tokenmaxxing assumed that models generating more tokens per response were higher-quality reasoners. In single-call evaluations, that correlated with reasoning depth. In agentic workflows, each task step is a separate inference call. A 30-step agent task multiplies the per-call cost by 30. At frontier model prices, that arithmetic makes high-token-count behavior a cost liability rather than a quality signal. The dominant constraint shifts from "which model is most capable per call" to "which model completes the task at an acceptable total cost." Willison observed this directly: real deployments of agents hit expensive ceilings that benchmark-pass-rate comparisons never surfaced.
Conditions
Holds when: agent tasks require many sequential model calls at frontier prices; the task completion cost scales with per-step token count.
Fails when: tasks are single-call or very short chains; cached intermediate outputs flatten the cost curve; sub-frontier models are already in use and token count was never the bottleneck.
Evidence
Willison described the shift in his September 2026 year-in-review keynote at WeAreDevelopers World Congress North America, observing that after tokenmaxxing emerged as a trend in late 2025, real agent deployments corrected it:
"the agents are expensive"
The observation captures the corrective signal. The tokenmaxxing proxy held when agents were hypothetical. It broke when practitioners ran agents at scale and received billing surprises.
Signals
- Cost overruns on tasks that appeared cheap in single-call benchmark conditions
- Teams routing agent-step calls to smaller or cheaper models rather than frontier models
- Per-task cost becoming a first-class evaluation criterion alongside benchmark accuracy
Counter-evidence
Some agentic tasks genuinely benefit from high-token reasoning at each step. Complex multi-constraint planning, adversarial review, and long-chain reasoning problems may justify the cost if they compress human review time. The critique of tokenmaxxing lands on tasks where token count measured verbosity rather than reasoning depth. For those tasks, the signal was always misleading; agents just made the cost of the mistake visible.
Cross-references
- Max extended thinking exhausts the full 128K output token budget on routine prompts and returns nothing: the mechanics of token budget exhaustion in extended thinking mode, a related failure where token-maximizing behavior produces no usable output at all.
- Agentic coding removes the time-cost filter that once kept poor architecture decisions out: agentic workflows shift the dominant constraint from time to cognitive load; cost is the third constraint that emerges at deployment scale.
- Agents can build functional game interfaces but cannot independently engineer engaging gameplay loops, because fun is a subjective judgment that no specification can substitute for.: from the same keynote, on what agents cannot self-evaluate regardless of token count or model quality.