Three new cards from two operators. All three are about what AI agents do in practice. Where they fail to retrieve, what they can be manipulated into revealing, and what tests no longer predict their real-world performance.
Theme 1: The retrieval breakdown at the pricing page
Kevin Indig published a study on July 13 covering 1,500 agent runs across 100 B2B products (B2B pricing pages are where AI agent retrieval breaks down and third-party directories take over vendor narratives). Agents were given buyer tasks (pricing and features, integrations, security) with no starting links. They had to find vendor sites independently.
Integrations and security returned 93% and 92% first-party answer rates. Pricing came back at 79%. The gap is explained by three failure modes: opacity (no published price), machine-unreadability (prices locked in JavaScript or PDFs), and access friction (bot-blocking, login walls). When any of these fires, the agent falls to third-party sources. Pricing generated 77% of all third-party citations in the study, versus near-zero for integrations and security.
The access friction number is the operative one for B2B marketers. Access errors appeared in just 7% of runs. When they did, third-party fallback jumped from 17% to 77%. Blocking agents from a pricing page makes it nearly five times more likely a buyer hears pricing from G2 or Capterra rather than the vendor.
"AI agents turn websites from showrooms into barcodes."
- Kevin Indig, B2B pricing pages are where AI agent retrieval breaks down and third-party directories take over vendor narratives. Access friction on pricing pages sends 77% of AI agent pricing citations to third-party directories outside vendor control.
Theme 2: The exfiltration vector hiding inside a patched tool
Simon Willison documented researcher Ayush Paul's attack on Claude's web_fetch tool in a July 15 post (Chaining a honeypot site's nested links through web_fetch exfiltrates private user data letter-by-letter, bypassing injection safeguards that only block direct URL attacks). Anthropic had already blocked web_fetch from visiting URLs injected directly into a conversation. That block left a gap: the tool could still follow links within pages it fetched.
Paul built a honeypot site with a social engineering pretext. The site's initial page linked to /a, which linked to /b, and so on. Each URL in the chain encoded the next character of exfiltrated data from the user's conversation memory. Claude navigated the chain thinking it was completing a legitimate authentication flow. It extracted user names, home city, and employer in the process.
Willison names the underlying structure the "lethal trifecta": private data accessible via memory + a web-fetching tool + that tool's ability to both receive hostile instructions and exfiltrate data through URL navigation. Anthropic patched by removing web_fetch's ability to follow embedded links. The attack also used a Claude-User agent check to serve the honeypot only to Claude, making it invisible to human visitors.
The trifecta pattern persists as a design constraint for any agent with memory and web access. The specific chained-link vector is closed. Novel paths through other tool combinations are not.
- Simon Willison, Chaining a honeypot site's nested links through web_fetch exfiltrates private user data letter-by-letter, bypassing injection safeguards that only block direct URL attacks. Chaining a honeypot site's nested links through web_fetch extracted private user data character by character, bypassing Anthropic's direct-injection safeguard.
Theme 3: The benchmark that stopped predicting
Willison also posted on July 16 after running Kimi K3 through his pelican benchmark (generate an SVG of a pelican on a bicycle) (Single-task visual benchmarks have decoupled from frontier model rankings; agentic tool-calling reliability now differentiates model performance). GLM-5.2 outperformed Claude Fable 5 and GPT-5.6 on the task. GLM-5.2 is not a comparable-tier model overall.
The observation: the pelican benchmark "has mostly been severed" from model ranking. The test measures spatial reasoning and SVG validity in a closed single-turn task. Frontier differentiation has moved elsewhere. Willison identifies agentic tool-calling reliability in long multi-turn conversations as the capability that now separates models in production.
The pelican test retains value as a cost probe and reproducibility check. It has lost value as a comparative signal. Single-task visual benchmarks optimized for one capability class do not predict performance in the class that agents actually run.
- Simon Willison, Single-task visual benchmarks have decoupled from frontier model rankings; agentic tool-calling reliability now differentiates model performance. Single-task visual benchmarks have decoupled from frontier model rankings; reliable agentic tool-calling in long conversations is now the differentiating capability.