a builder's codex
codex · operators · Simon Willison · ins_willison-blender-headless-agent-loop

Giving a coding agent access to a professional visual tool with a Python API enables autonomous render-inspect-edit iteration without human direction between rounds

By Simon Willison · Independent developer, co-creator of Django, creator of Datasette; technology journalist and blogger at simonwillison.net · 2026-09-05 · post · TIL: Using Blender with coding agents on macOS

Tier B · TL;DR
Giving a coding agent access to a professional visual tool with a Python API enables autonomous render-inspect-edit iteration without human direction between rounds

Claim

A frontier coding agent given access to Blender via its Python API can produce, render, and iteratively refine 3D scenes from natural language alone. The human provides the initial prompt; the agent drives the Python scripting loop that generates geometry, renders output, and revises based on its own inspection.

Mechanism

Modern visual tools expose a complete scripting interface through their Python APIs. Blender's API covers the full scene graph: geometry, materials, lighting, camera placement, and render settings. A coding agent given the application path and a natural language target writes Python that assembles the scene, triggers a render, inspects the result, and writes the next revision.

The loop is: natural language prompt to agent, agent writes Blender Python, Blender executes and produces a render, agent inspects the output image, agent revises the Python for the next pass. The human is outside this loop entirely between rounds.

"Modern frontier models have got really good at using Blender."

Willison tested this by prompting an agent to render a pelican riding a bicycle. The agent generated a scene with a pelican wearing a hat and scarf, surrounded by flowers and balloons, near a seaside setting. The full run cost approximately $4.24 at equivalent gpt-6-astra API pricing under a ChatGPT subscription.

Conditions

Holds when: the visual tool exposes a Python scripting API that covers the full scene-construction surface, and the coding agent has access to the application path and permission to invoke it. Best applied to structured, inspectable visual outputs where correctness is observable from the render image alone.

Fails when: the visual output requires human aesthetic judgment that the agent cannot evaluate from a flat render, or when the tool's API does not expose enough of the underlying scene graph for programmatic construction. Also less reliable for highly constrained output (exact physical dimensions, production-ready assets) where iteration without a human design review introduces compounding errors.

Evidence

Willison documented the result on September 5, 2026. The pelican scene output demonstrates a working render-inspect-edit loop from natural language, with the agent handling all Python scripting without human review between iterations. The $4.24 cost figure is based on Willison's calculation of equivalent API usage at gpt-6-astra pricing.

Signals

Counter-evidence

Willison's test used a qualitative, creative scene where "good enough" was defined by entertainment value rather than production accuracy. The same loop applied to architectural visualization, product rendering, or animation production would face much higher correctness requirements that iterative self-inspection may not meet.

The agent is also working from a single render frame per iteration. Complex 3D problems often require multiple viewpoints or animated previews to evaluate correctly. A flat render may not surface geometry errors, clipping, or camera placement problems that would be obvious in a turntable view.

Cross-references

Open the interactive view → View original source → Markdown source →