Simon Willison asks every new model for "an SVG of a pelican riding a bicycle", and people now wonder whether labs quietly train on pelicans. If Claude had practised pelicans, its pelicans should improve more across generations than birds nobody benchmarks. So I set the brown pelican beside two look-alikes, a white stork and a grey heron. Claude Opus 4.5, 4.8 and 5.5 drew all three as SVGs and built the pelican and stork as 3D models: same prompt, two tries each.
Short answer: no sign of a pelican-specific gain from 4.5 to 5.5. The pelican improves about as much as its look-alikes.
What this can't see: pelican practice that already happened before Opus 4.5 (4.5's pelicans are already decent), and anything outside Claude. It is also exploratory, not a planned test: I added the pelican after I had already rated the first zoo organisms, and the stork after the pelican. Further down, the zoo gets harder body plans (a bee, an octopus, a chameleon, a bat, a seahorse, a sea star), built by every Opus version from 4.5 to 5.5.
Scroll sideways, or tap to open full size.
Prompt: "Generate an SVG of a brown pelican" (and the same for the white stork and the grey heron). All three birds get better from 4.5 to 5.5, and at 5.5 the stork and heron look as good as the pelican (judged by eye; the drawings weren't rated blind). Opus 5.5's 2D pelican does get the long bill, the pouch and the dark hindneck right.
Scroll sideways, or tap to open full size.
(The mallard is there as a bird nobody benchmarks either; there is no 3D heron.) Here each model wrote a Blender Python script for a "biologically accurate" whole organism, with no reference image. Opus 5.5's pelican has the right neck but a far-too-small bill with no pouch: the pelican it knows in 2D doesn't carry into 3D.
Scroll sideways, or tap to open full size.
The grid shows each model's first try; both tries of every organism are below. The pelican and stork joined a small zoo of 6 organisms: a mallard, the fly agaric mushroom, maize, and Welwitschia, a desert plant that grows just two leaves in its lifetime. Opus 4.5's first mallard is a grey body with a green head; Opus 5.5's is a drake with a white collar, chestnut breast, blue wing patch and orange legs. No model built Welwitschia's two leaves that later split: each built many separate straps from the start.
I rated older-vs-newer pairs of matching tries blind (forced choice, models hidden, sides shuffled, checked against photos). The newer model looked more like the real organism in 32 of 36 pairs. Opus 5.5 was picked in 24 of its 24 pairs; between 4.5 and 4.8 it was closer, 8 of 12 for 4.8. Deliberately broken specimens (a headless bird, a capless mushroom) were mixed in to check I was paying attention; I caught 6 of 6. They were copies of rated Opus 5.5 specimens, so blinding was partial for 5.5 and the checks were easy. On their own sheets the pelican went to the newer model in 5 of 6 pairs and the stork in 5 of 6, so the rated 3D evidence shows no pelican edge either, though Opus 5.5 won every pair it was in, so these ratings are near their ceiling too.
Those organisms are famous and fairly simple in shape, so maybe they were easy. I added four with harder body plans: the western honey bee, the common octopus, the panther chameleon, and a velvet worm (Euperipatoides rowelli), an obscure animal. Opus 4.5, 4.8 and 5.5 built each twice (grid). The velvet worm was easy for all three. The other three showed a gap, so I ran every Opus version from 4.5 on that the subscription serves. The zoo still had no mammal or fish, so I added a grey-headed flying fox, White's seahorse and an ochre sea star (built in fives, no left or right side), all by every version.
Scroll sideways, or tap to open full size.
First tries shown. Opus 4.5's octopus stands upright like a squid, on arms made of strings of beads, and its chameleon is a swollen oval on stub legs. Opus 4.7's chameleon has tipped over, and some of its legs float free of the body. Opus 5.5's has the curled tail and the banded body. Every flying fox has wings; Opus 5 and 5.5 give them finger bones. Opus 4.5's and 4.8's seahorses are segmented columns with no real head; 4.6, 5 and 5.5 build the snout and curled tail. The sea star was easy for all of them. All 78 of the 78 runs in the dataset built a model. The grid I reviewed had one deliberately edited specimen mixed in, a chameleon with a leg removed, and I spotted it.
Click one to rotate it. Each links to the script the model wrote. These are the six zoo organisms; the harder body plans are in the dataset.
Write a complete Blender Python (bpy) script that procedurally generates a biologically accurate 3D model of a whole brown pelican (Pelecanus occidentalis).
Runtime: the script runs on Blender 4.2.0 — use only bpy APIs valid in Blender 4.2 (note Blender 4.x renamed several Principled BSDF sockets, e.g. 'Subsurface' -> 'Subsurface Weight'). Keep material setup minimal and defensive so one wrong socket name cannot abort the whole script (guard optional material tweaks in try/except).
Avoid these bpy 4.2 API mistakes — each one aborts the whole script:
- bmesh.ops has NO create_cylinder/create_cube/create_sphere. Make primitives with bpy.ops.mesh.primitive_*_add, or bmesh.ops.create_cone (a cone with equal radii and capped ends is a cylinder) / create_circle / create_icosphere.
- After an operator that deletes or recreates an object, any Python variable still pointing at the old object is dead ('StructRNA of type Object has been removed'); re-fetch it from bpy.data.objects or bpy.context after such a call.
- A node socket's default_value must match its type: a scalar socket wants a float, a color/vector socket wants a tuple — never assign a tuple to a float socket.
- bpy.context.active_object and bpy.context.object can be None; check before touching .data. Deselect with bpy.ops.object.select_all(action='DESELECT'), not a .clear() call.
Requirements:
- Build real geometry for the whole organism: every structure a specimen visibly has.
- Model exactly ONE whole specimen — not a cluster, not a scene, not a detached part.
- Leave the finished specimen in the scene. Saving it is handled for you — do not write any file, and do not clear the scene at the end.
- The script must run headless under `blender --background --python` with no user interaction and no external asset files.
- Output ONLY the Python script — no explanation, no markdown prose.
Generate an SVG of a brown pelican
Every 3D specimen with its final script, the 2D drawings with the models' raw replies, run metadata, the rating sheets and my answers: dataset. The code that ran and rendered everything is available on request (Community tab above).
Photos are from iNaturalist, each under its own licence (stated on its iNaturalist page), cropped and resized for this page and its figures. Everything else here is CC BY 4.0, except the photo panels inside the figures, which keep their photo's licence.
No leaderboard. 3D specimens are model-written code run headless in Blender 4.2 and rendered in Blender 4.5 with identical settings; 2D drawings are rendered in Chromium at one size.