PistonBenchI built my own version of Minecraft for PistonBench. Blocks have different names and short descriptions of how they work. Models have to reason from those descriptions to build a working door.
Get the data

Can models build
a piston door?

Working piston doors. Limited space. Pass/fail feedback.
A practical test of spatial reasoning, one block at a time.

Explore the results
55.1%

Best pass rate27 of 49 sizes · Claude Fable 5.2

13Models

49Door sizes

10Labs

Leaderboard

Hard benchmark. Pass rate across all 49 door sizes.

Filter by lab
13 models
Model
0%25%50%75%100%
Pass rate
Each size is weighted equally. Tied scores share a rank.Explore individual results

Score vs. cost

Average price per pass for the full 49-door benchmark.

Lower cost, higher score

Overall score (%)
Average price per pass (USD, log scale)
Select or hover a dot for details.Download cost data

A closer look
at every door

The same model can handle a tall door and fail a wide one. See where each build works.

Pass rate by door size

13 models
Door height
Door width

Select a door size to inspect the result.

How the benchmark works

One goal: a fully functional piston door, within the allowed space.

49 door sizes

Every width and height combination from 1 × 1 to 7 × 7. Each size contributes one binary result: passed or not passed.

Space depends on the door

The space allowance varies by size, at roughly three times the space of the smallest known build for that door size.

Hard means limited feedback

Models learn only whether the build passed or failed. After the first attempt, they get five turns to repair it, without diagnostic hints.

A simple score

Pass rate is the number of passed door sizes divided by 49. Every size has equal weight; models with the same score share a rank.

Price per pass

Average prices per pass cover the full benchmark, in USD.