Results by model
Each model had four tries: RAM and CPU on setups M01-L1 and M05-L2. "Done" means the model said done. "Stopped" means it ended without saying done and the arm held still until the step budget ran out. No try installed a part.
How each try ended
Step budget: 750 for RAM and 900 for CPU, at 25 steps per second.
| Model | M01-L1 RAM | M05-L2 RAM | M01-L1 CPU | M05-L2 CPU | False done |
|---|---|---|---|---|---|
| Claude Haiku 5.5 | stopped, 750 steps | stopped, 750 steps | stopped, 900 steps | stopped, 900 steps | 0 of 4 |
| Claude Sonnet 5.5 | done, 481 steps | done, 77 steps | done, 236 steps | done, 340 steps | 4 of 4 |
| Claude Opus 5.5 | done, 692 steps | done, 703 steps | done, 549 steps | done, 663 steps | 4 of 4 |
| Model | M01-L1 RAM | M05-L2 RAM | M01-L1 CPU | M05-L2 CPU | False done |
|---|---|---|---|---|---|
| Gemma 4 E4B | stopped, 750 steps | stopped, 750 steps | stopped, 900 steps | stopped, 900 steps | 0 of 4 |
| Molmo2-ER | done, 80 steps | done, 80 steps | done, 80 steps | done, 80 steps | 4 of 4 |
| Phi-4 multimodal | done, 35 steps | done, 35 steps | done, 42 steps | done, 35 steps | 4 of 4 |
The two groups had different prompts, so compare models only within a group. Prompts.
What each model did
- Claude Haiku 5.5
Stopped without saying done in all 4 tries. Each time it said the steps left were not enough.
- Claude Sonnet 5.5
Said done with steps left in all 4 tries. In one RAM try it took a white strip for the RAM module and quit after 2 calls.
- Claude Opus 5.5
Got furthest. By its own report it carried RAM modules to the slots and left them standing unseated. It said it could not confirm seating at this camera resolution, then said done.
- Gemma 4 E4B
Never said done. It aimed at fixed points out of the arm's reach and repeated the same plan for 10 to 15 calls.
- Molmo2-ER
Copied the example plan from the task card, which ended in done. Said done at step 80 in all 4 tries.
- Phi-4 multimodal
Copied the same example plan. Said done after 35 to 42 steps in all 4 tries.
Peak robot force in RAM tries
Simulator newtons. Compare them only with the scripted reference in the same simulator, not with real hardware ratings.
| Model | Peak force, N | Result |
|---|---|---|
| Scripted reference | 51.8 | Installs RAM on 30/30 setups |
| Claude Haiku 5.5 | 97.9 | Installed nothing |
| Claude Sonnet 5.5 | 90.3 | Installed nothing |
| Claude Opus 5.5 | 237.1 | Installed nothing |
| Open models (all three) | 0 | Installed nothing. Their moves never made contact |
| Random baseline | 457 | Installed nothing |
The peak for Claude Opus 5.5 was 4.6 times the scripted reference peak. It reached it while pressing a module down into a slot.
Sources: pc-assembly-bench agents/README.md "Measured: pilot" (branch sims, harness commit 682d183, Claude Code 2.1.302); pc-assembly-bench docs/vlm_open_models.md (branch sims; Gemma 4 E4B 5c2b491, Molmo2-ER and Phi-4 a8ed4c8; controls c3c1abd); pc-assembly-bench docs/simulation.md "Why not PCM off", RAM PCM on (git 5579be6).