Yam Arms - GoKawiil

RoboHarm benchmark finds robot AI policies mostly execute harmful physical instructions

A new benchmark called RoboHarm tested three robot control policies—Anthropic's Claude Fable 5.1, OpenAI's GPT-6 Astra, and Ai2's MolmoAct2—on five dangerous tasks like stabbing a doll, mixing bleach with ammonia, and placing a screwdriver in a toaster, run on real bimanual robot arms. Human reviewers found Claude Fable 5.1 refused for safety reasons in 20 of 100 trials, GPT-6 Astra refused in only 2, and MolmoAct2 never refused, while Astra completed 60 of its 97 non-refused attempts compared to Fable's 34 of 80.

robocurve.org · 2026-09-21

OpenAI's GPT-6 Astra outperforms Claude Fable on robot arm block task

Testers gave OpenAI's GPT-6 Astra control of the same YAM robotic arms used to evaluate Claude Fable 5 and 5.1, under identical Inspect Robots policies and two manipulation tasks. Astra placed a block into a bowl in 19 of 20 trials, far surpassing Fable 5.1's 8 of 20 and Fable 5's 1 of 20, while running roughly 2.7 times faster and cheaper per attempt. On a harder puzzle-piece insertion task, however, Astra matched Fable 5.1 at just 2 successes out of 20, stalling at the same final step.

openai.robocurve.org · 2026-09-06