GoKawiilComposo launches DishwasherBench to test AI models on virtual dishwasher loading
Composo released DishwasherBench, a browser-based benchmark called Spotless that simulates loading a Bosch SMV4HAX40G dishwasher with rigid-body physics. Both humans and language models (via MCP) can play the game, earning points for items that come out clean after a simulated Eco 50 wash cycle. The current version, 0.1.0, reliably scores whether cups, glasses and pots are placed correctly, but does not yet accurately judge whether plates and cutlery are stacked to wash clean.
GoKawiil's interpretation of the reporting above, not reported fact.
The project frames physical, real-world tasks like dishwasher loading as a new frontier for evaluating AI models beyond text and code, testing spatial reasoning and planning under constraints. Because every episode is a verifiable, replayable physics record, it could offer a more rigorous way to compare human and model performance on everyday manipulation tasks than typical benchmarks. However, Composo's own acknowledgment that plate and cutlery scoring is still unreliable suggests the benchmark's conclusions should be treated cautiously until further calibration.
- DishwasherBench simulates loading a real Bosch dishwasher model using rigid-body physics and spray/soil chemistry.
- Both humans (via browser) and AI models (via MCP) can play the same game, called Spotless.
- The current 0.1.0 release reliably scores cups, glasses and pots but not yet plates or cutlery cleanliness.
Bosch SMV4HAX40G Dishwasher — This is the exact real-world machine DishwasherBench's physics simulation is calibrated against, so owning one lets you see firsthand how loading geometry affects wash results. It's a solid, fully-integrated dishwasher with Bosch's quiet reliable engineering, great for anyone curious about the real mechanics behind the benchmark.
See Bosch SMV4HAX40G Dishwasher on Amazon → Affiliate link — we may earn a commission on purchases, at no extra cost to you. Product picked by AI based on this article; it is not a tested recommendation.Source: github.com, 2026-10-13
Published there as: “DishwasherBench – Can Frontier Models Stack a Dishwasher?”
Read the original report → The summary and analysis above are GoKawiil's own, written from reporting by the source above. Facts and quotes belong to the original publisher.