composo-ai/dishwasherbench is a game-like language model benchmark for loading dishes, built on a physics simulation of a single Bosch SMV4HAX40G. The game is called Spotless and can be played in a browser. No official release information was provided. The repository is written in TypeScript and uses the MIT license. The core of it is the process in which the model places dishes on a fixed grid with defined actions and is scored on the wash result.
The simulation stacks dishes with the rigid-body physics engine Jolt and traces the water jets of the Eco 50 program from the spray-arm positions of the actual device. Because the same loading settles to the same result every time, an episode can be reproduced or checked by washing again. A human uses the browser, a model uses the same grid and actions through MCP. There are 7 difficulty levels, and 46 literature-based loading phenomena are tested as executable wash pairs. Scoring at the highest level takes about 5 minutes, and the quick preview takes around 1 minute.
However, the published evaluation is just 6 episodes in total, run by 2 models across the 3 levels of the official tier. Across the 2 complete levels, both models surpassed the scripted reference loading — breakfast 174 points and dinner party 250 points — but wash judgment barely contributes to the score, so the difference came mainly from loading choices. The quality with which dishes are stacked is barely reflected in the score, and spots and drying are not modeled. The technical score is 6.0 on the grounds that it is an unverified small-scale benchmark, and the current results are close to a preliminary evaluation that looks at loading choices rather than washing ability.