The bundle-20261018e release of professorpalmer/bonsai-ada-surgery moved the full cache save for long conversations from immediately after the response to idle time. When the conversation continues, it extends incremental save connections, and when it reaches the cap, it performs a full save. The engine binary is the same as in the previous bundle, and this change focuses on reducing cases where saving blocks the next request.
This repository distributes the Windows bundle of the Bonsai 2 27B model along with work receipts. According to the repository description, it supports a 262k context (training-time maximum) at 12 GB, with decoding speeds from 54 to 97 tok/s and 2x prefill. The q8_0 KV setting and 9/9 grammar-based tool calling are also listed in the description. The weights use the same ternary method as the previous bundle. These are the configuration and figures presented by the repository; the core of this release is not a change to the model or inference engine, but a change in how saving is handled.
No basis was given for claiming that this change raises generation speed. Deferring the full save and extending incremental saves is tailored to reducing cases where the next request gets blocked. However, the cap on incremental save connections and the criterion for idle time were not given, so the detailed conditions of the actual saving behavior are hard to judge from this information alone.