yamz-labs/kyojin is an inference engine targeting AMD Strix Halo (ROCm, gfx1151). It is built on top of ExLlamaV3 and aims to run MoE models in the 300B class on a single 128GB mini PC. Merged on November 1, 2026, 1.8.5 advertises 'returning sessions, Strix Halo iGPU decode', adding 2341 lines and deleting 145 lines. The two axes of this release are session return and iGPU decoding.
The core of the session return side is the eviction policy of the recurrent cache. The new environment variable EXL3_RC_ANCHOR_MAX has a default value of 16384, and it defers checkpoints at power-of-two positions (2K·4K·8K·16K) from the LRU (least recently used, replacing items that have not been used recently) victim list for as long as other checkpoints remain. The code comment explains that this mechanism prevents the problem where long sessions push out the checkpoint at the end of the prompt, causing compressed and new sessions that share the same system prompt to resume from 0 even though the K/V pages remain in the cache. For prefix recovery, EXL3_PD_BULK was added, with a default of 1 on CUDA and 0 on ROCm. Instead of copying pages one at a time, it reads the prefix back from disk with a single pinned bulk copy, and the explanation adds that bulk copy is actually slower in ROCm unified memory. The default value of QWSPEC_NG_POOL, which determines the number of n-gram followers the server keeps, is 4, and lowering it to 1 leaves only a single follower.
The decoding side consists mostly of kernel and execution path adjustments. gdn.cu takes advantage of the fact that the state access pattern of ordinary GDN operations is the same as KDA, and makes v_split selectable as either 1 or 4 via EXL3_GDN_VSPLIT_GDN; the comment states that the outputs of the two settings are bit-identical. The same comment notes that in iGPU measurements, bandwidth rose from 95 → 172 GB/s (1 row) and 107 → 197 GB/s (history of 5 rows) for 48 x 512 B state rows, and that the kernel is -45%, 4K decode is -2.5%, and 5-row verify is -3.7%. Along with this, a skinny_set_w2 binding was added, column grouping of two columns (CPL=2) for the skinny partials kernel, a HIP-only glue module hip_glue.py, an environment variable cleanup module util/knobs.py, and test files related to anchor·knobs·prefix cache·HIP glue·Triton dequant.
However, those figures are developer-side iGPU measurements left in kernel comments, and no independent benchmarks or README table are given. Because quite a few items diverge in default value between CUDA and ROCm, they are not numbers to read and apply as-is to a different GPU configuration. 1.8.5 is a merged release that also contains the 1.8.4 changes, and the changelog for supported models or license changes cannot be confirmed from this commit's information. If your environment is one that runs a huge MoE on a single AMD APU while needing to hold sessions for a long time, it is a release worth checking, but the supported range is narrowed to Strix Halo (gfx1151).