Devotd Github
NewInference & serving

aristo94/gsqhalo.cpp — SSD Prompt Cache Restoration Oracle Document

Restored to the documentation of a llama.cpp fork for GSQ-quantized Qwen3.8-Flash-Next the 'restore oracle' section, which verifies whether a conversation read back from the SSD continues identically to the original conversation.

Repositoryaristo94/gsqhalo.cpp ↗

Restored to the documentation of a llama.cpp fork for GSQ-quantized Qwen3.8-Flash-Next the 'restore oracle' section, which verifies whether a conversation read back from the SSD continues identically to the original conversation.

GitHub description: llama.cpp fork for GSQ-quantized Qwen3.8-Flash-Next on AMD Strix Halo (gfx1151): faster ROCm prefill, MTP and 4×256K chats with Q8_0 KV and one KV stream per slot, persistent KV cache on SSD. Fork of halo-box/strix-llama.cpp.

stars
★ 11
Forks
0
Language
C++
License
MIT
Read original source ↗View on GitHub ↗

aristo94/gsqhalo.cpp is a llama.cpp fork that seeks to run GSQ-quantized Qwen3.8-Flash-Next on AMD Strix Halo(gfx1151). Commit 16f0929 on October 11, 2026 touched only two documentation files and did not change any code. In docs/gsqhalo/README.md, the 'restore oracle' section was restored, laying out the conditions and results under which a conversation read back from the SSD continues identically to a conversation that never leaves the slot.

The SSD KV cache is a feature that stores the model's previous conversation state on disk so that the conversation can be continued even after the slot is emptied. The README records that a 29.7K-token conversation came back from disk in 2.5 seconds, while prefilling it again from scratch took 33.7 seconds. The new oracle checks whether this restoration produces the same results as the original conversation. All four cases — reading back into its own slot, reading back into a different slot, reading back after a restart, and reading back while another slot is being created — were bit-for-bit identical to the baseline run (the case where each conversation was run without ever leaving the slot on its own). The slot state hashes were also identical immediately after restoration and for every prompt placement.

The point where the evidence stops is equally clear. When two conversations are generated at the same time, one conversation's results are not bit-for-bit identical to its results when generated alone. X and Y diverged from the baseline run at the 7th and 32nd token respectively, and the logprob difference was at most 0.12. On tokens whose probability exceeds 0.95, it shrank to at most 0.004. Because the batch shape differs, the rounding diverges, and the top-k selection of the QSA and MoE routers amplifies that difference.

With MTP enabled, all four restorations produced the same tokens as the baseline, but Y's second logprob differed by 0.08 at the 28th token. This difference arose because the draft head changed the verification batch. The measurements were made on the code at 7ece4bd, and the commit itself only added 45 lines and deleted 2 lines of documentation. A constraint also remains: state hashes cannot be compared between slots. Therefore, restoration identity can be regarded as a result confirmed under the condition of continuing the conversation without MTP, but it is difficult to regard it as complete identity including concurrent generation or MTP.