yuhai-china/ds4-spark is an inference engine that runs DeepSeek V4 Flash locally based on antirez/ds4, and it uses Markov speculative decoding (speculative decoding, that is, a method that computes candidate tokens first and verifies them together). What commit 9ba9202 of June 29, 2026 changed is the side that produces draft tokens in this speculative path. The structure changed so that the output head (output head) is applied directly on the GPU to the hidden state (hidden state, HC) of the 42nd layer to produce draft candidates.
The output head is the final part that turns the hidden state into a score array (logits) of vocab (vocab) size in order to pick the next token candidate. This commit runs the output head encode one more time inside the Metal command buffer at the point where the 42nd-layer HC is already stored on the GPU, leaves the result in markov_logits_gpu (505 KiB), and reads it back into markov_logits_cpu (505 KiB CPU) after the commands finish. As a result, only the argmax of the already computed logits remains in the draft stage. The commit message describes this process as sub-ms, and states that the removed CPU-side Q8_0 dequantization (dequant) was the bottleneck of 500M scalar multiplications and additions. Along with it, the output tensor check in weights_validate_layout was changed to drop the Q8_0 type requirement and keep only the general layout check.
However, this commit does not present any performance figures. The next step that the commit message itself lists was verification of the speculative decoding hit rate (hit rate), which presupposes a full GGUF rebuild, and those results have not yet been presented. The buffer cost of an additional 505 KiB on each of GPU and CPU comes along with it as well. The changes visible in the patch are limited to the Metal command buffer path, so no information is given about what impact there is on other backends. The only modified file is ds4.c, with 38 changed lines (31 additions, 7 deletions). The repository description says only that it is a local inference engine for DeepSeek V4 Flash, and does not specify a license or benchmarks. The source is commit 9ba9202 of github.com/yuhai-china/ds4-spark.