kindlingai/gr8 is an inference engine written from scratch in CUDA to run the full GLM-5.3 model on 3–6 DGX Spark (GB10) units. The November 7, 2026 commit e231adb ("GLM-5.3-Flash: context-parallel KV (#384) (#414)") added context-parallel KV to this engine's GLM-5.3-Flash build. It adds 1,353 lines and deletes 171 lines across 31 files, with CUDA kernels, runtime, and documentation all changed together.
Context-parallel KV is a method that distributes KV cache pages (in units of 64 rows) across multiple ranks, so that each rank holds only part of the full context instead of replicating it. In the Flash build the default is 0, so it must be enabled with EXTRA_ENV=GR8_CP=1; the first R tokens (GR8_CP_R) are replicated across all 12 attention bindings, and the pages after that are placed on a single rank by the (pos / 64) % TP rule. Decoding beyond R proceeds in five stages per DSA layer. Only the rows this rank is responsible for are inserted, only its own pool is scored to leave the top 512 per row, and then the candidates are all-gathered and merged bit-identically with the list of the replicated path. Next, the absorbed queries are gathered, partial attention is computed per head group, merged with log-sum-exp in rank order, and then the value projection is performed. FLASH.md states that the list and the cache bytes are the same as in the replicated path, but the output is not bit-identical. This is because the order in which the sums are merged differs.
The feature nevertheless remains at an experimental stage. In the Flash build, prefill for CP bindings is still rejected, and Flash prefill runs on the replicated path for prompts below R. m4sim rejects prompts exceeding R at --prefill. MTP layers hold the entire KV as-is, and checkpoints store only the 11 DSA layer pages that each rank is responsible for, with the entry format rising to 3 and entries in the previous format (2) being discarded. The documentation states that there are not yet any results measured on a cluster. A test (engine/tests/test_flash_cp.cu) that compares against the replicated path on a single GPU was added, but the throughput and accuracy differences when run over multiple ranks are not recorded. For serving long contexts with Flash, watching it for now seems to be the right call.