efficientmoe/moe-infinity is a library for serving MoE (Mixture of Experts, a model architecture that activates only some of several experts to reduce computation) models in PyTorch. The commit a0b48949a67082456befef910b9817b37f8a47b9 of October 26, 2026 added paged attention (a method of managing the key/value cache by dividing it into fixed-size blocks) serving for Qwen3.5-MoE to it. This model family was in a state where the output broke when decoding multiple tokens at once.
The added Qwen3_5MoePagedAttention replaces the attention implementation of only the gated full attention layers with HuggingFace's and connects it to the paged KV cache. The linear attention layer Gated DeltaNet and the vision tower use the original classes as they are, and the cache sizes itself only by the full attention layer index and keys its entries so that no KV region is allocated to the linear layers. The conv and recurrent states of Gated DeltaNet are stored separately per sequence, and only up to max_batch_size sequences (two snapshots per sequence) are kept on the GPU. The other sequences are moved to host memory in order of least recently used and are restored at the next forward.
For image and video prompts, M-RoPE positions are computed with HuggingFace's get_rope_index, and a per-sequence rope delta is applied on every subsequent decode. The same commit also fixed, on a weak-reference basis, the problem where the runner and the engine referenced each other, so that even after the engine was discarded the GPU KV store (5.4GiB per engine on a 32GB V100) remained until a cyclic GC; as a result, the 7 OOMs in test_engine.py on the V100 disappeared. Because of the fix to the offload path, the resident backbone placement for Qwen3.5-MoE changed to the GPU side. The new module boundary/, serving/linear_attention_state.py and a 1162-line test file were added.
However, linear attention states cannot be reused as a cache, so prefix reuse, chunked prefill and speculative execution are turned off with the reason hybrid_linear_attention_state_unsupported, and decode CUDA graph is not a target either. The compatibility document lists this feature only at the level of 'Qwen3.5 paged serving tiny-checkpoint validated (CPU)' and does not present GPU-measured performance figures. It is still too early to draw judgments that can be verified on the actual model.