eelgaev/legion is a Rust inference engine that runs Mixture-of-Experts (MoE) models on NVIDIA unified memory systems. According to the repository description and README, it targets the IBM AC922 and DGX Spark, and supports GGUF and Hugging Face safetensors weights. Its GPU kernels use PTX and ggml quantization kernels together. There is no public latest release, so this post covers the features presented in the repository rather than the changes of a specific version.
When a model does not fit in video memory (VRAM), the project presents keeping the experts in host memory and reading them in when needed, or caching frequently used experts in video memory. Both weight formats are handled in the same scheduler and speculative decoding path. Speculative decoding works by generating candidate tokens first and then verifying them. The README also describes continuous batching and prefix context caching, and the repository description explicitly states support for NVFP4 and FP8 weights. The license is Apache-2.0.
The README's performance table compares several devices and models, but it also states the measurement conditions and the unfinished items. It is therefore difficult to generalize the comparison results as performance across all environments. The repository was created on 2026-11-04 and has 4 stars, with no public releases. The currently verifiable evidence is limited to the repository description and the feature and performance materials in the README, so judging performance requires looking at the remaining measurement items.