jesdga95/basalt forks Strata and is a local inference engine for the Qwen3.8-Flash-Next MoE targeting Linux x86-64, NVIDIA Blackwell (sm_120) and CUDA 13. Storage-constrained experts are placed in system RAM instead of VRAM, and a second GPU can optionally be used as a peer expert tier so the model can run on a single workstation. A storage-constrained expert refers to a model component that does not currently fit in VRAM.
The README reports quality per quantization by processing 20 chunks of wikitext-2 at 512 tokens each. It uses the mean KLD against the Q8_0 baseline plus top-1 agreement: UD-Q4_K_XL scores KLD 0.068 and top-1 93.0%, while NVIDIA's official NVFP4 pack scores KLD 0.076 and top-1 93.0%. GSQ-RCO Q2_0 scores KLD 0.486 and top-1 80.6%, and GSQ-RCO IQ3_XXS scores KLD 0.277 and top-1 86.8%. Some packs include MTP layers refined through self-distillation, and a validation stage decides whether the draft output is finally accepted.
The repository is MIT licensed and contains the C++/CUDA engine, an OpenAI- and Anthropic-compatible Python server, tests, and conversion and quality-check tooling. Official .basalt packs are distributed on Hugging Face. However, the only supported model is Qwen3.8-Flash-Next, and the quality figures rest on a limited chunk evaluation on wikitext-2. So these results alone cannot tell you how quality holds up for other models or in other environments.