flashml-org/freetoken v0.1.3 is a Python local serving engine for running large models on a desktop. Its README says it drives MoE models with 290B+ parameters at conversational speed even on a gaming PC. The goal is to bundle GPUs, CPUs, host memory, and interconnect into a single inference platform. It is licensed under Apache-2.0, and the repository has received 14509 stars.
The engine focuses on serving models on edge hardware by leveraging multiple devices and memory resources together. Its README lists as core features a bandwidth-adaptive CPU-GPU co-execution policy (q*), whole-layer double-buffered prefill streaming, and global LRU expert caching. Semantic anchor checkpoints preserve recursive state and the KV cache when editing context, such as tool calls or reasoning blocks, to avoid recomputation. It also explains that VRAM can be reallocated between the expert cache and KV memory without restarting the engine or reloading weights. It supports DeepSeek-V4-Flash, Qwen3.6-35B-A3B, and GLM-5.2 in MXFP4, NVFP4, FP8, and BF16, and offers Anthropic- and OpenAI-compatible APIs. It documents Windows and Linux desktop apps, uv and pip installation, and a source build path.
That said, the README presents no separate test or benchmark figures. So it is difficult to compare the actual performance level based only on the claim that it runs at conversational speed on a gaming PC. The features and supported paths are concrete, but the basis for the performance claims cannot be verified with published numbers.