synchopate/nemotron-30b-cpu-256k is a repository that holds the configuration and code for running a 30B MoE model on a CPU with a 256K context, without a graphics processing unit. The repository was created on October 29, 2026, and FACTS lists no latest release. The README explains that it uses a hybrid Mamba-2 SSM, L3 NTA slicing, AVX-512 VNNI tile rearrangement, and 2MB huge pages. The throughput claim stated in the repository description is 30-50+ tok/s.
The purpose of the project is to present the optimized configuration needed to run a large language model with long context on a CPU. According to the README summary, the repository holds state space model patches, cache optimization code, a prediction module, and the related test files. The techniques listed in the repository description cover model execution, cache handling, and CPU compute paths together. The language is Python and the star count is 21. It is therefore a case where the components and the related files can be examined, not only the introductory text.
However, 30-50+ tok/s is a figure recorded in the repository description, and whether it has been independently verified is unknown. The repository has test files related to the KV cache and two benchmark reports, but that fact alone does not mean the performance claim has been verified. No license information is given either, so the conditions for reuse cannot be confirmed. The current material is therefore meaningful for looking at the implementation configuration of CPU inference optimization, but it has its limits as a basis for judging performance or reusability.