seuros/mdma is a Rust-based LLM inference engine described in its README. It implements a GGUF loader, tokenizer, transformer decoder, KV cache, and sampler, with a focus on serving models on hardware the user already owns. The available information lists no public releases, and the repository was created on September 22, 2026. The license is BSD-2-Clause, and it has 14 stars.
The engine supports CPU, ROCm, Vulkan, Level Zero, CUDA, Metal, and Coral Edge TPU under a single Backend trait. According to the README, the kernels for each backend were written directly rather than wrapping vendor libraries. Memory for weights, the KV cache, and workspaces is reserved before allocation, and requests that cannot be accommodated are rejected with RESOURCE_EXHAUSTED. If the whole model does not fit in GPU memory, --gpu-layers auto places as much as possible on the GPU. GPU kernels are checked against the CPU backend and must pass a verification gate where their greedy tokens match a fixed llama.cpp reference. The README explains that CUDA uses the driver API instead of a toolkit, and that Edge TPU programs are generated directly from a decoded instruction set.
That said, the README marks the project status as experimental. The roughly 1,000 tok/s figure is a self-reported number from a non-public system, and the README notes that the public reconstruction has not yet reached it. As an early-stage repository with 14 stars, it has also seen limited real-world validation. So rather than a serving engine with proven performance, it is closer to material worth examining for anyone interested in backends and verification structures for personal hardware.