zhulinsen/litemoe is a small inference engine for mixture-of-experts (MoE) models. No separate release version is given in FACTS. The repository description introduces it as an MoE inference engine you can read in an afternoon. It is written in roughly 1,600 lines of Python and uses vLLM's expert, attention, and router kernels, while the forward computation, key-value cache (KV cache), and generation loop are implemented directly.
MoE is a model architecture that picks some of several experts to do the computing. LiteMoE simplifies the computation path for single-GPU inference in this structure. According to the README, under fixed-batch conditions on A100 and L4, its own comparative throughput was 5–9 percent higher than vLLM's. It gives as the reason the removal of unnecessary operations such as batch masks and index lookups. The implementation language is Python and the license is Apache-2.0.
This throughput figure is the README's own comparative result; it does not mean support for a full serving stack or dynamic batching. The supported scope is also limited to single-GPU inference for Granite 3.1 1B and 3B. The README includes GSM8K test results too, but the presented scope alone makes it hard to judge performance in general serving environments. So it is closer to a resource for looking at a range-limited inference implementation.