kvcache-ai/mooncake is the serving platform for Kimi, the LLM service provided by Moonshot AI. The most recent release on the repository is v0.3.13.post1, and the given record does not contain the changes included in this version.
The center of the architecture is the KV cache, that is, the intermediate key and value results used in Attention computation. Mooncake places this KV in a distributed storage and transfer structure shared by several servers, and splits it into two layers: Transfer Engine and Mooncake Store. On top of that it separates prefill (the stage that computes the prompt and emits the first token) from decode (the stage that then generates tokens one by one), and adds KV pooling, which gathers the cache across requests for reuse. The implementation language is C++ and the license is Apache-2.0.
The README explains that Kimi can handle 75% more requests while meeting SLOs in real workloads. It also releases tools that analyze anonymized request traces and KV cache. The repository was created on June 25, 2024, and has 6845 stars.
The boundaries are clear, however. The record does not show what v0.3.13.post1 changed from the previous version, and it does not state which workload the 75% figure came from. No comparison with other serving frameworks and no separate benchmark numbers are given either. So this repository is better read as reference material for teams that operate Kimi directly or that want to build a structure sharing KV cache, and it is still too early to treat it as a general-purpose serving platform.