github.devotd.
NewInference & serving

nace-ai/kapitsa v0.1.36 — an inference engine for Apple silicon

nace-ai/kapitsa is a C++ inference engine that runs Qwen and Nemotron-family models on Apple silicon.

Repositorynace-ai/kapitsa ↗

Blazing fast Apple Silicon optimized inference engine built for Qwen and Nemotron. OpenAI + Anthropic API compatible. Built by folks from nace.ai

stars
★ 95
Forks
1
Language
C++
License
Apache-2.0
Latest release
v0.1.36
Read original source ↗View on GitHub ↗

The latest release of nace-ai/kapitsa is v0.1.36. This repository is a C++ inference engine for running Qwen and Nemotron-family models on Apple silicon. However, the provided information lists no changes for v0.1.36, so the specific contents of this release cannot be confirmed.

The project says it integrates model loading, compute kernels and a server, and provides Metal compute kernels plus per-model implementations. Speculative decoding is also part of the integration scope. That is a method in which a small model produces token candidates first and a large model verifies them, helping the generation process. According to the README, it also offers OpenAI- and Anthropic-compatible API interfaces and a chat screen. What sets it apart is being an Apple silicon runtime with a compatible server that can be connected to existing applications. The repository also includes documentation, test files and measurement tooling.

The performance table reflects the author's own measurements and was not independently reproduced in this analysis. It is therefore difficult to draw firm conclusions about speed differences against other runtimes or models from the published numbers alone. No changelog for v0.1.36 is given either, so the basis for judging whether performance or features improved is limited. The configuration and API compatibility that can be confirmed do show the scope of the project, but the performance claims still require separate verification.