lithos-ai/lithos-metal is an engine that handles large language model (LLM) inference and serving on Apple silicon. The latest release in the GitHub repository is v0.1.2. The available records do not list the specific items changed in this version, so the release changes cannot be confirmed. The repository generates layer-wise megakernels based on Metal shaders and is published under the Apache-2.0 license.
The core idea is to bundle multiple operations into layer-wise kernels and control the generation process inside the GPU in order to reduce execution overhead. DSpark speculative decoding produces the next token candidates first and then compares them against the actual generation result. In the default configuration it validates 7 candidates together with 1 reference token. The server supports Chat Completions, Responses, Anthropic Messages, and Ollama /api/chat formats. The default context is 32K, and it generates one request at a time.
According to the README, actual runtime verification was carried out on a 40-core M5 Max 48GB with the Qwen3.8-27B and Qwen3.6-35B-A3B combination. The presented 168 tok/s at a 128-token context and 127 tok/s at a 32K context are not direct measurements but target-compute projections. The projection assumes 6 tokens per pass across 8 passes, and it does not include drafting or acceptance processing costs. It therefore cannot be read as an end-to-end performance comparison. The 35B hybrid MoE integration is also experimental, so the current numbers are limited as a basis for judging real-world performance.