Apple Silicon Inference
Complete local model execution engines primarily specialized for Apple Silicon, MLX, Metal, or Apple unified memory. Excludes standalone kernels, production serving systems, and runtimes primarily designed for constrained mobile or embedded devices; those take precedence even on Apple hardware.
| Project | What it does | Stars | Language | License | Latest release |
|---|---|---|---|---|---|
| jundot/omlx | OMLX is a local Apple Silicon LLM server with continuous batching, SSD and KV caching, and menu-bar management. | ★ 22.9K | Python | Apache-2.0 | v0.7.1rc1 |
| lithos-ai/lithos-metal | A complete local LLM engine with Metal megakernels and speculative decoding for Apple Silicon. | ★ 291 | Python | Apache-2.0 | v0.1.2 |
| 1picassoai/coachwhip | An offline Apple-silicon MoE inference engine runs large models locally and exposes an OpenAI-compatible API. | ★ 58 | Metal | Apache-2.0 | v0.5.3 |