1picassoai/coachwhip is a mixture-of-experts (MoE) inference engine for Apple silicon, written in Metal and Rust. The latest release is v0.5.3, but the available record lists no changes specific to that version. The repository description states that it runs an 80B Qwen3-Coder-Next fully offline at about 13 tok/s on a 16GB Mac.
For model configurations that fit in memory, the engine keeps all 256 experts across 40 layers resident on the GPU, so no SSD streaming is needed. An expert is a bundle of weights that handles part of the computation in an MoE model. The 80B coding model and gpt-oss-120b take a separate path that streams experts from SSD. Unlike llama.cpp or the MLX family, the design owns its Metal kernels and its Candle-derived engine directly. The README lists a generation speed of 38 tok/s for Qwen3.6-35B on a 16GB M4 and a prompt processing speed of about 300 tok/s for a 1500-token prompt.
The requirements are Apple silicon M1 or later, 16GB of memory, macOS 15 or later, and 15GB of free disk space. An OpenAI-compatible API and the integration steps for Aider are also documented. That said, the performance figures come from the repository's own benchmarks and have not been independently verified, so the listed speeds are for reference only, and the available record contains no external verification backing up real-world performance.