Devotd Github
NewInference & serving

jay7828/dooms v0.6.1 — An inference engine for Apple Silicon

jay7828/dooms is a C++ inference engine that runs language, speech, and vision models on Apple Silicon using Metal and Core ML, and the latest release in the provided material is v0.6.1.

Repositoryjay7828/dooms ↗

jay7828/dooms is a C++ inference engine that runs language, speech, and vision models on Apple Silicon using Metal and Core ML, and the latest release in the provided material is v0.6.1.

GitHub description: Fast inference engine for Apple Silicon: 4-bit LLMs at 256 tok/s (MLX: 224), Whisper, Kokoro TTS. Metal + Core ML, Python/C++.

stars
★ 1
Forks
0
Language
C++
License
Apache-2.0
Latest release
v0.6.1
Read original source ↗View on GitHub ↗

jay7828/dooms is a C++ inference engine for Apple Silicon. The latest release in the provided material is v0.6.1, but the changelog for that version was not given, so what changed in this release cannot be confirmed. The repository description states that it supports 4-bit language model inference, Whisper speech recognition, and Kokoro speech synthesis. It also cites a figure of 256 tokens per second, while the MLX figure listed for comparison is 224 tokens per second.

The engine aims to run language, speech, vision-language, and vision-language-action models using Metal and Core ML on Apple Silicon. It provides C++, C, and Python interfaces, and the README explains that the Llama and Qwen 2 series are handled by its own Metal decoder while other GGUF models are supported through llama.cpp. The repository contains CMake build configuration, engine sources, and benchmark code, and the README presents comparative measurements of text, speech, and vision models along with reproduction scripts. The license is Apache-2.0.

However, the published performance figures are measurements taken in limited environments such as the M1 Pro, and the README does not guarantee the same performance on other machines. The throughput quoted in the repository description is therefore hard to treat as general Apple Silicon performance. Since the measurement conditions are restricted, any performance judgment should stay limited to the environment presented.