github.devotd.
ReleaseMultimodal

0xshug0/audio.cpp v0.9.2 — ASR Models and Streaming

A look at the ASR and TTS features of 0xshug0/audio.cpp v0.9.2, a pure C++ ggml-based speech inference engine, and its performance path changes.

Repository0xshug0/audio.cpp ↗

An all-in-one, pure C++ inference engine for audio models, powered by ggml. Supports TTS, STT, VAD, voice conversion, music generation, and more, with highly optimized performance. No Python dependency.

stars
★ 3.6K
Forks
427
Language
C++
License
NOASSERTION
Latest release
v0.9.2
Read original source ↗View on GitHub ↗

0xshug0/audio.cpp is a pure C++ speech inference engine built on ggml, supporting speech recognition and synthesis, voice activity detection, voice conversion, and music generation. The v0.9.2 release published on GitHub is a release that extends ASR and TTS capabilities together with performance paths and platform support. The project's scope — a local inference stack that handles multiple speech tasks without Python dependencies — carries over into this change as well.

ASR gains the experimental Cactus Compute Whistle model plus Danish models Hviske v6 and FireRed, VAD, and streaming. TTS adds speaker inversion embedding conditioning, loading checkpoints without a duration predictor, and max_ref_sec as the upper bound on reference audio length. On the performance path, ggml repacked CPU kernels are applied to LFM2-Audio quantized weights, and a transposed convolution tap that contributes nothing on CUDA is skipped. Speech-to-speech reuses the previous turn's prefill and encoder outputs. This change targets model coverage and inference optimization at the same time.

A CUDA block_reduce shared-memory race bug was fixed by backporting llama.cpp #26385, and a Metal padded-row bug fix is included. SSE streaming, a C API library, and Windows Vulkan build artifacts were also added. However, the provided release information contains no accuracy or throughput figures. Since Whistle is also experimental, the real quality and performance of the new paths are hard to judge from this information alone.