hardenedlinux/sonicboom is a native tensor runtime built from scratch in C++ and CUDA. The repository README states the current state as v0.0.2. It exposes functionality through shared libraries without Python and a stable C ABI, and uses GNU Guile as its first language binding.
This runtime plans and executes the computational stages for model inference. According to the README, decoding is handled by CUDA Graph replay, and the KV cache is kept on the GPU device. The planner fuses normalization, activation, and quantization, and uses ggml kernels for quantized matrix multiplication. The README describes the core size as 17.7k lines of C++ and CUDA. The same repository also contains SonicCross, a rewrite of PyTorch's code generator torchgen in Guile. The README's RTX 3050 8GB comparison uses the same GGUF and session as llama.cpp commit 43fe9c64281 and presents the median of three runs. Gemma 4 E4B-it Q3_K_M decoding was 52.7 vs 44.8 tok/s (1.18x), and Qwen2.5-0.5B-Instruct Q4_K_M was 344.7 vs 307.9 tok/s (1.12x).
In the same comparison, prefill was slower than llama.cpp at 0.84x and 0.91x respectively. The scope of validation is limited to CUDA sm_86 and the two model families. The README distinguishes that the correctness of prefix cache reuse was verified not by a speed benchmark but by the test_prefix_cache tests (12/12). Therefore the decode figures are comparison results from a limited environment, and this material alone does not show an advantage on other GPUs or models.