Devotd Github
NewInference & serving

nathanw1014/strix-halo-llamacpp v0.8.0 — llama.cpp build for Strix Halo

A repository that applies flash attention and MoE prefill fixes to llama.cpp for AMD Strix Halo and bundles the current Mesa drivers.

Repositorynathanw1014/strix-halo-llamacpp ↗

A repository that applies flash attention and MoE prefill fixes to llama.cpp for AMD Strix Halo and bundles the current Mesa drivers.

GitHub description: Performance-tuned llama.cpp for AMD Strix Halo (gfx1151): FA + MoE-prefill fixes with a bundled current Mesa driver. Vulkan and HIP; portable dir, Docker, and distrobox.

stars
★ 221
Forks
20
Language
Python
License
NOASSERTION
Latest release
v0.8.0
Read original source ↗View on GitHub ↗

nathanw1014/strix-halo-llamacpp is a llama.cpp build tuned for performance on AMD Strix Halo (gfx1151). The latest release in the repository is v0.8.0. According to the repository description and README, it patches the flash attention (FA) and mixture-of-experts model (MoE) prefill paths, and it ships the current Mesa drivers alongside. It covers the Vulkan and HIP paths, and it also provides a portable directory layout as well as Docker and distrobox setups.

The purpose of this repository is to collect in one place the build and the patches needed to run llama.cpp on Strix Halo. The README explains the measurements and the path to the raw results from comparing the upstream baseline build against its own build on the same hardware and in the same session. It also treats prefill at deep context and quantized cache behavior separately. So the material provided lets you see which comparisons were made in a specific configuration.

The performance figures in the README, however, are results reproduced with a specific build and settings, and they are not a general performance guarantee. No evidence is given that the same results hold on other GPUs or configurations. The license is listed as NOASSERTION. This repository is a concrete build option for Strix Halo, but it does not provide the basis for generalizing that performance.