github.devotd.
NewInference & serving

jpb18/polystrata-hip — An AMD HIP fork of the Strata-family inference engine

jpb18/polystrata-hip is a derived repository that ports the Strata-family inference engine strata-poly to AMD HIP so that mixture-of-experts models can run on AMD GPUs.

Repositoryjpb18/polystrata-hip ↗

Run large AI models on your own PC: Qwen3.8-Flash-Next, GLM 5.3 Flash, Qwen3.6, Ornith 1.5 and Gemma 4. One-click install for Windows and Linux, OpenAI/Anthropic API on localhost, image input. Built on Strata.

stars
★ 1
Forks
0
Language
C++
License
MIT
Read original source ↗View on GitHub ↗

jpb18/polystrata-hip is a personal fork that ports the Strata-family experimental inference engine strata-poly to AMD HIP. No new version or public release is presented in FACTS. The repository description sets out the purpose of running models such as Qwen3.8-Flash-Next, GLM 5.3 Flash, Qwen3.6, Ornith 1.5 and Gemma 4 on a personal PC. One-click installers for Windows and Linux, OpenAI and Anthropic APIs used on localhost, and image input are also included in the description.

The core of this repository is the path to running mixture-of-experts (MoE) models on AMD accelerators. A mixture-of-experts model uses only some of its experts in the computation, depending on the input. According to the repository description, experts that fit in graphics memory are kept as a cache and the rest are computed by the CPU. It uses Q8_0 for the attention key/value cache, and introduces an implementation of VRAM budget control and adaptive expert caching. It also cites a figure of 70.7 tokens/second, including MTP speculation, when running Qwen3.6-35B-A3B UD-Q3_K_M with roughly 32K context on an RX 9060 XT 16GB.

Performance and quality validation were carried out only on the combination of Windows 11, Radeon RX 9060 XT 16GB, Ryzen 5 5600X and 32GB memory, and the repository does not present it as a guarantee. These results should therefore be read as a record from that configuration. You can look at local inference optimization and measurements on AMD GPUs, but it is not evidence showing performance or behavior on other hardware.