Devotd Github
NewInference & serving

cedriceuh/freetoken-rdna3 173df60 — Re-measurement of all profiles

A FreeToken tuned build that runs Mixture-of-Experts models locally on AMD Radeon RX 7900 series GPUs re-measured every profile and checkpoint and updated the README speed table.

Repositorycedriceuh/freetoken-rdna3 ↗

A FreeToken tuned build that runs Mixture-of-Experts models locally on AMD Radeon RX 7900 series GPUs re-measured every profile and checkpoint and updated the README speed table.

GitHub description: Fast local LLM inference on AMD Radeon RX 7900 XTX / XT (RDNA3, ROCm): big Mixture-of-Experts models like Qwen3.8-Flash-Next on one or two consumer GPUs. Up to 4 parallel agents with 262k context each, or one 1M-token context, OpenAI/Anthropic API, NVFP4 and EXL3 checkpoints. A tuned build of FreeToken.

stars
★ 3
Forks
0
Language
Python
License
Apache-2.0
Read original source ↗View on GitHub ↗

cedriceuh/freetoken-rdna3 is a tuned build of FreeToken that runs Mixture-of-Experts (MoE) models such as Qwen3.8-Flash-Next locally on one or two consumer AMD Radeon RX 7900 XTX/XT (RDNA3) cards. Commit 173df60 of November 6, 2026 re-ran the measurements for all profiles and checkpoints (NVFP4, EXL3 3.05/4.05 bpw) with the tree merged up to e5856ac, and swapped out the README speed table and banner. Fifteen files were changed, with 396 lines changed (215 insertions, 181 deletions), and these are not source code but documentation and profile settings (rdna3/profiles/*.env).

The biggest change is decode (TG, generated tokens per second). In an RX 7900 XTX+XT paired configuration, NVFP4 went from 92-115 tok/s to 107-133 tok/s, EXL3 3.05 bpw went from 117-145 tok/s to 134-168 tok/s, and EXL3 4.05 bpw went from 105-112 tok/s to 118-128 tok/s. XTX alone went from 38-47 to 45-54 tok/s for NVFP4, and XT alone went from 30-36 to 36-42 tok/s. The processing throughput for reading new blocks (PP) and the first-token latency of agent turns are unchanged according to the commit message, but in serve.sh's sampling settings a single request ran at 122.1 tok/s, higher than the previously recorded 106.2 tok/s.

These figures are values the project re-measured in its own environment. According to the commit message, for each row of the table a server was launched one at a time via rdna3/serve.sh with the profile as-is, a first run was performed once to fill the kernel cache, and then the measurement was taken with greedy decoding on image 640e1e5. There were no GPU errors across the 19 servers, but one server with the MTP (Multi-Token Prediction) head — the auxiliary device that predicts multiple tokens at once — disabled decoded slowly for the first 4 minutes, and it did not reproduce on re-runs. The banner's overall track changing from 150 to 175 tok/s is an adjustment of the graph's axis scale, not a separate measurement.

So this commit is closer to a verification record than to improved code. The numbers are worth referencing for anyone wanting to run MoE locally on AMD consumer cards, but no license and no external validation are presented, and some latency figures remain unexplained. Rather than taking them at face value, it is left to each person to confirm them in their own card configuration.