Devotd Github
NewInference & serving

thefahien/elpis-on-7900xtx — speculative decoding setup for RX 7900 XTX

A repository with the setup and measurements for running Qwen3.8-27B EXL3 and DFlash2 together on an AMD RX 7900 XTX.

Repositorythefahien/elpis-on-7900xtx ↗

A repository with the setup and measurements for running Qwen3.8-27B EXL3 and DFlash2 together on an AMD RX 7900 XTX.

GitHub description: Qwen3.8-27B EXL3 + DFlash2 speculative decoding on an AMD RX 7900 XTX (exllamav3-rocm): recipe, patches, benchmarks

stars
★ 8
Forks
0
Language
Python
License
MIT
Read original source ↗View on GitHub ↗

thefahien/elpis-on-7900xtx provides a setup for running Qwen3.8-27B EXL3 together with the DFlash2 draft model on an AMD RX 7900 XTX. It is not a new release; it covers the installation steps and run configuration written in the repository's README. The target audience is anyone configuring this model combination on an AMD graphics card. The repository description states that it covers recipes using exllamav3-rocm, patches, and benchmarks.

Speculative decoding is a method that uses a draft model alongside a large model to help with that model's text generation. The README includes not only the patches and installation steps needed to run it, but also benchmark scripts and raw result files. The document reports higher code-task throughput than llama.cpp on the same RX 7900 XTX. So you can follow the setup while reviewing the measurements as well. Note, however, that this speed comparison is a result reported by the repository's own documentation.

The llama.cpp side used for comparison is a different, smaller model, and no output-quality comparison was performed. The README also summarizes long context, temperature, and memory limits as constraints. Therefore the reported speed cannot be read as a performance advantage between equivalent models, or as evidence of result quality. This repository is best treated as a reference for how the model combination was configured on an RX 7900 XTX and for its limited comparison results.