Devotd Github
NewInference & serving

raketenkater/ggrun — Catalog Refresh Guard and Index Conversion Correction

raketenkater/ggrun fixed the guard so that only actual leaderboard scores are counted when the catalog is refreshed, and stabilized index conversion through Theil-Sen fitting and version-notation determination.

Repositoryraketenkater/ggrun ↗

raketenkater/ggrun fixed the guard so that only actual leaderboard scores are counted when the catalog is refreshed, and stabilized index conversion through Theil-Sen fitting and version-notation determination.

GitHub description: llama.cpp/ik_llama.cpp launcher: loads big MoE models across mismatched multi-GPU rigs by exact VRAM math.

stars
★ 280
Forks
18
Language
Go
License
MIT
Latest release
v3.2.11
Read original source ↗View on GitHub ↗

raketenkater/ggrun is a launcher that runs llama.cpp and ik_llama.cpp, allowing large MoE models to be loaded via VRAM calculations even in environments where the GPU configuration does not match the model. The 2026-11-04 commit 0ce4909 is a merge of PR #118, in which the catalog refresh guard came to count only actual leaderboard scores rather than carried-over estimates, and the index conversion fit no longer sways on a single pair. In the CHANGELOG it remains only as an 'Unreleased' entry, and the latest dated release is v3.2.13 (2026-10-29).

What was touched is not the model placement but the model catalog refresh path. According to the 2026-11-02 review record in the test file, in v5 the carried-over '~' estimates filled in every row that had no score, so a scrape that had lost its actual scores still passed the occupancy check, and the top 24 actual score rows went missing. The guard now counts only actual score rows and compares them with the previous values. Alongside that, the least-squares fit in log space was replaced with a median-of-slopes approach (Theil-Sen fit), so that a single mismatched pair no longer causes the converted values in the low-score region to swing widely, while the R² threshold of 0.8 is kept as is. Also, if the exponent and the intercept are the same within 0.02, it is treated as meaning that not the scores but only the version notation on the page changed, and the previous version is kept. The change spans three files: tools/models/update_recommendations.py, tests/test_update_recommendations.py, and CHANGELOG.md.

The limitations are clear as well. The change stays confined to the catalog refresh script and the regression tests, and no figures on runtime speed or VRAM placement are given. A finalized version number has not been attached yet either, and this commit also fills in the previously missing v3.2.12 (2026-10-18) entry. The fact that the risk of the score mapping being off is reduced can be confirmed in the code and tests, but how well the catalog aligns with the actual leaderboard still needs to be watched. Source: https://github.com/raketenkater/ggrun/commit/0ce49098484073d53719f977d268a77b32bb61fe