Devotd Github
NewInference & serving

lemonade-sdk/lemonade — MTP automation, registration of the Qwen3.5-35B-A3B-HRX model

The latest commit in lemonade-sdk/lemonade added MTP inference automation to the HRX backend and registered the Qwen3.5-35B-A3B-HRX model in the catalog.

Repositorylemonade-sdk/lemonade ↗

The latest commit in lemonade-sdk/lemonade added MTP inference automation to the HRX backend and registered the Qwen3.5-35B-A3B-HRX model in the catalog.

GitHub description: Lemonade helps users discover and run local AI apps by serving optimized LLMs right from their own GPUs and NPUs. Join our discord: https://discord.gg/5xXzkMu8Zk

stars
★ 5.9K
Forks
529
Language
C++
License
Apache-2.0
Latest release
v2026.42.1
Read original source ↗View on GitHub ↗

The latest commit a655222 (November 7, 2026, PR #4014) in the lemonade-sdk/lemonade repository added MTP automation to the HRX backend, which serves LLMs optimized for local GPUs and NPUs. At the same time, Qwen3.5-35B-A3B-HRX was registered as a new entry in the server model catalog. The commit touched 5 files, adding 103 lines and deleting 7 lines. The original can be seen at https://github.com/lemonade-sdk/lemonade/commit/a6552224573ffa128bf8ba49e238c6c604d18870.

HRX is a narrow backend that reuses the llama-server protocol but supports only the local GGUF chat launch path. The center of this change is automatically appending --spec-type draft-mtp when a model whose catalog label includes mtp is launched on HRX. MTP (Multi-Token Prediction) is a method where the model predicts multiple tokens at once; as stated in the header comment, HRX supports a built-in MTP head and does not handle a separate draft checkpoint. The entry added to the catalog takes Qwen3.5-35B-A3B-UD-Q4_K_XL.gguf from unsloth/Qwen3.5-35B-A3B-MTP-GGUF as its checkpoint, and carries the labels chat, tool-calling, mtp and the size value 23.1. On the deployment side, the HRX runtime goes up from hrx-b99 to hrx-b129, and the sha256 of the new binary is pinned.

Automation is not always advantageous. If merge_args is left false, no automatic arguments are appended, and if the user writes --spec-type themselves, the automatic setting is not overwritten. The contract tests now include checks that the catalog entry parses and that arguments are not duplicated on repeated calls, but the commit message states that a separate heavy MTP inference test was taken out of CI. So this commit only shows that the MTP path was attached, and it does not present any figures on how much faster the inference becomes. Neither the license nor performance measurements are presented alongside it, so it is a stage that needs to be checked before real-world use.