github.devotd.
ReleaseInference & serving

ericlbuehler/mistral.rs v0.9.5 — Delimiter handling fix and support additions

In v0.9.5 of the LLM inference project ericlbuehler/mistral.rs, tool-call/inference delimiter handling and model plus CUDA support all changed.

Repositoryericlbuehler/mistral.rs ↗

Fast, flexible LLM inference

stars
★ 7.8K
Forks
738
Language
Rust
License
MIT
Latest release
v0.9.5
Read original source ↗View on GitHub ↗

ericlbuehler/mistral.rs is a project that presents itself as fast and flexible LLM inference. The v0.9.5 release fixed the conditions under which the delimiters that mark tool calls and inference are applied. The delimiters apply only when they are output as special tokens. This change limited delimiter handling to special-token output. The changes for this release, published in the repository, also include adding support for the Qwen3.8-Flash-Next model and support for CUDA 13.4.

The three changes span the output handled during inference and the range of supported execution platforms. The delimiter condition fix concerns when the tool-call and inference delimiters are applied. Qwen3.8-Flash-Next support added that model to the supported targets, and CUDA 13.4 support added that CUDA version. The core of v0.9.5 is therefore the adjustment of the delimiter application conditions plus the two added support items. The information provided contains no detailed implementation or usage procedure for any of these items.

The release information presents no behavioral examples from before and after the fix, nor any performance comparison. It is therefore not possible to judge here what difference the delimiter condition change makes in actual responses, or how far the new model and CUDA support extend. What can be confirmed in this release is the target of the change and the fact that support was added; effects beyond that scope are not backed by the given information.