Devotd Github
NewInference & serving

llamastash/llamastash v0.10.0 — a local LLM tool that bundles multiple engines

llamastash/llamastash is a Rust-based terminal tool that handles several local language model execution engines through a common interface, and the repository's latest release is v0.10.0.

Repositoryllamastash/llamastash ↗

llamastash/llamastash is a Rust-based terminal tool that handles several local language model execution engines through a common interface, and the repository's latest release is v0.10.0.

GitHub description: Zero-overhead, terminal-native local-LLM orchestrator. Run any engine, any model via the same proxy.

stars
★ 253
Forks
20
Language
Rust
License
MIT
Latest release
v0.10.0
Read original source ↗View on GitHub ↗

llamastash/llamastash is a Rust terminal tool that handles local language model execution engines such as llama.cpp, vLLM, and SGLang in a single program. The repository's latest release is v0.10.0. The provided material lists no changes in this version, so the specific modifications cannot be confirmed. The repository description introduces the tool as one that runs multiple engines and models behind the same proxy.

The core of this project is bundling the different usage patterns of each execution engine behind a common interface. It packs a screen-based interface, a command line, a background service, and an OpenAI-specification proxy server into a single program. Besides llama.cpp, vLLM, and SGLang, it also serves configured compatible servers on the same address and through the same interface. The initial setup tool prepares the runner and the startup model and tests the configuration. It also includes an agent skill feature, and the license is MIT. It therefore focuses on local runtime environments that want to keep the same approach even when switching engines.

The repository says it provides installation and usage guides, per-engine explanations, and testing and performance measurement data. However, the momentum score given in the material is 1.5, lower than novelty 8.0, tech 7.0, and quality 8.0. This shows that even though the setup handling multiple engines in one place is notable, the grounds for asserting that the project has strong momentum are limited.