Devotd Github
NewInference & serving

mostlygeek/llama-swap v267 — Local Model Server Switching Tool

mostlygeek/llama-swap is a Go tool that connects OpenAI- and Anthropic-compatible local model servers and switches models according to the request.

Repositorymostlygeek/llama-swap ↗

mostlygeek/llama-swap is a Go tool that connects OpenAI- and Anthropic-compatible local model servers and switches models according to the request.

GitHub description: Reliable model swapping for any local OpenAI/Anthropic compatible server - llama.cpp, vllm, etc

stars
★ 6.1K
Forks
523
Language
Go
License
MIT
Latest release
v267
Read original source ↗View on GitHub ↗

mostlygeek/llama-swap is a server relay tool that runs multiple generative AI models locally and switches models according to the request. The latest release is v267. The provided information does not list what changed in this version, so no specific feature changes can be presented as release content. It is written in Go and released under the MIT license.

The tool supports OpenAI- and Anthropic-API-compatible servers, and is described as also connecting to servers such as stable-diffusion.cpp, audio.cpp, and ComfyUI, in addition to llama.cpp and vLLM. According to the README, a single configuration file defines model switching and which models run concurrently, and unused models can be unloaded automatically or preloaded at startup. It also provides a web interface. Instead of handling each model server separately, the structure is to select the needed model through a relay tool. It is also described as consisting of a single executable and a single configuration file, with no external dependencies.

So the key confirmed point is that it helps switch among several kinds of local model servers within one tool. However, the provided information alone is not enough to judge the specific changes in v267 or the stability of each server combination. Rather than asserting this version's improvements based on the release number alone, it is more accurate to limit the view to the feature scope that can be confirmed.