gautam1858/banach is an inference tool that reads mixture-of-experts (MoE) models from local storage and runs them without loading them fully into main memory. It is written in Rust and was released under the Apache-2.0 license. The latest release is v0.1.3. The repository introduces the goal of running MoE models in the 300B–3T range on the user's own computer. However, the provided material does not say what changed in v0.1.3.
The core of the tool is that instead of loading the whole model into main memory, it reads the selected experts from storage and processes them within a memory budget. According to the README, the plan command computes the expected speed and memory allocation based on the model header. The repository also describes an accuracy check that compares tokens and logits against llama.cpp, along with continuous integration checks. The post presents an approach of attempting trillion-parameter models such as Kimi K2 on a 64GB desktop.
The performance evidence offered is the post's 22GB model comparison. The repository reports prompt processing 2.2 times faster than llama.cpp on that model, but that is a figure the project itself provided and it is limited to prompt processing. Performance on other models or under other conditions is not in the provided material. So it is better regarded as early evidence for an approach of running large MoE models on local storage, rather than a result confirming broad performance superiority.