General Local Engines
Portable inference engines designed to run many model architectures and quantization formats on local hardware; it excludes forks dedicated to one GPU vendor and desktop model launchers.
| Project | What it does | Stars | Language | License | Latest release |
|---|---|---|---|---|---|
| ollama/ollama | A general-purpose inference engine that runs many open models locally or in the cloud. | ★ 183.7K | Go | MIT | v0.40.6 |
| ggml-org/llama.cpp | A portable C/C++ inference engine executes large language models locally. | ★ 131.6K | C++ | MIT | b11719 |
| niko1221/strata | A local engine for Qwen3.8-Flash-Next with one-click setup and OpenAI- or Anthropic-compatible localhost APIs. | ★ 27.3K | C++ | MIT | v0.1.45 |
| antirez/ds4 | Runs selected DeepSeek, GLM, and Qwen models locally across Metal, CUDA, and ROCm with SSD streaming. | ★ 23.9K | C | MIT | — |
| flashml-org/freetoken | An engine that serves massive MoE models locally across GPU, CPU, and host memory. | ★ 14.6K | Python | Apache-2.0 | v0.1.3 |
| magnitudedev/magnitude | A general inference engine that autotunes kernels to run local models across CPUs and multiple GPU vendors. | ★ 7K | Rust | Apache-2.0 | @magnitudedev/cli@0.2.9 |
| jitllm/jitllm | A Go inference engine that JIT-generates CPU/GPU kernels and runs models larger than memory across multiple backends. | ★ 48 | Go | Apache-2.0 | v0.2.6 |
| seuros/mdma | A complete local GGUF inference engine spanning CPU, ROCm, Vulkan, CUDA, Metal, Level Zero, and Edge TPU backends. | ★ 14 | Rust | BSD-2-Clause | — |