jesdga95/basalt

A general-purpose inference engine that runs many open models locally or in the cloud.

A portable C/C++ inference engine executes large language models locally.

A local engine for Qwen3.8-Flash-Next with one-click setup and OpenAI- or Anthropic-compatible localhost APIs.

Runs selected DeepSeek, GLM, and Qwen models locally across Metal, CUDA, and ROCm with SSD streaming.

OMLX is a local Apple Silicon LLM server with continuous batching, SSD and KV caching, and menu-bar management.

An engine that serves massive MoE models locally across GPU, CPU, and host memory.