New
ollama/ollama v0.40.5 — Local and cloud model runtime
ollama/ollama is a runtime server that runs open models locally or in the cloud and connects them to a range of tools.
CURATED OPEN SOURCE INTELLIGENCE
#llm-infra
New
surya-koritala/glyd — Lossless Compression of bf16 Weights and KV Cache
A look at the lossless GPU memory compression approach in surya-koritala/glyd and the measured results published in its README.
Release
ashishsinha1602/schemagate v1.6.0 — Adds permission restriction notice
ashishsinha1602/schemagate v1.6.0 is a release that tells the model about schemas hidden by permissions and requires the answer INSUFFICIENT for queries that lack grounding.
New
srkrz23/vramkit — two-track VRAM compression for weights and the KV cache
srkrz23/vramkit is a Python repository implementing a lossless weight codec and KV cache compression that applies different quantization axes to keys and values.
New
jevagentharness/jevagentharness — a model-routing proxy for coding agents
jevagentharness/jevagentharness is a proxy that classifies each step of a coding agent and assigns a model; it covers the README's Terminal-Bench 2.0 measurements and their limits.
New
perplexityai/space — a Rust-based agent sandbox
perplexityai/space is a Rust-based sandbox platform that isolates and runs agent and user code.
New
indras-mirror/xolotl — a large MoE inference engine for a single GPU
Xolotl, a C++ engine that extends Strata, serves several large models through an OpenAI/Anthropic-compatible server, and its README reports a decode speed of 26.8 tok/s for DeepSeek-V4.1-Flash 552B on an RTX 4090.
New
nandhakishorm/laya v0.4.9 — non-autoregressive decision engine
A look at v0.4.9 of nandhakishorm/laya, the scope of its engine that performs selection, scoring, and yes/no decisions, and the limitations it publicly states.
New
rtk-ai/rtk v1.0.1 — a CLI proxy for cutting token usage
rtk-ai/rtk is a Rust CLI proxy that compresses, routes, and filters development command output to reduce LLM token consumption.
Release
ggml-org/llama.cpp b11700 — Add Granite Switch decision converter
llama.cpp b11700 added decision conversion and support paths for Granite Switch family models.
New
unclecode/crawl4ai v0.9.5 — the crawler that turns the web into Markdown
unclecode/crawl4ai v0.9.5 is an open-source crawler/scraper that converts websites into Markdown for LLMs.
Release
ggml-org/llama.cpp b11683 — Vulkan Host Memory Import Fixes
llama.cpp nightly b11683 is a build that fixes the Vulkan backend's host memory import failure logging and AMD's buffer sharing path.
New
antirez/ds4 — a local inference engine for Metal, CUDA, ROCm
antirez/ds4 is a C-based local inference engine that runs a set of models on Metal, CUDA, and ROCm.
Release
yetone/magpie v0.1.1257 — Plugin overhaul and JS gateway
yetone/magpie v0.1.1257 is a feature release that overhauls the plugin functionality and adds Lody agent support.
New
magnitudedev/magnitude — a local inference engine tailored to the hardware
magnitudedev/magnitude is a Rust-based open-source engine that tunes kernels to the hardware and uses them for model inference.
Release
scaleapi/agentenv-framework v0.9.1331 — Per-run configuration overrides
scaleapi/agentenv-framework v0.9.1331 ships an example MCP server bundle that runs through a gateway, plus per-run configuration overrides.
Release
nousresearch/hermes-agent v0.22.2 — Fixes gateway startup issue
nousresearch/hermes-agent v0.22.2 is a release that replaces the faulty v0.22.1 build and fixes the gateway startup issue as well as retry behavior for local model conversations on laptops.
New
infiniflow/ragflow v1.0.0-rc1 — a RAG·agent combined engine
infiniflow/ragflow is an open-source engine that combines RAG and agent capabilities to provide a context layer for LLMs.
New
flashml-org/freetoken v0.1.3 — a MoE serving engine for desktops
An introduction to what flashml-org/freetoken offers for serving large MoE models on edge hardware, and to the limits of its performance information.
Release
ggml-org/llama.cpp b11630 — Daily Build Automated Release
ggml-org/llama.cpp b11630 is a daily build automated release whose changes are described only at the level of commit messages.
No matching stories.