New
ollama/ollama v0.40.5 — Local and cloud model runtime
ollama/ollama is a runtime server that runs open models locally or in the cloud and connects them to a range of tools.
CURATED OPEN SOURCE INTELLIGENCE
#inference
New
surya-koritala/glyd — Lossless Compression of bf16 Weights and KV Cache
A look at the lossless GPU memory compression approach in surya-koritala/glyd and the measured results published in its README.
New
fesens/opentpu — An Open-Source AI Accelerator Designed by an AI Agent
fesens/opentpu is an open-source AI accelerator designed by an AI agent, gathering everything from hardware to compiler into one repository and presenting run results on a Kintex-7 card.
New
srkrz23/vramkit — two-track VRAM compression for weights and the KV cache
srkrz23/vramkit is a Python repository implementing a lossless weight codec and KV cache compression that applies different quantization axes to keys and values.
New
l-forster/open-jet v0.5.0 — Offline Agent and CUDA Inference Engine
l-forster/open-jet provides an offline coding agent for local models and a CUDA inference engine for Qwen3.8-27B; the latest release is v0.5.0.
New
indras-mirror/xolotl — a large MoE inference engine for a single GPU
Xolotl, a C++ engine that extends Strata, serves several large models through an OpenAI/Anthropic-compatible server, and its README reports a decode speed of 26.8 tok/s for DeepSeek-V4.1-Flash 552B on an RTX 4090.
New
nandhakishorm/laya v0.4.9 — non-autoregressive decision engine
A look at v0.4.9 of nandhakishorm/laya, the scope of its engine that performs selection, scoring, and yes/no decisions, and the limitations it publicly states.
Release
niko1221/strata v0.1.45 — Qwen3.6 image input and concurrent requests
niko1221/strata v0.1.45 adds image input support for Qwen3.6-35B-A3B and handles up to 8 concurrent requests, with performance gains that vary depending on available VRAM.
New
alexsjones/llmfit v1.1.17 — Model recommendations by hardware
alexsjones/llmfit v1.1.17 is a Rust command-line tool that recommends language models suited to your hardware and shares measured benchmarks.
New
nvidia/nemoclaw — Coding Agent Stack for the OpenShell Sandbox
nvidia/nemoclaw is a reference stack that runs coding agents such as Hermes, LangChain Deep Agents, and OpenClaw inside the NVIDIA OpenShell sandbox with managed inference.
Release
ggml-org/llama.cpp b11700 — Add Granite Switch decision converter
llama.cpp b11700 added decision conversion and support paths for Granite Switch family models.
Release
ggml-org/llama.cpp b11683 — Vulkan Host Memory Import Fixes
llama.cpp nightly b11683 is a build that fixes the Vulkan backend's host memory import failure logging and AMD's buffer sharing path.
New
antirez/ds4 — a local inference engine for Metal, CUDA, ROCm
antirez/ds4 is a C-based local inference engine that runs a set of models on Metal, CUDA, and ROCm.
New
magnitudedev/magnitude — a local inference engine tailored to the hardware
magnitudedev/magnitude is a Rust-based open-source engine that tunes kernels to the hardware and uses them for model inference.
New
zihan-su/self_gradient_forcing_plus — SGF+ Video Generation
An SGF+ repository that separates context history and denoising parameters in autoregressive video generation, producing rollouts of up to 24 hours of video from a 5-second training window.
New
flashml-org/freetoken v0.1.3 — a MoE serving engine for desktops
An introduction to what flashml-org/freetoken offers for serving large MoE models on edge hardware, and to the limits of its performance information.
Release
Niko1221/Strata v0.1.44.1 — Legacy CPU SIGILL hotfix
Strata v0.1.44.1 fixes SIGILL errors on older CPUs, build problems, and the server's crash retry behavior.
Release
ggml-org/llama.cpp b11630 — Daily Build Automated Release
ggml-org/llama.cpp b11630 is a daily build automated release whose changes are described only at the level of commit messages.
No matching stories.