l-forster/open-jet is a Python repository that bundles an offline coding agent using local models with its own inference engine. The latest release is v0.5.0. The provided material contains no changelog for this version, so no additions or fixes can be confirmed. It runs Qwen3.8-27B with its own CUDA megakernel on NVIDIA RTX 30/40/50-series cards with 24GB or more of memory, and falls back to llama-server on other models or GPUs.
Instead of a general-purpose inference engine, this repository is structured to replace the llama.cpp server with CUDA kernels tuned for Qwen3.8-27B. In the developer's own RTX 3090 measurements cited in the README, decode speed was 1.53–1.93× that of llama.cpp, and prompt processing speed was reported as rising from 1050-1140 tok/s to 1530-1600 tok/s. The design also verifies 4–5 draft tokens per decode step with a single persistent kernel launch. The core license is AGPL-3.0-only, and commercial features carry separate conditions.
The performance figures are single-conversation results measured by the developer on an RTX 3090; they do not represent batch performance, and no independent verification is presented. Compared with llama.cpp, the KL divergence is 0.0009 and the top-token match rate is 99.08%, so the output is close but not identical. Anything outside the supported range switches to llama-server, so the performance claims should be read as limited to specific model and GPU conditions.