Devotd Github
NewInference & serving

truespar/paddock d8ee0629 — NVIDIA GPU support for running Flint 40B

truespar/paddock added support for running Recursive's coding-agent model Flint (40B, 2.5B active) on NVIDIA Ada-or-newer GPUs via a commit dated November 5, 2026.

Repositorytruespar/paddock ↗

truespar/paddock added support for running Recursive's coding-agent model Flint (40B, 2.5B active) on NVIDIA Ada-or-newer GPUs via a commit dated November 5, 2026.

GitHub description: Native Rust inference server for open models on NVIDIA and Apple Silicon. OpenAI- and Anthropic-compatible APIs, GGUF + safetensors + MLX, FP8/NVFP4/MXFP4/Q8/Q4, built-in Studio

stars
★ 135
Forks
19
Language
Rust
License
NOASSERTION
Latest release
v0.1.18
Read original source ↗View on GitHub ↗

truespar/paddock is a native Rust inference server for open models. A commit dated November 5, 2026, d8ee0629 (message "Flint on NVIDIA GPUs"), made it possible to run Recursive's Flint model on this server. Flint was added to the list of supported models in the README, where it is described as text-only, with 40B scale and 2.5B active parameters, intended for coding agents, and stated to run only on NVIDIA GPUs of Ada or newer.

The center of this change is the GPU execution path. 6,845 lines were added and 648 lines deleted across 30 files, and the new file gpu_model/laguna/load_flint.rs (410 lines) contains a Flint-specific model loader. gpu/nvpf.rs, gpu/w16r.rs, gpu_model/laguna/nv.rs, and gpu_model/laguna/attend.rs were newly added, and attention.rs gained a bf16-output prefill attention path. There is a PfQ type that chooses the q plane from either f32 or prescaled f16, along with a fallback whereby bf16 results are produced only in forms that use an f16 pool with head dimension 128, while other forms keep the existing f32 calls. batch_ops.rs gained qk_norm_rope_batch_at, which performs qk normalization and RoPE per row block, and models.toml grew by 91 lines and tests/gpu_flint.rs by 1,113 lines.

Still, the scope of support is clear. According to the README, Flint is text-only and presupposes NVIDIA GPUs of Ada or newer, so the Apple Silicon path and image input are not covered by this commit. The newly added bf16 prefill path is also conditional, falling back to the f32 path outside supported forms. Performance figures and benchmark results are not included in this commit, and no license was given either. It is premature to judge what level of speed and accuracy will come out, so it appears worth expecting only if you have an Ada-or-newer GPU.