Devotd Github
NewInference & serving

carloslfu/slotstream — Q2 verification record and rollback of the direct-read candidate

The October 25, 2026 commit in carloslfu/slotstream recorded Q2 verification measurements and rolled back the direct forecast-read candidate.

Repositorycarloslfu/slotstream ↗

The October 25, 2026 commit in carloslfu/slotstream recorded Q2 verification measurements and rolled back the direct forecast-read candidate.

GitHub description: Run a 105 GB AI model on a Mac that can't hold it. Slotstream streams Qwen3.8-Flash-Next (125B mixture of experts) from your SSD and caches the busiest experts in memory, so it runs on Macs with 16 to 64 GB. One native Swift binary on MLX and Metal, no Python, offline. Works with Claude Code, Codex and Ollama or OpenAI clients.

stars
★ 474
Forks
37
Language
Swift
License
MIT
Latest release
v0.2.29
Read original source ↗View on GitHub ↗

carloslfu/slotstream is a project that streams the Qwen3.8-Flash-Next (125B mixture-of-experts model) from SSD so that it runs on Macs with 16–64GB of memory. The commit of October 25, 2026, 7578fb21e57ec362d19f5050922e0f583914935d, added the Q2 verification measurement record and rolled back the direct forecast-read candidate. The changes are contained in CLAUDE.md and MEASUREMENTS.md, along with the measurement and execution records under db/. The source is that GitHub commit.

The measurement record examined the interaction between read-ahead and GPU execution in ordinary code and prose requests. It compared partial-miss situations by invalidating the same logical expert in the native engine and in the Splash engine, and all 8 conditions passed the clean-execution criteria. In the comparison with per-read tracing enabled, intervals of about 14ms were observed, but the cost of trace logging differed from engine to engine. In the control group with tracing disabled on both sides, the interval dropped to about 8.4ms. Read-ahead also helped native execution, but the cause when it overlaps with GPU work was not confirmed.

The direct-read candidate was a small change that made Q2 forecast data be read at the requested position with no intermediate copy. It preserved correctness but slowed down full requests, and since the recorded diagnostics showed no benefit, it was rolled back. The measurement could not attribute the latency to any one of read-ahead transfer, command delivery, or concurrent I/O. Therefore a rewrite of the shared read scheduler is not a confirmed solution, and this record alone cannot be used to claim performance parity or broad task quality.