github.devotd.
ReleaseInference & serving

niko1221/strata v0.1.45 — Qwen3.6 image input and concurrent requests

niko1221/strata v0.1.45 adds image input support for Qwen3.6-35B-A3B and handles up to 8 concurrent requests, with performance gains that vary depending on available VRAM.

Repositoryniko1221/strata ↗

Qwen3.8-Flash-Next on any consumer hardware: one-click install for Windows / Linux. Strata inference engine, OpenAI/Anthropic API on localhost, optional image input.

stars
★ 26.7K
Forks
2.8K
Language
C++
License
MIT
Latest release
v0.1.45
Read original source ↗View on GitHub ↗

niko1221/strata v0.1.45 is the release that adds image input for Qwen3.6-35B-A3B. Strata is an inference engine that runs models locally, and this version processes up to 8 requests concurrently. Image input requires the --vision option and a 0.9GB mmproj file.

The repository description states that Qwen3.8-Flash-Next runs on consumer hardware and that one-click installers are provided for Windows and Linux. The OpenAI and Anthropic APIs are served on localhost, and image input is an optional feature. The repository has recorded 26454 stars. In the release notes' R9700 32GB baseline comparison, concurrent request performance is 1.14~1.61x that of llama.cpp. Decoding is recorded as 1.42x faster, and 4K prompt processing as 1.80x faster. Each figure is expressed as a multiple relative to llama.cpp.

However, unless all experts fit in VRAM, the slots erode the expert cache. In that case the performance gain drops to 1.0~1.15x, and single requests become 4~11% slower. The release notes state that Qwen3.8-Flash-Next responses are byte-for-byte identical to v0.1.44.1. Therefore the performance numbers should be read as limited to the concurrent request conditions of the R9700 32GB, and it is hard to expect any gain when VRAM headroom is insufficient or when using single requests.