Devotd Github
NewLLM infrastructure

pushkinist/rmlx 4c6e22eb — Script cleanup and mlx-c 0.8 support

pushkinist/rmlx deleted unused development scripts, merged scripts of similar families into a single tool, and additionally added mlx-c 0.8 build support.

Repositorypushkinist/rmlx ↗

pushkinist/rmlx deleted unused development scripts, merged scripts of similar families into a single tool, and additionally added mlx-c 0.8 build support.

GitHub description: Native Rust, single-binary MLX inference server for Apple Silicon. OpenAI/Anthropic-compatible LLM serving — text, vision, audio, embeddings — with the widest weight×KV quantization matrix of any MLX server. No Python, no GGUF.

stars
★ 16
Forks
5
Language
Rust
License
Apache-2.0
Latest release
v0.4.3
Read original source ↗View on GitHub ↗

The latest commit of pushkinist/rmlx, 4c6e22eb (2026-11-07), is the result of an audit of development scripts. Among the scripts for the MLX inference server for Apple Silicon, files that run nothing were deleted, and scripts of similar families were merged into a single tool with modes or flags. The rules are written in scripts/README.md, and check-scripts in make ci checks them.

Bash files decreased from 115 to 80, and Python files decreased from 48 to 53. The combined size of the tools, modules, and lib shrank from 128 files and 39,730 lines to 85 files and 34,415 lines. Conversely, the verification side grew. The recall tests grew from 22,233 lines to 32,349 lines, and the newly added scripts-rules gate alone is 3,316 lines. Fixtures went from 203 to 959, four gates received recall tests for the first time, and roughly 150 cases were added to the GPU test gate.

The way the tools are used also changed. The binary path is unified into the single --binary flag, and the RMLX_BIN, RMLX_BINARY, and RMLX environment variables are no longer used. Relative paths or an empty RMLX_HOME are rejected by the shell tools, the Makefile recipes, and wikitext2_ppl.py. The schema canary proof moved to proofs/schema-constraint under RMLX_HOME, and the Wikitext-2 cache moved to cache/wikitext-2. The smoke checks became stricter, and not a single one was dropped. serve_bench.sh, ssd_canary.sh, and wikitext2_ppl.py end with 75 if nothing is recorded.

Meanwhile, mlx-c 0.8 support is included in the same commit. rmlx-mlx builds against the mlx-c 0.8 C API as well as Homebrew's latest mlx-c, 0.6, and 0.7; because 0.8 added a nullable global_scale before sorted_indices in mlx_gather_qmm, build_support.rs distinguishes the C API by the number of function arguments and a marker export, and stops the build if the shape is different. At runtime too, the gather_qmm and SDPA wrappers reject C API mismatches. The pins remain mlx 0.32.3 and mlx-c 0.7.0, and 0.8 has only been verified as far as a compile-only clippy against the 0.7 combination in make ci and the 0.8.0 headers, and a macos-26 hosted build. There is no record of running GPU tests or models on 0.8. The hosted build job also changed to macos-26, since GitHub no longer offers macos-14. The tidied rules keep running in CI, so they are safe for repeated-execution environments, but the 0.8 support stays at the compile and build level, and the actual GPU path behavior has not been confirmed while the 0.7 pin is in use.