jundot/omlx is an LLM inference server for Apple Silicon. The latest release is v0.7.1rc1, and the provided repository history contains no change list for this version, so what changed in this release cannot be confirmed. The repository README presents continuous batching and a two-tier memory/SSD KV cache as its core features, and describes management through a macOS menu bar app.
The project provides the inference server needed to run models locally together with operational features. According to the README, the KV cache retains the previous context even when the context changes during a conversation, so it is reused across requests. It also proposes keeping frequently used everyday models in memory and automatically swapping in heavy models when a request arrives. The implementation language is Python and the license is Apache-2.0. The README badge states Python 3.11~3.13, and the install path points to the release .dmg and Homebrew.
There are no performance figures in the README body, which links out to the external site oMLX.ai. Therefore the provided material alone cannot be used to judge throughput or latency, or the performance benefit of the cache. The latest release is an RC and no change list has been confirmed, so the specific changes in v0.7.1rc1 also fall beyond the scope of the material.