rockmatrix-ai/edgerock is a multimodal model research repository targeting the problem where, during long inference, the key-value cache grows, occupies accelerator memory, and limits concurrent serving. It publishes the EdgeRock-4B model along with inference and evaluation code, and currently has no registered releases. It is written in Python and uses the Apache-2.0 license.
EdgeRock-4B is based on Qwen3.5-4B, converting all 8 all-attention layers while keeping 24 recurrent layers and the feedforward layers. Its recurrent state summarizes context, sliding-window attention handles recent tokens, and Rock Memory Anchors store selected long-range information in a fixed-capacity bank for lookup. The repository cites 72 MiB of attention key-value per sequence and 121.76 MiB of persistent history. It also reports that at a 256K context BF16 attention key-value is 99.1% smaller than Qwen3.5-4B, and that the average AIME 2024–2026 pass rate is 82.9%. The mean across six visual reasoning benchmarks is 84.9%, versus 95.3% for the Qwen3.5-4B given for comparison. The runnable code lives in the runtime directory, and the documentation covers custom wheel installation and demo run procedures.
As the basis for these numbers, the repository points to a technical report and arXiv 2610.10118, but all the figures are the repository's own claims. It has 19 commits and 0 forks, with no tests and no evaluation harness for reproduction. So the published architecture and code can be examined, but the performance claims should be held in reserve until independently verified.