cosmonomic/pulsarlm is an LLM inference engine for persistent chatbots and agents. It removes old, low-importance tokens from the context to cap the size and inference cost of long conversations. No new release version was given.
Its custom attention kernel accumulates attention mass per token page and periodically removes pages with low mass. The system prompt and recent pages are kept. Because deleting past tokens can break the prefix cache that most engines rely on, the architecture periodically writes context snapshots to external storage and resumes conversations from the latest snapshot. Page state is held in a host-memory LRU cache with a 4 GiB default and is not persisted.
The repository reports bf16 on an RTX 5090: 15.8k tok/s prefill and 184 tok/s decode for a single request with Gemma 4 E2B under default settings. Decode with 16 concurrent requests is 2.4k tok/s, with prefill measured on a 5k-token prompt and decode measured for one 512-token request and sixteen 256-token requests. Separate Qwen3 8B figures are given as 6.7k tok/s prefill and 89 tok/s decode. Models implemented are few, and testing is limited to x86_64 and the RTX 5090. Other GPUs and aarch64 wheels are unverified, so the performance and compatibility evidence is limited.