ArtificialAnalysis/aa-agentperf-local v0.3.4 was released on October 15, 2026. The repository replays real agent task trajectories to measure local LLM serving performance. This release adds recipes for NVIDIA DGX and RTX Spark, and removes Spark's NPU from the list of detected accelerators. Shared GPU memory on Windows is now counted as well.
The model recipes add LFM2.5 24B A2B for llama.cpp and Qwen3.8 27B Q4 for DGX Spark. Recipes pinned to Splash 1.2.1 are included too. There is also a change that converts llama.cpp draft models at deploy time when they are only provided in the safetensors format. It is an update that adds hardware- and model-specific execution conditions to a measurement method that replays real agent task trajectories.
The new recipes confirmed in this change cover DGX and RTX Spark with the listed model combinations. The given release information does not state whether the same recipes apply to other hardware or models. So this update is better read as a change that widens the execution recipes available in specific environments and the scope of memory accounting, rather than as proof of improved performance across local LLM benchmarking in general.