ggml-org/llama.cpp v0.7.0 is the release of an engine that runs large language model inference locally in C and C++. This version refactors the server-side chat session API around common_chat_session and adds a GPU cache for experts held in host memory. The default port changed from 8080 to 9931.
The chat session changes include feed and finish. The GPU cache targets experts residing in host memory, and the README explains that this feature makes decoding up to 2.2x faster. This release also adds LiliCorr speculative decoding along with K2 Horizon and EmbeddingGemma-2 support. In other words, it is a release in which the server API, the inference path and model support all change together. The project is MIT licensed and its primary language is C++. The ggml operations document and the lib llama public API are managed separately.
That said, the measurement conditions behind the up to 2.2x figure are not given in the provided information, so it cannot be read as performance that applies to every model or environment. If you use the server chat session API, the refactored interface is the core of the change. Judging this release is better placed on the actual scope of the API and inference feature changes than on the performance number.