ggml-org/llama.cpp build b12144 has been released. The repository is a project that implements large language model inference in C/C++. This version changes the layout of the mixture-of-experts weight data in the CUDA path so that it is laid out in a single line. The intent is to let tile size optimization apply better to both integer computation and matrix multiplication. A fix for a 16-bit floating-point performance regression is included as well. The changes are stated in the b12144 release notes.
A mixture of experts refers to a model structure that uses only some of several expert components depending on the input. What this record covers is the weight data layout of that structure. The change that lays the data out in a single line focuses on making tile size optimization apply better to both integer computation and matrix multiplication. So the core of this release is not a model feature addition but a change in data handling in the CUDA path and a fix for the 16-bit floating-point performance regression. The repository description is C/C++-based large language model inference, and the detailed changes can be checked at https://github.com/ggml-org/llama.cpp/releases/tag/b12144.
The provided release information contains no performance figures from before and after the change. Neither the effect of the tile size optimization on actual processing speed nor the degree to which the 16-bit floating-point performance regression improved can be judged from this information alone. What can be confirmed is limited to the mixture-of-experts weight layout change in the CUDA path and the fix for the 16-bit floating-point performance regression. It is hard to interpret this as a release whose performance improvement has been confirmed.