modelscope/evalscope released v1.14.0. This version, published as a GitHub release on October 20, 2026, adds resources for decision-making, code, table, finance, spatial, and multimodal evaluation, and expands the evaluation metrics and dataset loading (the process of reading in evaluation resources). It also fixes a number of scoring, caching, concurrency, and web task errors.
evalscope is a framework that handles the evaluation and performance benchmarking of large language models (LLMs), vision-language models (VLMs), and AI-generated content (AIGC) in a single tool. Performance benchmarking means comparing models under the same conditions to measure performance, and this repository collects that procedure in one place. The repository description summarizes the tool concisely while defining it as something users can use by swapping the metrics and datasets. Its structure handles the resources and metrics used for evaluation as user-configurable settings, and this version widened that scope. The repository has 3571 GitHub stars.
However, the given release information contains no evaluation scores obtained with the newly added resources and no model-by-model comparison results. There are no performance improvement figures either. Which resources went in under which names must be checked in the release notes (https://github.com/modelscope/evalscope/releases/tag/v1.14.0). Since error fixes came along with it, this version reads as a release that pairs the addition of resources with fixes to execution-side errors. With no score comparisons, the superiority between models cannot be determined from this information alone.