Devotd Github
NewRAG & retrieval

fengzhenfei/carrel v0.4.1 — Search and Codebase Evaluation Report Added

fengzhenfei/carrel has moved up to 0.4.1 and published its own evaluation report covering text retrieval benchmarks and 24 code queries.

Repositoryfengzhenfei/carrel ↗

fengzhenfei/carrel has moved up to 0.4.1 and published its own evaluation report covering text retrieval benchmarks and 24 code queries.

GitHub description: Ontology-Augmented Generation for AI agents.

stars
★ 121
Forks
3
Language
Python
License
MIT
Latest release
v0.2.0
Read original source ↗View on GitHub ↗

fengzhenfei/carrel is a self-hosted knowledge base that uses ontology-augmented generation for AI agents. The November 4, 2026 commit 23ef852 raises the package version from 0.4.0 to 0.4.1 and adds to the repository the 0.4.1 change and upgrade guide together with a newly written search and codebase evaluation report.

The axis of this release is not added functionality but the publication of evaluation results. The README states that 0.4.1 recorded, on 275 text queries, a grounded Hit@5 (the proportion of cases in which the correct evidence is contained in the top 5 search results) of 93.45% and MRR@10 (the average of the rank at which the correct evidence is placed among the search results) of 0.8141. Placed side by side with the 92.36% and 0.7979 that 0.4.0 recorded on the same queries, this reads as a slight improvement. The evaluation document writes that on 38 cross-document queries a single response covered an average of 76.75% of the evidence groups, and states that this is higher than 0.4.0's 68.73%. The text retrieval replay returned 10 direct hits using Qwen3-Embedding-0.6B and Qwen3-Reranker-4B, and records that it used 454 evidence groups and 5 knowledge bases.

The measurement for code was carried out separately. The README summarizes that, across 24 code queries, Haiku answered 13 of them completely with the first response alone, and answered 23 when Skill (the follow-up query capability in which the agent narrows the search and carries the instructions onward) is allowed. In 0.4.1, Skill displays the graph source passage separately, with the same reference as direct hits. When comparing two versions of a document clause, you can read both in the first response or re-fetch whichever side is missing, and follow-up queries carry over the time range and the source and graph versions.

That said, these numbers are a self-evaluation performed directly by the repository. The 24 code questions have a small sample, and the text retrieval figures were reproduced within a configuration of 275 queries, 454 evidence groups, and 5 knowledge bases; 16 page images and 55 document questions were also included in the evaluation. The scores measured with the search profile KB_SEARCH_TEXT_DISCOVERY=true turned on should also be checked. Items such as graph reconstruction stability or ontology governance still point to the full technical report of 0.4.0. It is still too early to conclude that retrieval quality has noticeably improved in 0.4.1.