The latest commit of souheil007/rag_full_version, a9aeab5 (2026-10-17), adds only a single design document rather than code. 100 lines were added to plans/plan9.md, and there are no deletions. The repository description states that this project is a document RAG that achieves response times under 2 seconds through IBM Docling AST parsing, Pixtral-based VLM table repair, dual-representation serialization, and hybrid RRF retrieval. This document is a draft of 'Plan 9' that redesigns the evaluation method and table repair procedure of that RAG.
The first axis of the plan is evaluation separation. In the current structure, spoken queries with an empty ground-truth document ID (questions that cannot be answered) are also included in the retrieval metrics and aggregated as 0 points, which can make the retrieval performance numbers look lower. This plan applies retrieval evaluation (Hit Rate@K, MRR, Precision, Recall) only to answerable queries, and separately computes abstention accuracy for spoken queries — counting whether they were answered with the intent of 'I don't know' — so that the scorecard is separated.
The second axis is VLM round-trip validation. A corrupted table is passed to Pixtral to fix its layout, and the result is fed back into TableSanityChecker to check whether the correspondence with the original text and the column calculations are correct; if it does not pass, it is rejected with repaired=False and method='vlm_validation_failed', leaving the original table in the AST. It also includes cell invariant validation that prevents header normalization from shifting values into data cells, as well as documentation cleanup that distinguishes the VLM conversion target of 5% (cost SLA) from detection accuracy (Recall/Precision).
That said, this commit contains only the plan. evaluator.py, table_repairer.py, tests, and documentation updates remain as checklist items only, and no actual implementation or measurement results are included. Whether the retrieval metric separation and VLM round-trip validation currently work, and how often table repair gets rejected, cannot be known from this commit alone. The license and the star count were also not given.