ai-safety-institute/transect is a transcript analysis tool for agent evaluation built on top of inspect-scout. It requires Python 3.12 or later. The November 5, 2026 commit 2655747 adds a run index that bundles and displays the results of processing a single sample across multiple epochs (scan rounds). The version in pyproject.toml was bumped from 0.6.1 to 0.7.0, and the diff is on the order of 854 added lines and 57 deleted lines.
render() now produces a report file per epoch, such as report_epoch2.html, and additionally writes report_index.html alongside them. The index goes into the first position of report_paths and is opened by default. It contains each epoch's report and that epoch's run status, the reliability of the judgment surface aggregated across multiple epochs, and the scan status of the whole run.
The aggregate reliability is computed by the newly added src/transect/report/collection.py and rendered by the index.html.j2, runs.html.j2, and collection_reliability.html.j2 templates. The table splits its rows by recorded judge configuration and label vocabulary, and shows together the number of valid votes, the sample count expected by the configuration, the frame's mean judge_agreement, pairwise agreement, α and AC1 computed from the original votes, the number of judge re-labels, and the number of reversals found in random sample checks. The index is outside section_order, so it does not read transcripts and does not require a chart runtime. If there is only one epoch, no index is created.
However, these values combine judge audits from multiple epochs, and the documentation explicitly states that they measure consistency rather than accuracy. Because the table's rows are split by judge configuration and label vocabulary, comparisons should be read within the same configuration. They are closer to observing trends across repeated runs under the same configuration than to a single run.