EvalDock (evaldock/evaldock) is an end-to-end (end-to-end) evaluation framework that runs different agents, and plugin configurations within the same agent, as a single unit of work and evaluates them. It is written in TypeScript, and the repository was created on September 30, 2026. There is no public release yet.
The problem this tool targets is that agent evaluation is fragmented across frameworks. EvalDock evaluates different agents and plugins as one set. It wraps each agent's different execution method in an execution adapter, while sharing the dataset, the trace format, and the evaluation judgment. End-to-end means that the process of tool calls and outputs after the task is handed over, along with the scores for those results, is handled in a single flow. Running the same task leaves tool calls and outputs, scores, and evidence together, so agents can be compared side by side, and so can plugin configurations inside an agent. Even when the execution methods differ, the structure that routes everything through a shared layer raises comparability. It selects agents, plugins, and tasks in the workbench and manages execution. The repository's GitHub star count is 47.
The limitations are written out as they are in the README. Small-sample estimation and the execution trace graph are separated out as plans for the next version, and are not yet enabled in the public build. The license is not stated either. The research record's scores are freshness 7.0, technical 5.0, completeness 6.0, and momentum 2.1, with only momentum low. The design that raises comparability through a shared adapter and a trace/evaluation layer is clearly sound, but the features missing from the public build narrow the implementation scope. Whether the sample estimation and the trace graph actually work has to be checked in the next release.