sierra-research/tau2-bench has released its latest version v2.0.0. τ-bench is a benchmark that jointly evaluates agent–user conversations and tool use in realistic domains. v2.0.0 provides the tau3 domain, voice evaluation and policy-compliance tasks as simulators and evaluators.
Evaluation splits into two branches. The text-based half-duplex mode reports turn-by-turn conversations and tool use, where one side finishes before the next turn continues, while the voice full-duplex mode evaluates the entire conversation through a real-time audio API that allows both sides to speak at the same time. The README describes this project as a simulation framework for evaluating customer service agents in realistic domains. Each domain defines a policy, the tools available to the agent, evaluation tasks and optional user tools, testing the ability to select tools while respecting the domain rules. The provided domains are mock, airline, retail, telecom and banking_knowledge. The implementation language is Python, the license is MIT, and the GitHub star count is 2270.
However, the material reaches only as far as feature descriptions and repository metrics. No per-agent evaluation results or performance figures are given, and the only numeric figure is the repository momentum score of 3.9. Voice evaluation presupposes a real-time audio API, and the provided domains also target the customer service area. It is worth referencing as a baseline for comparing customer support agents, but this material alone is not enough to judge which model sits at which level.