toloka/tolokaforge is a general-purpose LLM benchmarking harness that bundles tool use, browser, mobile, coding, and long-horizon task evaluations. The 2026-11-06 commit 3a13b4c touched this harness's Langfuse integration package, tolokaforge_langfuse. The data safety gate of this package is a mechanism that scans data going outward, such as traces or attached files, to find values that look like passwords and block their export. This change narrows the decision criteria so that the gate blocks fewer cases in which legitimate data was mistaken for a secret, while still blocking actual secrets.
The exemption rules were narrowed. Database URLs that point to a loopback address or a compose service name (db, postgres), whose password is the same as the user name and is a single character class of 12 characters or fewer, and GPG_KEY, a 40-digit hexadecimal fingerprint, are exempt. URLs with another @ after the host, URLs followed by host, hostaddr, or service parameters, and URLs exceeding 1KiB are not exceptions. dotenv lines are read regardless of the case of the name, but a value whose name is not uppercase is not treated as a secret, so lines such as `txn_key = compute(x)` do not get caught. The number of exemptions is tallied per rule, and when the run ends, a single warning line remains together with the rule name.
Alongside the default secrets: reject, secrets: mask was added. In this mode, only the secrets in the file are substituted and re-encoded, sha256 and bytes are computed based on the masked bytes, and masked files are also counted as attached. The trace metadata schema grew from 34 keys to 36 keys, and the bookkeeping group gained secrets and secrets_masked. The transcript scrub changed its order so that it masks the whole thing first and then truncates. This was because the 10 characters following the key prefix were not masked due to a format mismatch, so the remaining characters could go out as they were.
That said, this change is limited to the inspection right before the Langfuse connector exports data. Files that cannot be hidden even by masking, or that do not come back as YAML or JSON after being masked, are skipped just as in the case of reject. The evaluation harness as a whole and model performance were not touched, and the commit message acknowledges that credentials of forms unknown to the gate can still pass. The same message states that it comes from a follow-up change that includes tests bound to the masking policy and shared vector files. The problem of the inspection becoming quadratically slow on inputs where the same name continues at length was also fixed with a per-part limit, and the commit message records that before the fix, a 96KB KEY string took 21 seconds. At the present stage, this is a change that provides real help to teams that had been held back by false positives in transmission blocking.