promptfoo/promptfoo is a TypeScript tool that automatically evaluates and red teams the prompts and models of LLM applications. It ships as both a CLI and a library. Its scope extends beyond prompts to agents and retrieval-augmented generation (RAG, a method that searches for supporting evidence and attaches it to the answer). Red teaming is a test that pushes a model into producing harmful output in order to find vulnerabilities. Evaluation cases are written in a declarative config file and run from the command line or in CI/CD. The README states that LLM evaluation runs locally. The latest public release is code-scan-action-0.2.2.
A declarative config compares the performance of several models such as GPT, Claude, Gemini, and DeepSeek, and runs vulnerability scanning, red teaming, and penetration testing. It can be automated as a CI/CD check, and the results can be shared. The repository description states that OpenAI and Anthropic use the tool. The repository was created on April 28, 2023 and currently has 26203 stars, with TypeScript as its language and an MIT license.
Still, the public record alone is not enough to judge how far the project has come. The evaluation record keeps only a momentum score of 6.6, and no figures are given for the freshness, technical, or quality scores. The changes in code-scan-action-0.2.2 are not listed in the source material either. The OpenAI and Anthropic use cases are claims made only in the repository description. Benchmark figures and standardized evaluation metrics are absent from the public material as well. So the place to try it first looks less like a production judgment tool and more like CI: attach it to the pipeline, change prompts and models, and watch whether regressions appear.