Rapidly test and validate AI prompts, agents, and RAG systems across multiple models. Run vulnerability scans and performance comparisons with zero complex setup.
21,800 stars1,924 forksTypeScriptUpdated 6/2/2026100% free · open source
What it does
Helps developers systematically test and compare AI prompts, models, and AI systems before putting them into production
When to use it
•When building an AI product and want to validate prompt reliability
•Before deploying a RAG (retrieval-augmented generation) system to ensure quality
•When comparing performance across different AI models and configurations
Quick start
1Install via npm: npm install -g promptfoo
2Create a config file defining test cases and models
3Run tests with: promptfoo eval
4Review performance metrics and comparison report
Ready-to-paste prompt
promptfoo eval -c config.yaml
Topics
ci
ci-cd
cicd
evaluation
evaluation-framework
llm
llm-eval
llm-evaluation
llm-evaluation-framework
llmops
pentesting
prompt-engineering
prompt-testing
prompts
rag
red-teaming
testing
vulnerability-scanners
What's inside — free to inspect
No purchase needed
Read the entire source before you build — unlike paid marketplaces that hide it behind a buy button.