workflowstacks
devtools

Test AI with promptfoo

Founders get AI vulnerability scanning with promptfoo, used by OpenAI and Anthropic, with 22k+ GitHub stars

From the repo · “Test your prompts, agents, and RAGs. Red teaming/pentesting/vulnerability scanning for AI. Compare performance of GPT, Claude, Gemini, DeepSeek, and more. Simple declarative configs with command line and CI/CD integration. Used by OpenAI and Anthropic.”

intermediate⏱ 30 minutes💵 Free (self-hosted)
26,331 stars+287★ this week2,559 forksTypeScriptGuide quality 9/10 · reviewed Jun 5Updated 10/23/2026100% free · open source
What it is

Scan AI systems for security vulnerabilities

Real use case example

"A founder, Alex, wants to validate the security of their AI-powered customer service chatbot. Alex imports their chatbot model into promptfoo, creates a simple configuration, and runs a vulnerability scan. The result is a detailed report highlighting potential security issues for Alex to address."

When to use it
  • •When building an AI product and want to validate prompt reliability
  • •Before deploying a RAG (retrieval-augmented generation) system to ensure quality
  • •When comparing performance across different AI models and configurations
Ready-to-paste prompt
promptfoo eval -c config.yaml
Saves to your device
Use with Claude
New

Skip the builder — one click puts this in Claude, Cursor, Antigravity and more.

⚡ Runs out of the box

A prompt/skill package — nothing to install beyond adding it to your AI tool.

Try it instantly — no install
Claude Code
mkdir -p ~/.claude/skills/promptfoo && curl -fsSL https://workflowstacks.com/api/skills/promptfoo/claude-skill -o ~/.claude/skills/promptfoo/SKILL.md
Open in another AI app

Opens the app with this repo with the prompt ready to go — no copy-paste needed.

Connect the whole catalog (MCP)
claude mcp add --transport http workflowstacks https://workflowstacks.com/api/mcp

Adds a WorkflowStacks connector to Claude Code: search and load any skill here by chatting.

How Test AI with promptfoo works
Codeflow
Free to inspect

Test AI with promptfoo is a very large TypeScript project (~469k lines across 2019 code files, plus 968 test files). You install it into your AI tool with one command; there is nothing to run yourself. Last commit this month, MIT license, has a test suite.

Size
Very large codebase
~469k lines · 2019 code files · days to read — use, don't read
Setup
Install as a skill / plugin
Add it to Claude Code (or your AI tool) with one command — 17 skills inside. Nothing to run yourself.
Runs on
Inside your AI tool
Helper scripts use Node.js · Docker
TypeScript 97%CSS 2%JavaScript 1%
Where to start reading
  1. 1
    README.md
    Start here — what it does and how to install it
  2. 2
    AGENTS.md
    The instructions the AI actually follows
  3. 3
    src/main.ts
    Where the program starts running
  4. 4
    src/index.ts
    Inside src/ — the main logic begins here
  5. 5
    examples/AGENTS.md
    A worked example — copy this to get going
What's in each folder
src/Core code — the actual logic1691 files
site/Frontend / UI1279 files
plugins/Sub-packages / plugins18 files
examples/Examples you can copy1245 files
docs/Documentation15 files
drizzle/Folder52 files
scripts/Helper scripts17 files
code-scan-action/Folder13 files
READMEHas testsDocumentedCI checksDocker readyExamples includedMIT licenseUpdated this month
Quick Actions
Details
Creator
promptfoo
Language
TypeScript
Category
devtools
Published
4/28/2023

Are you the creator of this tool? and earn 85% of every sale.

Show it off in your README:

Featured on WorkflowStacks
[![Featured on WorkflowStacks](https://workflowstacks.com/api/badge/promptfoo.svg)](https://workflowstacks.com/skills/promptfoo?utm_source=github&utm_medium=badge)
🔥 Hot this week, in your inbox
Skills like this, every Monday.

The five fastest-growing open-source AI skills, ranked by GitHub star growth. One email a week, unsubscribe anytime.

Do this with a working workflow

Pages we wrote and tested that put a skill like this to work — importable n8n templates, outcome guides and Claude Desktop configs.

MCP config

GitHub MCP config

Search repos, read code, open issues and PRs from Claude.

Free n8n template

Weekly Client Report

Every Monday, turn the week’s metrics into a clear client-facing narrative report — what happened, why, what’s next — and email it automatically.

Outcome guide

Automate your weekly client reports

Keep each client’s metrics in a Google Sheet. Every Monday the workflow reads the week’s numbers, writes a plain-English narrative — what moved, why, and what’s next — and emails it to the client.

Related skills

More devtools tools founders pair with this one.

devtools★ 194,368
firecrawl – Scalable Web Scraping and Search API for Founders
Use firecrawl to quickly search, scrape, and interact with web data at any scale. Ideal for founders building AI agents, data pipelines, or content tools. 190k+ GitHub stars.
devtools★ 118,942
browser-use: Automate Web Tasks
Founders use browser-use to automate online tasks with ease, ideal for startups needing AI-powered browser automation.
devtools★ 98,350
Playwright: Unified Cross‑Browser Testing for Faster Releases
Accelerate QA with Playwright’s single API that covers Chromium, Firefox and WebKit—perfect for founders building reliable web apps. 97k+ GitHub stars.
devtools★ 642
QuantInvestStrats: Visualize and Report Financial Data in Python
Empower your fintech startup with Python tools for visualizing market data, generating performance reports, and analyzing quantitative investment strategies. Ideal for founders building data‑driven finance products.
devtools★ 1,036
AIL framework: Leak Info Analysis
Detect info leaks with AIL framework, a Python tool for founders needing darkweb scraping and data mining insights.
devtools★ 64
Build Tools for VMware Aria: Automate VCD Deployments
Simplify automation solutions for Virtual Infrastructure Administrators and Automation Developers with Build Tools for VMware Aria, 60 GitHub stars.