workflowstacks
devtools

Test AI with Langwatch

Evaluate LLMs and AI agents with Langwatch, a platform for founders working with AI technologies, backed by 3.3k+ GitHub stars

From the repo · “Open-Source Agent Observability & Evals for AI Platform teams. Trace, test, route and govern every agent & coding assistant in the company.”

4,988 stars+35★ this week417 forksTypeScriptGuide quality 9/10 · reviewed Jul 3Updated 10/22/2026100% free · open source
What it does

Langwatch evaluates and tests Large Language Models (LLMs) and AI agents to help founders assess their performance and capabilities.

When to use it
  • •When selecting an LLM for a specific task or project
  • •When comparing the performance of different LLMs or AI agents
  • •When testing and fine-tuning LLMs or AI agents for a particular use case
Ready-to-paste prompt
langwatch --llm gpt-3 --task text-classification --dataset my-dataset.json
Heads up: Langwatch requires a GitHub personal access token to fetch LLM models, which must be stored in a .env file at the root of the project
Saves to your device
Use with Claude
New

Skip the builder — one click puts this in Claude, Cursor, Antigravity and more.

🛠️ Technical setup

Expect 20–40 minutes in a terminal — or let your AI agent drive it.

Try it instantly — no install
Claude Code
mkdir -p ~/.claude/skills/langwatch && curl -fsSL https://workflowstacks.com/api/skills/langwatch/claude-skill -o ~/.claude/skills/langwatch/SKILL.md
Open in another AI app

Opens the app with this repo with the prompt ready to go — no copy-paste needed.

Connect the whole catalog (MCP)
claude mcp add --transport http workflowstacks https://workflowstacks.com/api/mcp

Adds a WorkflowStacks connector to Claude Code: search and load any skill here by chatting.

How Test AI with Langwatch works
Codeflow
Free to inspect

Test AI with Langwatch is a very large TypeScript project (~2M lines across 13180 code files, plus 5429 test files). It is a full software project: use it through its install path rather than reading it end to end. Last commit this month, Apache-2.0 license, has a test suite.

Size
Very large codebase
~2M lines · 13180 code files · days to read — use, don't read
Setup
Developer setup
A real software project. Use it via its install path; don't expect to read it all.
Runs on
Node.js
No API keys detected
TypeScript 80%Gherkin 9%Go 7%Python 2%Jupyter Notebook 1%
Where to start reading
  1. 1
    README.md
    Start here — what it does and how to install it
  2. 2
    CLAUDE.md
    The instructions the AI actually follows
  3. 3
    cmd/haven/main.go
    Where the program starts running
  4. 4
    go.mod
    Dependencies and the commands it exposes
  5. 5
    skills/README.md
    Inside skills/ — an example of what the AI is told to do
What's in each folder
skills/Prompts, skills & agent definitions142 files
pkg/Core code — the actual logic119 files
services/Backend / API811 files
packages/Sub-packages / plugins493 files
plugins/Sub-packages / plugins16 files
cmd/Command-line entry points8 files
docs/Documentation1459 files
platform/Folder11538 files
READMEHas testsDocumentedCI checksApache-2.0 licenseUpdated this month
Quick Actions
Details
Creator
langwatch
Language
TypeScript
Category
devtools
Published
9/9/2023

Are you the creator of this tool? and earn 85% of every sale.

Show it off in your README:

Featured on WorkflowStacks
[![Featured on WorkflowStacks](https://workflowstacks.com/api/badge/langwatch.svg)](https://workflowstacks.com/skills/langwatch?utm_source=github&utm_medium=badge)
🔥 Hot this week, in your inbox
Skills like this, every Monday.

The five fastest-growing open-source AI skills, ranked by GitHub star growth. One email a week, unsubscribe anytime.

Related skills

More devtools tools founders pair with this one.

devtools★ 194,091
firecrawl – Scalable Web Scraping and Search API for Founders
Use firecrawl to quickly search, scrape, and interact with web data at any scale. Ideal for founders building AI agents, data pipelines, or content tools. 190k+ GitHub stars.
devtools★ 118,865
browser-use: Automate Web Tasks
Founders use browser-use to automate online tasks with ease, ideal for startups needing AI-powered browser automation.
devtools★ 98,310
Playwright: Unified Cross‑Browser Testing for Faster Releases
Accelerate QA with Playwright’s single API that covers Chromium, Firefox and WebKit—perfect for founders building reliable web apps. 97k+ GitHub stars.
devtools★ 64
Build Tools for VMware Aria: Automate VCD Deployments
Simplify automation solutions for Virtual Infrastructure Administrators and Automation Developers with Build Tools for VMware Aria, 60 GitHub stars.
devtools★ 98
RedditHarbor: Easy Reddit Data
Get effortless Reddit data extraction for your database, ideal for founders needing social insights.
devtools★ 60
backage – Real‑time badge endpoints for your GitHub Packages
Provides a simple API that serves dynamic badges for GitHub Packages, letting founders instantly showcase version, downloads, and status in dashboards and READMEs.