automation

opik: Debug and Scale LLM Apps with 22k+ GitHub Stars

Name: opik: Debug and Scale LLM Apps with 22k+ GitHub Stars
Author: comet-ml

Turn debugging, evaluation, and monitoring of LLM, RAG, and agentic workflows into a single, production‑ready dashboard. Ideal for founders building AI‑first products.

intermediate⏱ 1-2 hours💵 Free (self-hosted)

22,474 stars+140★ this week1,847 forksPythonGuide quality 9/10 · reviewed Oct 5Install verified Sep 21Updated 10/9/2026100% free · open source

What it is

Use opik to debug and test your AI-powered applications.

Real use case example

"A founder named Alex uses opik to debug a LLM-powered chatbot. They set up opik to automatically track errors and discrepancies in responses. After a few hours, opik sends them an alert when it detects a pattern of misclassifications. Alex investigates and refines the model, greatly improving its accuracy."

Best insideCursorCodex CLIContinue

When to use it

•When you need to identify and fix issues in your LLM application's workflow
•When you want to evaluate and compare the performance of different LLM models
•When you need to monitor and analyze the performance of your RAG systems and agentic workflows

Ready-to-paste prompt

opik.trace(my_llm_model, input_data='example_input', output_data='example_output')

Heads up: Make sure to set the `OPIK_API_KEY` environment variable with a valid Comet ML API key to use opik's production-ready dashboards and automated evaluations

Saves to your device

Use with Claude

New

Skip the builder — one click puts this in Claude, Cursor, Antigravity and more.

🔑 Needs an API key✓ Install verified 9/21/2026

Works after you add credentials — the setup agent will ask you for them.

Try it instantly — no install

Claude Code

mkdir -p ~/.claude/skills/opik && curl -fsSL https://workflowstacks.com/api/skills/opik/claude-skill -o ~/.claude/skills/opik/SKILL.md

Open in another AI app

Opens the app with this repo with the prompt ready to go — no copy-paste needed.

Connect the whole catalog (MCP)

claude mcp add --transport http workflowstacks https://workflowstacks.com/api/mcp

Adds a WorkflowStacks connector to Claude Code: search and load any skill here by chatting.

How opik: Debug and Scale LLM Apps with 22k+ GitHub Stars works

Codeflow

Free to inspect

Opik: Debug and Scale LLM Apps with 22k+ GitHub Stars is a very large Python project (~754k lines across 7930 code files, plus 1676 test files). Setup is light: installs like a normal app — add your API keys. Reading the code is optional. Last commit this month, Apache-2.0 license, has a test suite.

Size

Very large codebase

~754k lines · 7930 code files · days to read — use, don't read

Setup

One-command install

Installs like a normal app — add your API keys. Reading the code is optional.

Runs on

Python

Needs API keys (.env)

Python 59%TypeScript 39%Shell 1%

Where to start reading

1
README.md
Start here — what it does and how to install it
2
AGENTS.md
The instructions the AI actually follows
3
.env.template
The API keys and settings you must provide

What's in each folder

apps/Sub-packages / plugins5279 files

extensions/Sub-packages / plugins30 files

sdks/SDKs / client libraries5369 files

tests_end_to_end/Folder214 files

deployment/Deployment / infrastructure49 files

scripts/Helper scripts28 files

tests_load/Folder23 files

.github/CI / automation (GitHub Actions)113 files

READMEHas testsDocumentedCI checksApache-2.0 licenseUpdated this month

Use with Claude

New

Skip the builder — one click puts this in Claude, Cursor, Antigravity and more.

🔑 Needs an API key✓ Install verified 9/21/2026

Works after you add credentials — the setup agent will ask you for them.

Try it instantly — no install

Claude Code

mkdir -p ~/.claude/skills/opik && curl -fsSL https://workflowstacks.com/api/skills/opik/claude-skill -o ~/.claude/skills/opik/SKILL.md

Open in another AI app

Opens the app with this repo with the prompt ready to go — no copy-paste needed.

Connect the whole catalog (MCP)

claude mcp add --transport http workflowstacks https://workflowstacks.com/api/mcp

Adds a WorkflowStacks connector to Claude Code: search and load any skill here by chatting.

Quick Actions

Details

Creator

comet-ml

Language

Python

Do this with a working workflow

Pages we wrote and tested that put a skill like this to work — importable n8n templates, outcome guides and Claude Desktop configs.

Outcome guide

Turn meeting notes into a summary email automatically

Paste the transcript into a private form and the workflow writes and emails a concise recap — decisions, owners, open questions — that people will actually read.

Outcome guide

Automated marketing performance reports

The workflow reads your performance metrics and writes the narrative for you — this week at a glance, what moved and why, what’s next — then sends it on schedule.