ai-agent

flashinfer

Name: flashinfer
Author: flashinfer-ai

FlashInfer: Kernel Library for LLM Serving

6,744 stars+119★ this week1,637 forksCudaUpdated 10/22/2026100% free · open source

What it does

FlashInfer is a kernel library that accelerates the serving of large language models (LLMs) for AI applications, reducing latency and increasing throughput.

When to use it

•When building real-time chatbots or virtual assistants that require fast response times
•When serving large language models in cloud or edge environments with limited resources
•When optimizing AI workflows that involve multiple LLMs or complex inference pipelines

Ready-to-paste prompt

python examples/infer.py --model-name bert-base-uncased --input-text 'What is the meaning of life?'

Heads up: FlashInfer requires a compatible CUDA version (11.1 or later) and a NVIDIA GPU with sufficient VRAM to run efficiently, so ensure your environment meets these prerequisites before using the library

Saves to your device

Use with Claude

New

Skip the builder — one click puts this in Claude, Cursor, Antigravity and more.

✅ Light setup

Installs with a command or two; your AI agent can do it for you.

Try it instantly — no install

Claude Code

mkdir -p ~/.claude/skills/flashinfer && curl -fsSL https://workflowstacks.com/api/skills/flashinfer/claude-skill -o ~/.claude/skills/flashinfer/SKILL.md

Open in another AI app

Opens the app with this repo with the prompt ready to go — no copy-paste needed.

Connect the whole catalog (MCP)

claude mcp add --transport http workflowstacks https://workflowstacks.com/api/mcp

Adds a WorkflowStacks connector to Claude Code: search and load any skill here by chatting.

How flashinfer works

Codeflow

Free to inspect

Flashinfer is a very large Cuda project (~1.2M lines across 1885 code files, plus 642 test files). Setup is light: installs like a normal app. Reading the code is optional. Last commit this month, Apache-2.0 license, has a test suite.

Size

Very large codebase

~1.2M lines · 1885 code files · days to read — use, don't read

Setup

One-command install

Installs like a normal app. Reading the code is optional.

Runs on

Python

No API keys detected

Cuda 70%Python 25%C++ 5%

Where to start reading

1
README.md
Start here — what it does and how to install it
2
AGENTS.md
The instructions the AI actually follows
3
flashinfer/__main__.py
Where the program starts running
4
pyproject.toml
Dependencies and the commands it exposes
5
flashinfer/__init__.py
Inside flashinfer/ — the main logic begins here

What's in each folder

flashinfer/Core code — the actual logic1203 files

docs/Documentation79 files

examples/Examples you can copy12 files

csrc/Folder2813 files

include/Folder262 files

benchmarks/Evaluations & benchmarks155 files

scripts/Helper scripts76 files

ci/Folder9 files

READMEHas testsDocumentedCI checksExamples includedApache-2.0 licenseUpdated this month

Use with Claude

New

Skip the builder — one click puts this in Claude, Cursor, Antigravity and more.

✅ Light setup

Installs with a command or two; your AI agent can do it for you.

Try it instantly — no install

Claude Code

mkdir -p ~/.claude/skills/flashinfer && curl -fsSL https://workflowstacks.com/api/skills/flashinfer/claude-skill -o ~/.claude/skills/flashinfer/SKILL.md

Open in another AI app

Opens the app with this repo with the prompt ready to go — no copy-paste needed.

Connect the whole catalog (MCP)

claude mcp add --transport http workflowstacks https://workflowstacks.com/api/mcp

Adds a WorkflowStacks connector to Claude Code: search and load any skill here by chatting.

Quick Actions

Details

Creator

flashinfer-ai

Language

Cuda