ai-agent

Flashinfer

FlashInfer: Kernel Library for LLM Serving
6,340 stars1,393 forksPythonUpdated 9/5/2026100% free · open source
What it does

FlashInfer is a kernel library that accelerates the serving of large language models (LLMs) for AI applications, reducing latency and increasing throughput.

When to use it
  • When building real-time chatbots or virtual assistants that require fast response times
  • When serving large language models in cloud or edge environments with limited resources
  • When optimizing AI workflows that involve multiple LLMs or complex inference pipelines
Ready-to-paste prompt
python examples/infer.py --model-name bert-base-uncased --input-text 'What is the meaning of life?'
Heads up: FlashInfer requires a compatible CUDA version (11.1 or later) and a NVIDIA GPU with sufficient VRAM to run efficiently, so ensure your environment meets these prerequisites before using the library
Saves to your device
Use with Claude
New

Skip the builder — one click puts this in Claude, Cursor, Antigravity and more.

✅ Light setup

Installs with a command or two; your AI agent can do it for you.

Try it instantly — no install
Claude Code
mkdir -p ~/.claude/skills/flashinfer && curl -fsSL https://workflowstacks.com/api/skills/flashinfer/claude-skill -o ~/.claude/skills/flashinfer/SKILL.md
Open in another AI app

Opens the app with this repo with the prompt ready to go — no copy-paste needed.

Connect the whole catalog (MCP)
claude mcp add --transport http workflowstacks https://workflowstacks.com/api/mcp

Adds a WorkflowStacks connector to Claude Code: search and load any skill here by chatting.

What's inside — free to inspect

No purchase needed

Read the entire source before you build — unlike paid marketplaces that hide it behind a buy button.

22
top-level files
21
folders
80.0M
source size
Apache-2.0
license
Key files
.pre-commit-config.yaml
AGENTS.md
README.md
requirements-test.txt
requirements.txt
File tree
.claude/
.devcontainer/
.github/
3rdparty/
3rdparty_patches/
benchmarks/
ci/
csrc/
docker/
docs/
examples/
flashinfer/
flashinfer-cubin/
flashinfer-jit-cache/
flashinfer-jit-cache-provider/
include/
licenses/
profiler/
scripts/
tests/
tools/
.clang-format
.clang-format-ignore
.dockerignore
Quick Actions
Details
Creator
flashinfer-ai
Language
Python
Category
ai-agent
Published
7/22/2023

Are you the creator of this tool? Claim your listing → and earn 85% of every sale.