ai-agent

vllm: Fast LLM Inference

Name: vllm: Fast LLM Inference
Author: vllm-project

Get high-throughput LLM serving with vllm.

intermediate⏱ 1-2 hours💵 Free (self-hosted)

91,391 stars22,001 forksPythonGuide quality 8/10Updated 9/10/2026100% free · open source

What it is

Run large language models (LLMs) quickly and efficiently

Real use case example

"A founder wants to build a language translation feature for their travel agency's website. They use vllm to host their LLM model and set up an API endpoint to translate user input in real-time, allowing them to deliver instant results."

Best insideClaude DesktopCursor

When to use it

•When you need to serve LLMs in a production environment with high traffic
•When you want to reduce the memory footprint of your LLM deployment
•When you need to integrate LLMs with other AI services or microservices

Ready-to-paste prompt

python -m vllm.serve --model-name bert-base-uncased --port 8000

Heads up: vllm requires a compatible version of the transformers library, so ensure you have the correct version installed by running `pip install transformers==4.20.1`

Saves to your device

Use with Claude

New

Skip the builder — one click puts this in Claude, Cursor, Antigravity and more.

🛠️ Technical setup

Expect 20–40 minutes in a terminal — or let your AI agent drive it.

Try it instantly — no install

Claude Code

mkdir -p ~/.claude/skills/vllm && curl -fsSL https://workflowstacks.com/api/skills/vllm/claude-skill -o ~/.claude/skills/vllm/SKILL.md

Open in another AI app

Opens the app with this repo with the prompt ready to go — no copy-paste needed.

Connect the whole catalog (MCP)

claude mcp add --transport http workflowstacks https://workflowstacks.com/api/mcp

Adds a WorkflowStacks connector to Claude Code: search and load any skill here by chatting.

How vllm: Fast LLM Inference works

Codeflow

Free to inspect

Vllm: Fast LLM Inference is a very large Python project (~1M lines across 3062 code files, plus 1717 test files). It is a full software project: use it through its install path rather than reading it end to end. Last commit this month, Apache-2.0 license, has a test suite.

Size

Very large codebase

~1M lines · 3062 code files · days to read — use, don't read

Setup

Developer setup

A real software project. Use it via its install path; don't expect to read it all.

Runs on

Python · Docker

No API keys detected

Python 84%Rust 6%Cuda 5%C++ 3%Shell 1%

What happens, step by step

Users get fast LLM results

You give

User query

1Install vLLM

Install with uv pip

setup.py

2Run Model

Run LLM model

vllm/__init__.py

3Get Results

Get LLM results

You get

LLM results

Where to start reading

1
README.md
Start here — what it does and how to install it
2
AGENTS.md
The instructions the AI actually follows
3
pyproject.toml
Dependencies and the commands it exposes
4
vllm/__init__.py
Inside vllm/ — the main logic begins here
5
examples/__init__.py
A worked example — copy this to get going

What's in each folder

vllm/Core code — the actual logic2812 files

docs/Documentation333 files

examples/Examples you can copy258 files

rust/Folder427 files

csrc/Folder304 files

benchmarks/Evaluations & benchmarks131 files

tools/Helper scripts45 files

requirements/Folder20 files

READMEHas testsDocumentedCI checksDocker readyExamples includedApache-2.0 licenseUpdated this month

Use with Claude

New

Skip the builder — one click puts this in Claude, Cursor, Antigravity and more.

🛠️ Technical setup

Expect 20–40 minutes in a terminal — or let your AI agent drive it.

Try it instantly — no install

Claude Code

mkdir -p ~/.claude/skills/vllm && curl -fsSL https://workflowstacks.com/api/skills/vllm/claude-skill -o ~/.claude/skills/vllm/SKILL.md

Open in another AI app

Opens the app with this repo with the prompt ready to go — no copy-paste needed.

Connect the whole catalog (MCP)

claude mcp add --transport http workflowstacks https://workflowstacks.com/api/mcp

Adds a WorkflowStacks connector to Claude Code: search and load any skill here by chatting.

Quick Actions

Details

Creator

vllm-project

Language

Python

Related skills

More ai-agent tools founders pair with this one.

ai-agent★ 276,806

Track GitHub Stars with 996.ICU

ai-agent★ 264,191

Superpowers

An agentic skills framework & software development methodology that works.

ai-agent★ 244,424

Linux: Open Source Kernel

Get the Linux kernel source tree for custom development, ideal for startup founders in need of a flexible OS foundation

ai-agent★ 210,750

Vue

This is the repo for Vue 2. For Vue 3, go to https://github.com/vuejs/core

ai-agent★ 204,069

Improve Code with andrej-karpathy-skills

Get better code behavior with andrej-karpathy-skills, 165k+ GitHub stars, for founders using LLMs.

ai-agent★ 197,010

FreeDomain: Free Website Domain

Get a free domain with FreeDomain.