workflowstacks
ai-agent

lmdeploy: Accelerate LLM Compression and Deployment

Scale your AI stack fast—lmdeploy lets founders compress, deploy, and serve large language models in Python with 8.1k+ GitHub stars. Ideal for teams building AI agents.
8,103 stars763 forksPythonGuide quality 9/10 · reviewed Oct 6Updated 10/9/2026100% free · open source
What it does

LMDeploy is a toolkit for compressing, deploying, and serving large language models (LLMs) to reduce their size and improve performance

When to use it
  • •When you need to deploy an LLM in a resource-constrained environment
  • •When you want to reduce the latency of your LLM-based application
  • •When you need to serve multiple LLMs and want to optimize storage and memory usage
Ready-to-paste prompt
lmdeploy compress --model-name bert-base-uncased --output-path compressed_model --compression-ratio 0.5
Heads up: Make sure you have the necessary dependencies installed, including PyTorch and Transformers, and that your Python version is compatible with the requirements specified in the README, currently Python 3.8 or later
Saves to your device
Use with Claude
New

Skip the builder — one click puts this in Claude, Cursor, Antigravity and more.

✅ Light setup

Installs with a command or two; your AI agent can do it for you.

Try it instantly — no install
Claude Code
mkdir -p ~/.claude/skills/lmdeploy && curl -fsSL https://workflowstacks.com/api/skills/lmdeploy/claude-skill -o ~/.claude/skills/lmdeploy/SKILL.md
Open in another AI app

Opens the app with this repo with the prompt ready to go — no copy-paste needed.

Connect the whole catalog (MCP)
claude mcp add --transport http workflowstacks https://workflowstacks.com/api/mcp

Adds a WorkflowStacks connector to Claude Code: search and load any skill here by chatting.

How lmdeploy: Accelerate LLM Compression and Deployment works
Codeflow
Free to inspect

Lmdeploy: Accelerate LLM Compression and Deployment is a very large Python project (~260k lines across 1189 code files, plus 199 test files). Setup is light: installs like a normal app. Reading the code is optional. Last commit this month, Apache-2.0 license, has a test suite.

Size
Very large codebase
~260k lines · 1189 code files · days to read — use, don't read
Setup
One-command install
Installs like a normal app. Reading the code is optional.
Runs on
Python · Docker
No API keys detected
Python 70%C++ 19%Cuda 10%
Where to start reading
  1. 1
    README.md
    Start here — what it does and how to install it
  2. 2
    lmdeploy/__main__.py
    Where the program starts running
  3. 3
    pyproject.toml
    Dependencies and the commands it exposes
  4. 4
    lmdeploy/__init__.py
    Inside lmdeploy/ — the main logic begins here
  5. 5
    examples/lite/qwen3_30b_a3b_awq.py
    A worked example — copy this to get going
What's in each folder
lmdeploy/Core code — the actual logic722 files
src/Core code — the actual logic533 files
docs/Documentation137 files
examples/Examples you can copy2 files
autotest/Folder152 files
benchmark/Evaluations & benchmarks13 files
builder/Folder12 files
requirements/Folder12 files
READMEHas testsDocumentedCI checksDocker readyExamples includedApache-2.0 licenseUpdated this month
Quick Actions
Details
Creator
InternLM
Language
Python
Category
ai-agent
Published
6/15/2023

Are you the creator of this tool? and earn 85% of every sale.

Show it off in your README:

Featured on WorkflowStacks
[![Featured on WorkflowStacks](https://workflowstacks.com/api/badge/lmdeploy.svg)](https://workflowstacks.com/skills/lmdeploy?utm_source=github&utm_medium=badge)
🔥 Hot this week, in your inbox
Skills like this, every Monday.

The five fastest-growing open-source AI skills, ranked by GitHub star growth. One email a week, unsubscribe anytime.

Related skills

More ai-agent tools founders pair with this one.

ai-agent★ 297,315
superpowers
An agentic skills framework & software development methodology that works.
ai-agent★ 277,113
Track GitHub Stars with 996.ICU
Get a popular GitHub star counter for your startup, backed by 276k+ GitHub stars, ideal for founders.
ai-agent★ 251,420
Linux – 251k+ GitHub stars for building custom kernel solutions
Get the complete Linux kernel source in C, perfect for founders needing deep system control or tailored OS components.
ai-agent★ 912
llms-txt-hub: Central Hub for AI‑Ready Docs & Tools
A curated directory of llms.txt‑compliant documentation and implementations, perfect for founders building AI agents with TypeScript and Next.js.
ai-agent★ 56
Claude Opus: AI Pair Programmer by noBlock125
Get an AI pair programmer with Claude Opus interface toolkit, for founders leveraging AI in development.
ai-agent★ 29,691
Simplify Secrets with Infisical
Founders get secure access management with Infisical, an open-source platform with 28k+ GitHub stars.