ai-agent

Rtp LLM

RTP-LLM: Alibaba's high-performance LLM inference engine for diverse applications.
1,330 stars271 forksCudaUpdated 9/8/2026100% free · open source
What it does

RTP-LLM is a high-performance large language model inference engine for diverse applications, enabling efficient deployment of LLMs in various scenarios.

When to use it
  • When you need to deploy large language models for natural language processing tasks
  • When you require a high-performance inference engine for real-time applications
  • When you want to optimize the computational resources for LLM deployment
Ready-to-paste prompt
python examples/inference.py --model-name bert-base-uncased --input-text 'What is the capital of France?'
Heads up: Ensure that you have the required CUDA version (11.1 or later) and cuDNN version (8.0 or later) installed to successfully build and run RTP-LLM
Saves to your device
Use with Claude
New

Skip the builder — one click puts this in Claude, Cursor, Antigravity and more.

✅ Light setup

Installs with a command or two; your AI agent can do it for you.

Try it instantly — no install
Claude Code
mkdir -p ~/.claude/skills/rtp-llm && curl -fsSL https://workflowstacks.com/api/skills/rtp-llm/claude-skill -o ~/.claude/skills/rtp-llm/SKILL.md
Open in another AI app

Opens the app with this repo with the prompt ready to go — no copy-paste needed.

Connect the whole catalog (MCP)
claude mcp add --transport http workflowstacks https://workflowstacks.com/api/mcp

Adds a WorkflowStacks connector to Claude Code: search and load any skill here by chatting.

What's inside — free to inspect

No purchase needed

Read the entire source before you build — unlike paid marketplaces that hide it behind a buy button.

21
top-level files
16
folders
696.5M
source size
Apache-2.0
license
Key files
.pre-commit-config.yaml
pyrightconfig.json
README_cn.md
README.md
File tree
.githooks/
.github/
3rdparty/
arch_config/
bazel/
benchmark/
deps/
docker/
docs/
example/
package/
patches/
picture/
rtp_llm/
scripts/
tools/
.bazelignore
.bazeliskrc
.bazelrc
.clang-format
.dockerignore
.flake8
.gitattributes
.gitignore
Quick Actions
Details
Creator
alibaba
Language
Cuda
Category
ai-agent
Published
12/27/2023

Are you the creator of this tool? Claim your listing → and earn 85% of every sale.