ai-agent

1Cat-vLLM

V100 / SM70-focused vLLM engineering fork for modern LLM inference.
1,372 stars+59★ this week259 forksPythonUpdated 10/21/2026100% free · open source
What it does

Runs large language models (e.g., Qwen3.5 27B/35B) at 4‑bit AWQ precision on Tesla V100 (SM70) GPUs using a custom vLLM build.

When to use it
  • •You have a multi‑GPU V100 cluster and need to serve a 27‑35B model at low latency.
  • •You want to cut VRAM usage with 4‑bit AWQ quantization without losing much quality.
  • •You need an open‑source alternative to proprietary serving stacks that works with CUDA 12.8.
Ready-to-paste prompt
curl http://localhost:8000/v1/completions \
  -H "Content-Type: application/json" \
  -d '{"model":"Qwen3.5-27B-AWQ","prompt":"Write a concise 150‑word executive summary for a seed‑stage AI startup.","max_tokens":200,"temperature":0.7}'
Heads up: The server will only start on V100 (SM70) GPUs with CUDA 12.8; trying to run on a different GPU or CUDA version triggers a compilation error.
Saves to your device
Use with Claude
New

Skip the builder — one click puts this in Claude, Cursor, Antigravity and more.

✅ Light setup

Installs with a command or two; your AI agent can do it for you.

Try it instantly — no install
Claude Code
mkdir -p ~/.claude/skills/1cat-vllm && curl -fsSL https://workflowstacks.com/api/skills/1cat-vllm/claude-skill -o ~/.claude/skills/1cat-vllm/SKILL.md
Open in another AI app

Opens the app with this repo with the prompt ready to go — no copy-paste needed.

Connect the whole catalog (MCP)
claude mcp add --transport http workflowstacks https://workflowstacks.com/api/mcp

Adds a WorkflowStacks connector to Claude Code: search and load any skill here by chatting.

What's inside — free to inspect

No purchase needed

Read the entire source before you build — unlike paid marketplaces that hide it behind a buy button.

30
top-level files
19
folders
368.0M
source size
Apache-2.0
license
Key files
.pre-commit-config.yaml
AGENTS.md
README.md
File tree
.buildkite/
.gemini/
.github/
assets/
benchmarks/
cmake/
csrc/
docker/
docs/
examples/
flash_qla/
flash-attention-v100/
flashinfer-sm70/
requirements/
rust/
scripts/
tests/
tools/
vllm/
.clang-format
.coveragerc
.dockerignore
.git-blame-ignore-revs
.gitignore
Quick Actions
Details
Creator
1CatAI
Language
Python
Category
ai-agent
Published
3/10/2026

Are you the creator of this tool? and earn 85% of every sale.

Show it off in your README:

Featured on WorkflowStacks
[![Featured on WorkflowStacks](https://workflowstacks.com/api/badge/1cat-vllm.svg)](https://workflowstacks.com/skills/1cat-vllm?utm_source=github&utm_medium=badge)
🔥 Hot this week, in your inbox
Skills like this, every Monday.

The five fastest-growing open-source AI skills, ranked by GitHub star growth. One email a week, unsubscribe anytime.