workflowstacks
ai-agent

paddler: Scale LLMs

Founders get a simple load balancer for self-hosting large language models with paddler, used by the community
intermediate⏱ 1-2 hours💵 Free (self-hosted)
1,646 stars92 forksRustGuide quality 9/10Updated 7/19/2026100% free · open source
What it is

You can self-host Large Language Models (LLMs) with paddler to create highly customized applications.

Real use case example

"A solo developer is creating a chatbot for their startup's customer support. They use paddler to host their LLM, set up load balancing, and serve queries from their users. They configure paddler to pull product data from Amazon to answer customers' product-related questions. With paddler, their chatbot can now provide more accurate and helpful responses."

Best insideSelf-hosted
When to use it
  • •When you need to deploy and manage multiple LLMs or VLMs in a self-hosted environment
  • •When you require a simple and lightweight load balancer for your AI models
  • •When you want to use a ggml ecosystem-based solution for deploying and serving AI models
Ready-to-paste prompt
curl -X POST 'http://localhost:8080/infer' -H 'Content-Type: application/json' -d '{"model": "my_llm", "prompt": "Write a story about a character who learns a new skill"}'
Heads up: Make sure you have Rust and Cargo installed on your system, as paddler is built using Rust and requires these tools to build and run
Saves to your device
Use with Claude
New

Skip the builder — one click puts this in Claude, Cursor, Antigravity and more.

✅ Light setup

Installs with a command or two; your AI agent can do it for you.

Try it instantly — no install
Claude Code
mkdir -p ~/.claude/skills/paddler && curl -fsSL https://workflowstacks.com/api/skills/paddler/claude-skill -o ~/.claude/skills/paddler/SKILL.md
Open in another AI app

Opens the app with this repo with the prompt ready to go — no copy-paste needed.

Connect the whole catalog (MCP)
claude mcp add --transport http workflowstacks https://workflowstacks.com/api/mcp

Adds a WorkflowStacks connector to Claude Code: search and load any skill here by chatting.

How paddler: Scale LLMs works
Codeflow
Free to inspect

Paddler: Scale LLMs is a large Rust project (~46k lines across 731 code files, plus 258 test files). Setup is light: runs with Docker. Last commit 2 months ago, Apache-2.0 license, has a test suite.

Size
Large codebase
~46k lines · 731 code files · ~6 h to skim
Setup
Light setup
Runs with Docker.
Runs on
Rust · Docker
No API keys detected
Rust 84%TypeScript 8%Python 5%CSS 1%Nix 1%
Where to start reading
  1. 1
    README.md
    Start here — what it does and how to install it
  2. 2
    CLAUDE.md
    The instructions the AI actually follows
  3. 3
    Cargo.toml
    Dependencies and the commands it exposes
  4. 4
    example/README.md
    A worked example — copy this to get going
What's in each folder
example/Examples you can copy2 files
paddler_tests/Folder232 files
paddler_balancer/Folder163 files
resources/Folder139 files
paddler_agent/Folder99 files
paddler_messaging/Folder91 files
paddler_client_python/Folder81 files
paddler_client_javascript/Folder75 files
READMEHas testsDocumentedCI checksDocker readyExamples includedApache-2.0 licenseUpdated 2 mo ago
Quick Actions
Details
Creator
intentee
Language
Rust
Category
ai-agent
Published
4/27/2024

Are you the creator of this tool? Claim your listing → and earn 85% of every sale.

Related skills

More ai-agent tools founders pair with this one.

ai-agent★ 276,806
Track GitHub Stars with 996.ICU
Get a popular GitHub star counter for your startup, backed by 276k+ GitHub stars, ideal for founders.
ai-agent★ 264,191
Superpowers
An agentic skills framework & software development methodology that works.
ai-agent★ 244,424
Linux: Open Source Kernel
Get the Linux kernel source tree for custom development, ideal for startup founders in need of a flexible OS foundation with 236k+ GitHub stars
ai-agent★ 210,750
Vue
This is the repo for Vue 2. For Vue 3, go to https://github.com/vuejs/core
ai-agent★ 204,069
Improve Code with andrej-karpathy-skills
Get better code behavior with andrej-karpathy-skills, 165k+ GitHub stars, for founders using LLMs.
ai-agent★ 197,010
FreeDomain: Free Website Domain
Get a free domain with FreeDomain. For startup founders.