Engine-agnostic LLM gateway in Rust. Full OpenAI & Anthropic API compatibility across vLLM, TRT-LLM, TokenSpeed, SGLang, OpenAI, Gemini & more. Industry-first gRPC pipeline, KV cache-aware routing, chat history, tokenization caching, Responses API, embeddings, WASM plugins, MCP, and multi-tenant auth.
Read the entire source before you build โ unlike paid marketplaces that hide it behind a buy button.
Are you the creator of this tool? Claim your listing โ and earn 85% of every sale.
More mcp-server tools founders pair with this one.