ai-agent

InferenceX: Benchmark Continuous AI Inference Across Leading GPUs

Name: InferenceX: Benchmark Continuous AI Inference Across Leading GPUs
Author: SemiAnalysisAI

Use InferenceX to run reproducible, multi‑hardware inference benchmarks (Kimi, MiniMax, DeepSeek, GLM, etc.) and compare performance for your LLM workloads. Ideal for founders building AI products who need data‑driven hardware decisions. 1.8k+ GitHub stars.

From the repo · “Open Source Continuous Inference Benchmark Research Platform — Kimi K3 2.8T, MiniMax M3, DeepSeekv4, GLM5 - GB200 NVL72 vs MI355X vs B200 vs GB300 NVL72 & soon™ TPUv6e/v7/Trainium2/3 | 开源持续推理基准研究平台 — Kimi K2.7-Code、MiniMax M3、DeepSeekv4、GLM5 - GB200 NVL72 vs MI355X vs B200 vs GB300 NVL72，即将推出™ TPUv6e/v7/Trainium2/3”

1,896 stars+52★ this week337 forksPythonGuide quality 9/10 · reviewed Oct 6Updated 10/21/2026100% free · open source

What it does

InferenceX is an open-source platform for benchmarking and researching continuous inference models across various hardware setups, including kimik3, MiniMax M3, and DeepSeekv4

When to use it

•Comparing the performance of different AI models on various hardware configurations
•Evaluating the efficacy of inference optimization techniques
•Benchmarking AI models for deployment on edge devices or in cloud environments

Ready-to-paste prompt

python3 benchmark.py --model MiniMax_M3 --hardware MI355X --batch_size 32 --num_iterations 100

Heads up: Ensure you have the necessary dependencies, including Python 3.8 or later, and the required hardware setups, as specified in the README, to avoid compatibility issues

Saves to your device

Use with Claude

New

Skip the builder — one click puts this in Claude, Cursor, Antigravity and more.

🛠️ Technical setup

Expect 20–40 minutes in a terminal — or let your AI agent drive it.

Try it instantly — no install

Claude Code

mkdir -p ~/.claude/skills/inferencex && curl -fsSL https://workflowstacks.com/api/skills/inferencex/claude-skill -o ~/.claude/skills/inferencex/SKILL.md

Open in another AI app

Opens the app with this repo with the prompt ready to go — no copy-paste needed.

Connect the whole catalog (MCP)

claude mcp add --transport http workflowstacks https://workflowstacks.com/api/mcp

Adds a WorkflowStacks connector to Claude Code: search and load any skill here by chatting.

How InferenceX: Benchmark Continuous AI Inference Across Leading GPUs works

Codeflow

Free to inspect

InferenceX: Benchmark Continuous AI Inference Across Leading GPUs is a very large Python project (~73k lines across 374 code files, plus 116 test files). It is a full software project: use it through its install path rather than reading it end to end. Last commit this month, Apache-2.0 license, has a test suite.

Size

Very large codebase

~73k lines · 374 code files · days to read — use, don't read

Setup

Developer setup

A real software project. Use it via its install path; don't expect to read it all.

Runs on

Python

No API keys detected

Python 93%Shell 6%

Where to start reading

1
README.md
Start here — what it does and how to install it
2
AGENTS.md
The instructions the AI actually follows
3
operatorx/main.py
Where the program starts running

What's in each folder

inferencex-e2e/Folder648 files

experimental/Folder200 files

operatorx/Folder92 files

power_model/Folder80 files

collectivex/Folder50 files

ubenchX/Folder26 files

shared/Folder2 files

.github/CI / automation (GitHub Actions)34 files

READMEHas testsDocumentedCI checksApache-2.0 licenseUpdated this month