Crawlee: Reliable Web Scraping
From the repo · “Crawlee—A web scraping and browser automation library for Node.js to build reliable crawlers. In JavaScript and TypeScript. Extract data for AI, LLMs, RAG, or GPTs. Download HTML, PDF, JPG, PNG, and other files from websites. Works with Puppeteer, Playwright, Cheerio, JSDOM, and raw HTTP. Both headful and headless mode. With proxy rotation.”
Use Crawlee to automatically visit lots of websites in your Node.js projects.
"A solo developer, building a product recommendation platform, can use Crawlee to pull product details from Amazon and Home Depot websites by setting up a few lines of code and configuring the library's settings. They can then integrate this data with their custom AI model. By doing this, they can provide accurate and up-to-date product information to their customers."
- •When you need to extract data from websites that use JavaScript rendering
- •When you want to automate browser interactions for data extraction or testing
- •When you need to download files from websites in a scalable and reliable way
const { Crawlee } = require('crawlee'); const crawler = new Crawlee.Crawler({ puppeteer: { launchOptions: { headless: false } } }, async ({ page, statusCode }) => { await page.goto('https://www.example.com'); const title = await page.title(); console.log(title); }); await crawler.run();Skip the builder — one click puts this in Claude, Cursor, Antigravity and more.
Installs with a command or two; your AI agent can do it for you.
mkdir -p ~/.claude/skills/crawlee && curl -fsSL https://workflowstacks.com/api/skills/crawlee/claude-skill -o ~/.claude/skills/crawlee/SKILL.mdOpens the app with this repo with the prompt ready to go — no copy-paste needed.
Crawlee: Reliable Web Scraping is a very large TypeScript project (~75k lines across 1309 code files, plus 231 test files). Setup is light: installs like a normal app. Reading the code is optional. Last commit this month, Apache-2.0 license, has a test suite.
- 1README.mdStart here — what it does and how to install it
- 2package.jsonDependencies and the commands it exposes
Skip the builder — one click puts this in Claude, Cursor, Antigravity and more.
Installs with a command or two; your AI agent can do it for you.
mkdir -p ~/.claude/skills/crawlee && curl -fsSL https://workflowstacks.com/api/skills/crawlee/claude-skill -o ~/.claude/skills/crawlee/SKILL.mdOpens the app with this repo with the prompt ready to go — no copy-paste needed.
Are you the creator of this tool? and earn 85% of every sale.
Show it off in your README:
[](https://workflowstacks.com/skills/crawlee?utm_source=github&utm_medium=badge)The five fastest-growing open-source AI skills, ranked by GitHub star growth. One email a week, unsubscribe anytime.
Do this with a working workflow
Pages we wrote and tested that put a skill like this to work — importable n8n templates, outcome guides and Claude Desktop configs.
Context7 (live docs) MCP config
Pull up-to-date library docs into Claude so it codes against the real API.
MCP configPuppeteer (browser) MCP config
Let Claude control a headless browser — navigate, screenshot, scrape.
MCP configPlaywright (browser) MCP config
Drive a real browser for testing and automation from Claude.
Related skills
More devtools tools founders pair with this one.