apify/crawlee-python
Crawlee—A web scraping and browser automation library for Python to build reliable crawlers. Extract data for AI, LLMs, RAG, or GPTs. Download HTML, PDF, JPG, PNG, and other files from websites. Works with Parsel, BeautifulSoup, Playwright, and raw HTTP. Both headful and headless mode. With proxy rotation.
8,682 stars. Used by 1 other package. Actively maintained with 59 commits in the last 30 days. Available on PyPI.
Stars
8,682
Forks
706
Language
Python
License
Apache-2.0
Category
Last pushed
Mar 28, 2026
Commits (30d)
59
Dependencies
13
Reverse dependents
1
Get this data via API
curl "https://pt-edge.onrender.com/api/v1/quality/perception/apify/crawlee-python"
Open to everyone — 100 requests/day, no key needed. Get a free key for 1,000/day.
Related tools
seleniumbase/SeleniumBase
APIs for browser automation, testing, and bypassing bot-detection.
apify/crawlee
Crawlee—A web scraping and browser automation library for Node.js to build reliable crawlers. In...
Kaliiiiiiiiii-Vinyzu/patchright
Undetected version of the Playwright testing and automation library.
intoli/user-agents
A JavaScript library for generating random user agents with data that's updated daily.
microlinkhq/browserless
The headless Chrome/Chromium driver on top of Puppeteer. Take screenshots, generate PDFs,...