D4Vinci/Scrapling
🕷️ An adaptive Web Scraping framework that handles everything from a single request to a full-scale crawl!
Supports multiple fetcher types (StealthyFetcher, DynamicFetcher, AsyncFetcher) to bypass anti-bot systems like Cloudflare Turnstile, with adaptive parsing that automatically relocates selectors when website structures change. The Spider framework enables concurrent, multi-session crawls with built-in pause/resume, proxy rotation, and real-time streaming stats. Integrates with AI agent ecosystems via MCP server and OpenClaw skills for programmatic scraping workflows.
28,517 stars and 392,823 monthly downloads. Used by 2 other packages. Actively maintained with 126 commits in the last 30 days. Available on PyPI.
Stars
28,517
Forks
2,123
Language
Python
License
BSD-3-Clause
Category
Last pushed
Mar 11, 2026
Monthly downloads
392,823
Commits (30d)
126
Dependencies
6
Reverse dependents
2
Get this data via API
curl "https://pt-edge.onrender.com/api/v1/quality/perception/D4Vinci/Scrapling"
Open to everyone — 100 requests/day, no key needed. Get a free key for 1,000/day.