michalboryczko/crawler-generator-agent
Autonomous agent analyzes websites and generates production-ready crawling plans with validation datasets. Powered by a cutting-edge multi-agent orchestration architecture to discover site structure, extract optimal CSS selectors, and prepare comprehensive test suites.
Stars
—
Forks
—
Language
Python
License
—
Category
Last pushed
Jan 11, 2026
Commits (30d)
0
Get this data via API
curl "https://pt-edge.onrender.com/api/v1/quality/agents/michalboryczko/crawler-generator-agent"
Open to everyone — 100 requests/day, no key needed. Get a free key for 1,000/day.
Higher-rated alternatives
vakra-dev/reader
Open-source, production-grade web scraping engine built for LLMs. Scrape and crawl the entire...
joaobenedetmachado/scrapit
A (really) easy way to web scrape
firecrawl/open-scouts
🔥 AI-powered web monitoring platform. Create automated scouts that search the web and send email...
memvid/maw
Crawl any website into a single searchable file. Query it forever, offline.
BrowserCash/teracrawl
High-performance web crawler API optimized for LLMs. Turn any search or website into clean...