vakra-dev/reader
Open-source, production-grade web scraping engine built for LLMs. Scrape and crawl the entire web, clean markdown, ready for your agents.
Leverages [Ulixee Hero](https://ulixee.org/), a headless browser with built-in anti-bot defenses (TLS fingerprinting, Cloudflare bypass, DNS-over-TLS), managed through a pooled architecture with automatic recycling and health monitoring. Provides two core primitives—`scrape()` for converting URLs to cleaned markdown/HTML, and `crawl()` for breadth-first site discovery—with configurable browser pooling, proxy rotation strategies, batch concurrency, and graceful degradation handling all abstracted away.
474 stars and 196 monthly downloads. Available on npm.
Stars
474
Forks
32
Language
TypeScript
License
Apache-2.0
Category
Last pushed
Feb 02, 2026
Monthly downloads
196
Commits (30d)
0
Dependencies
9
Get this data via API
curl "https://pt-edge.onrender.com/api/v1/quality/agents/vakra-dev/reader"
Open to everyone — 100 requests/day, no key needed. Get a free key for 1,000/day.
Related agents
joaobenedetmachado/scrapit
A (really) easy way to web scrape
firecrawl/open-scouts
🔥 AI-powered web monitoring platform. Create automated scouts that search the web and send email...
BrowserCash/teracrawl
High-performance web crawler API optimized for LLMs. Turn any search or website into clean...
memvid/maw
Crawl any website into a single searchable file. Query it forever, offline.
ma-pony/deepspider
智能爬虫工程平台 - 基于 DeepAgents + Patchright 的 AI 爬虫 Agent | Intelligent Web Scraping Platform -...