spider-rs/spider-clients
Python, Javascript, and Rust libraries for the Spider Cloud API.
Multi-language bindings (Python, JavaScript, Rust, Go, CLI) for Spider Cloud's web crawling API, enabling concurrent scraping with real-time streaming and headless Chrome rendering. Core features include AI-driven smart mode for automated crawling strategies, HTTP proxy support, cron scheduling, and dynamic prompt scripting for JavaScript-heavy sites. Designed for both local deployment and cloud-hosted workflows with fine-grained control via URL blacklisting, whitelisting, and crawl depth budgeting.
Stars
23
Forks
9
Language
Rust
License
MIT
Category
Last pushed
Mar 07, 2026
Commits (30d)
0
Get this data via API
curl "https://pt-edge.onrender.com/api/v1/quality/agents/spider-rs/spider-clients"
Open to everyone — 100 requests/day, no key needed. Get a free key for 1,000/day.
Compare
Higher-rated alternatives
spider-rs/spider
Web crawler and scraper for Rust
Dicklesworthstone/pi_agent_rust
High-performance AI coding agent CLI written in Rust with zero unsafe code
RightNow-AI/openfang
Open-source Agent Operating System
liquidos-ai/AutoAgents
A multi-agent framework written in Rust that enables you to build, deploy, and coordinate...
Ataraxy-Labs/sem
Semantic version control CLI. Entity-level diff, blame, graph, and impact analysis for code. 16...