apify-mcp-server and crawl4ai-mcp-server
These are **competitors**: both provide MCP servers that expose web scraping/crawling capabilities to AI agents, with Apify offering managed cloud-based scrapers and pre-built actors while Crawl4AI offers a self-hosted, open-source alternative.
About apify-mcp-server
apify/apify-mcp-server
The Apify MCP server enables your AI agents to extract data from social media, search engines, maps, e-commerce sites, or any other website using thousands of ready-made scrapers, crawlers, and automation tools available on the Apify Store.
Implements both hosted HTTPS (with OAuth support and output schema inference) and local stdio transports compatible with Claude Desktop, VS Code, and Cursor, allowing AI agents to dynamically discover and invoke Apify Actors as MCP tools. Includes Skyfire agentic payment integration, enabling models to execute paid scraping tasks autonomously without requiring API tokens, and supports real-time tool discovery with capability detection across heterogeneous MCP clients.
About crawl4ai-mcp-server
sadiuysal/crawl4ai-mcp-server
🕷️ A lightweight Model Context Protocol (MCP) server that exposes Crawl4AI web scraping and crawling capabilities as tools for AI agents. Similar to Firecrawl's API but self-hosted and free. Perfect for integrating web scraping into your AI workflows with OpenAI Agents SDK, Cursor, Claude Code, and other MCP-compatible tools.
Implements a stdio-based MCP server using Playwright for browser automation, exposing granular tools for single-page scraping (`scrape`), multi-page breadth-first crawling (`crawl`), and large-scale sitemaps (`crawl_site`/`crawl_sitemap`) with optional persistent storage to disk. Supports adaptive crawling that intelligently stops when sufficient content is gathered, safety constraints blocking private IPs, and integration with OpenAI Agents SDK, Cursor, and Claude Code through standardized MCP configuration.
Related comparisons
Scores updated daily from GitHub, PyPI, and npm data. How scores work