pragmar/mcp-server-webcrawl
MCP server tailored to connecting web crawler data and archives
Implements a Python-based MCP server with boolean fulltext search (FTS5) across crawled web data, supporting seven crawler formats including ArchiveBox, HTTrack, Katana, and WARC archives. Provides field-specific filtering by HTTP status, resource type, and content, plus preset audit prompts for SEO, broken links, and performance analysis. Integrates directly with Claude Desktop via stdio transport to enable LLMs to autonomously query and analyze archived web content.
Stars
37
Forks
14
Language
Python
License
—
Category
Last pushed
Dec 08, 2025
Commits (30d)
0
Get this data via API
curl "https://pt-edge.onrender.com/api/v1/quality/mcp/pragmar/mcp-server-webcrawl"
Open to everyone — 100 requests/day, no key needed. Get a free key for 1,000/day.
Higher-rated alternatives
firecrawl/firecrawl-mcp-server
🔥 Official Firecrawl MCP Server - Adds powerful web scraping and search to Cursor, Claude and...
bartholomej/node-csfd-api
ÄŒSFD API in JavaScript. Amazing NPM library for scrapping csfd.cz. Now with MCP server
apify/apify-mcp-server
The Apify MCP server enables your AI agents to extract data from social media, search engines,...
ScrapeGraphAI/scrapegraph-mcp
ScapeGraph MCP Server
brightdata/brightdata-mcp
A powerful Model Context Protocol (MCP) server that provides an all-in-one solution for public...