philschmid/clipper.js
HTML to Markdown converter and crawler.
Leverages Mozilla's Readability for intelligent content extraction and Turndown for HTML-to-Markdown conversion, with optional Playwright-based crawling for batch processing entire sites. Supports multiple input formats (URLs, local HTML files, directories) and output formats (Markdown, JSONL), making it useful for dataset generation and web archival workflows. Can be chained with tools like poppler for PDF-to-Markdown conversion pipelines.
614 stars. No commits in the last 6 months.
Stars
614
Forks
39
Language
TypeScript
License
Apache-2.0
Category
Last pushed
Jan 09, 2024
Commits (30d)
0
Get this data via API
curl "https://pt-edge.onrender.com/api/v1/quality/rag/philschmid/clipper.js"
Open to everyone — 100 requests/day, no key needed. Get a free key for 1,000/day.
Higher-rated alternatives
any4ai/AnyCrawl
AnyCrawl 🚀: A Node.js/TypeScript crawler that turns websites into LLM-ready data and extracts...
kreuzberg-dev/html-to-markdown
High performance and CommonMark compliant HTML to Markdown converter. Maintained by the...
lightfeed/extractor
Using LLMs and AI browser automation to robustly extract web data
ScrapeGraphAI/Scrapegraph-ai
Python scraper based on AI
paulpierre/markdown-crawler
A multithreaded 🕸️ web crawler that recursively crawls a website and creates a 🔽 markdown file...