lightfeed/extractor
Using LLMs and AI browser automation to robustly extract web data
Combines Playwright browser automation with LLM-powered structured extraction via Zod schemas, supporting multiple LLM providers (OpenAI, Anthropic, Gemini, Ollama) and LangChain integration. Features HTML-to-markdown conversion with content filtering, JSON recovery for malformed LLM outputs, URL validation, anti-bot evasion, and pairs with @lightfeed/browser-agent for AI-driven page navigation before data extraction.
Available on npm.
Stars
60
Forks
6
Language
TypeScript
License
Apache-2.0
Category
Last pushed
Mar 13, 2026
Monthly downloads
568
Commits (30d)
0
Dependencies
7
Get this data via API
curl "https://pt-edge.onrender.com/api/v1/quality/rag/lightfeed/extractor"
Open to everyone — 100 requests/day, no key needed. Get a free key for 1,000/day.
Related tools
any4ai/AnyCrawl
AnyCrawl 🚀: A Node.js/TypeScript crawler that turns websites into LLM-ready data and extracts...
kreuzberg-dev/html-to-markdown
High performance and CommonMark compliant HTML to Markdown converter. Maintained by the...
ScrapeGraphAI/Scrapegraph-ai
Python scraper based on AI
paulpierre/markdown-crawler
A multithreaded 🕸️ web crawler that recursively crawls a website and creates a 🔽 markdown file...
adbar/trafilatura
Python & Command-line tool to gather text and metadata on the Web: Crawling, scraping,...