revokslab/codecrawl
🌊 Turn entire codebases into LLM-ready data. Extract data, search, and llms.txt from any repo with a single API.
Crawls public repositories from GitHub/GitLab and converts them into vector-searchable markdown or structured data with semantic indexing, enabling AI applications to query codebases with contextual understanding. Supports async job-based processing for large repositories, with configurable output formats (markdown, XML, plain text), token counting, and file analytics. Provides both hosted API and self-hosted deployment options, with SDK support for Node.js and planned additional integrations.
Stars
79
Forks
8
Language
TypeScript
License
AGPL-3.0
Category
Last pushed
Jan 25, 2026
Commits (30d)
0
Get this data via API
curl "https://pt-edge.onrender.com/api/v1/quality/embeddings/revokslab/codecrawl"
Open to everyone — 100 requests/day, no key needed. Get a free key for 1,000/day.
Higher-rated alternatives
cocoindex-io/cocoindex
Data transformation framework for AI. Ultra performant, with incremental processing. 🌟 Star if...
dtsola/xiaoyaosearch
小遥搜索，听懂你的话、看懂你的图，用AI找到本地任何文件。让搜索像聊天一样简单。XiaoyaoSearch: Understands your words, reads your...
Ryandonofrio3/osgrep
Open Source Semantic Search for your AI Agent
justincasher/lean-explore
A search engine for Lean 4 declarations
yoanbernabeu/grepai
Semantic Search & Call Graphs for AI Agents (100% Local)