illuin-tech/colpali
The code used to train and run inference with the ColVision models, e.g. ColPali, ColQwen2, and ColSmol.
Implements multi-vector embeddings for document retrieval by extracting ViT patch representations from vision language models and applying ColBERT-style late interaction ranking, eliminating the need for OCR or layout recognition pipelines. Supports multiple VLM backbones including PaliGemma, Qwen2-VL, and SmolVLM with dynamic resolution capabilities. Integrates with Hugging Face's transformers ecosystem and includes benchmark evaluation against the ViDoRe dataset for visual document retrieval.
2,555 stars. Actively maintained with 4 commits in the last 30 days.
Stars
2,555
Forks
236
Language
Python
License
MIT
Category
Last pushed
Mar 01, 2026
Commits (30d)
4
Get this data via API
curl "https://pt-edge.onrender.com/api/v1/quality/rag/illuin-tech/colpali"
Open to everyone — 100 requests/day, no key needed. Get a free key for 1,000/day.
Related tools
AnswerDotAI/byaldi
Use late-interaction multi-modal models such as ColPali in just a few lines of code.
jolibrain/colette
Multimodal RAG to search and interact locally with technical documents of any kind
nannib/nbmultirag
Un framework in Italiano ed Inglese, che permette di chattare con i propri documenti in RAG,...
OpenBMB/VisRAG
Parsing-free RAG supported by VLMs
chiang-yuan/llamp
[EMNLP '25] A web app and Python API for multi-modal RAG framework to ground LLMs on...