illuin-tech/vidore-benchmark
Vision Document Retrieval (ViDoRe): Benchmark. Evaluation code for the ColPali paper.
Evaluates end-to-end document retrieval pipelines (multi-stage retrieval, hybrid approaches, custom preprocessing) on ViDoRe v3 datasets through a modular `BasePipeline` interface. Integrates with MTEB for single-model retriever evaluation and tracks both retrieval metrics and computational costs (indexing/search time). Supports language filtering, custom pipeline implementations via Python or CLI, and community-driven result submission for leaderboard comparison.
265 stars and 3,130 monthly downloads. Available on PyPI.
Stars
265
Forks
35
Language
Python
License
MIT
Category
Last pushed
Feb 27, 2026
Monthly downloads
3,130
Commits (30d)
0
Dependencies
10
Get this data via API
curl "https://pt-edge.onrender.com/api/v1/quality/rag/illuin-tech/vidore-benchmark"
Open to everyone — 100 requests/day, no key needed. Get a free key for 1,000/day.
Related tools
beir-cellar/beir
A Heterogeneous Benchmark for Information Retrieval. Easy to use, evaluate your models across...
superlinear-ai/raglite
🥤 RAGLite is a Python toolkit for Retrieval-Augmented Generation (RAG) with DuckDB or PostgreSQL
HKUDS/LightRAG
[EMNLP2025] "LightRAG: Simple and Fast Retrieval-Augmented Generation"
HKUDS/RAG-Anything
"RAG-Anything: All-in-One RAG Framework"
DataScienceUIBK/Rankify
🔥 Rankify: A Comprehensive Python Toolkit for Retrieval, Re-Ranking, and Retrieval-Augmented...