mburaksayici/smallevals
smallevals — CPU-fast, GPU-blazing fast offline retrieval evaluation for RAG systems with tiny QA models.
Supports vector database abstraction across 8+ backends (Chroma, Elasticsearch, Milvus, pgvector, Qdrant, Weaviate, FAISS, Pinecone), abstracting vector DB operations behind a unified interface. Generates synthetic evaluation datasets and retrieval metrics by prompting QAG-0.6B (a Qwen-based model) to create questions from document chunks, then measures if the vector DB retrieves those chunks in top-k results. Includes a Dash-based dashboard for visualizing retrieval scores and includes a CLI for batch synthetic QA generation from local documents.
Available on PyPI.
Stars
18
Forks
2
Language
Python
License
—
Category
Last pushed
Dec 04, 2025
Monthly downloads
41
Commits (30d)
0
Dependencies
28
Get this data via API
curl "https://pt-edge.onrender.com/api/v1/quality/vector-db/mburaksayici/smallevals"
Open to everyone — 100 requests/day, no key needed. Get a free key for 1,000/day.
Higher-rated alternatives
danny-avila/rag_api
ID-based RAG FastAPI: Integration with Langchain and PostgreSQL/pgvector
naxoc/riffrag
A local RAG builder for code with a Claude Code skills creator
kxgst228/rag-forge
Modular RAG framework with hybrid retrieval, intelligent chunking, and multi-provider LLM support
GoparapukethaN/rag-forge
Modular RAG framework with hybrid retrieval, intelligent chunking, and multi-provider LLM support
AlbertMein/rag-document-processing
RAG pipeline components: document loaders, chunking, vector stores, retrieval