Vector DB Benchmarking Vector Databases

Performance evaluation frameworks and tools for comparing vector databases across metrics like latency, throughput, recall, and cost. Includes benchmarking suites, tuning optimization systems, and comparative analysis frameworks. Does NOT include individual vector database implementations, retrieval pipeline frameworks, or production RAG applications.

There are 27 vector db benchmarking tools tracked. 1 score above 70 (verified tier). The highest-rated is lancedb/lancedb at 94/100 with 9,425 stars and 6,082,637 monthly downloads. 2 of the top 10 are actively maintained.

Get all 27 projects as JSON

curl "https://pt-edge.onrender.com/api/v1/datasets/quality?domain=vector-db&subcategory=vector-db-benchmarking&limit=20"

Open to everyone — 100 requests/day, no key needed. Get a free key for 1,000/day.

#	Tool	Score	Tier	Stars	Language
1	lancedb/lancedb Developer-friendly OSS embedded retrieval library for multimodal AI. Search...	94	Verified	9,425	HTML
2	zilliztech/VectorDBBench Benchmark for vector databases.	64	Established	1,038	Python
3	qdrant/vector-db-benchmark Framework for benchmarking vector search engines	54	Established	353	Python
4	prrao87/lancedb-study Comparing LanceDB and Elasticsearch for full-text search and vector search...	42	Emerging	29	Python
5	vector-index-bench/vibe Vector Index Benchmark for Embeddings (VIBE) is an extensible benchmark for...	40	Emerging	36	Python
6	myscale/vector-db-benchmark Framework for benchmarking fully-managed vector databases	37	Emerging	80	Python
7	lancedb/docs Documentation for LanceDB, the Multimodal Lakehouse for AI	35	Emerging	6	Python
8	avnlp/vectordb Production-ready Haystack/LangChain pipelines for Hybrid & Parent-Child...	34	Emerging	4	Python
9	tiannuo-yang/VDTuner [ICDE 2024] VDTuner - Automated Performance Tuning for Vector Data...	33	Emerging	35	Python
10	scientist-labs/lancelot Ruby bindings for the Lance columnar data format. Built on the Lance Rust...	26	Experimental	7	Ruby
11	microsoft/VBench An Approximate Vector-Analytics Benchmark for Relational Databases	24	Experimental	2	Python
12	M4iKZ/Vector-Arena A comprehensive, multiprocessing-isolated benchmark for evaluating vector...	23	Experimental	1	Python
13	Praxis-Collective/tdr-vector-decay Native temporal decay scoring for vector database retrieval — no...	22	Experimental	—	Python
14	lennylxx/lancedb-csharp LanceDB C# SDK	20	Experimental	1	C#
15	haja-k/tidb-vector-llm-testbed Experimental framework for evaluating TiDB’s vector search capabilities with...	15	Experimental	—	Python
16	ktsouvalis/mscacs-thesis Official implementation for the MSc Thesis: "Performance Evaluation of...	15	Experimental	—	Python
17	shift-ai007/vector-database-guide Vector database comparison for production RAG — Pinecone vs pgvector vs...	14	Experimental	—	—
18	gordonmurray/metabare.com A lightweight digital asset manager (DAM) focused on simplicity, fast...	14	Experimental	4	Python
19	Vill-Lab/2024-TPAMI-VGDCU Benchmark for VG-based Detection and Chart Understanding (IEEE T-PAMI 2024)	13	Experimental	6	—
20	Devparihar5/chromdb-vs-pgvector-benchmark This repository contains a comprehensive benchmark suite for comparing...	12	Experimental	1	Python
21	offlabel-scaffolds/scaffold-vector-db-app Vector database application with Qdrant/Weaviate	11	Experimental	—	Python
22	Ashish-Soni08/DKE-Thesis-OVGU Master thesis research and implementation for the Data and Knowledge...	11	Experimental	—	—
23	PeterShin23/db-benchmark Comparing popular Vector DB options used for RAG search	11	Experimental	—	Python
24	tpancholi/vector-db-benchmark A python repository to benchmark different vector db based on search...	11	Experimental	—	Python
25	bijay-odyssey/vectorDB A Qdrant Vector Database demo for semantic search and RAG, using Sentence...	11	Experimental	—	Python
26	aimanmuzaffar95/vector-databases-benchmarks Benchmark and compare vector databases (pgvector, Qdrant, Milvus, Weaviate,...	11	Experimental	—	Python
27	Eurphus/Vector-DB-Testing Vector Database testing for semantic search capabilities for use in LLM applications	11	Experimental	2	Python

Comparisons in this category

lancedb and lancedb-study (94 vs 42) lancedb and lancelot (94 vs 26) lancedb and docs (94 vs 35) lancedb and lancedb-csharp (94 vs 20) VectorDBBench and vector-db-benchmark (64 vs 54) VectorDBBench and vector-db-benchmark (64 vs 37) VectorDBBench and VBench (64 vs 24) VectorDBBench and Vector-Arena (64 vs 23) vector-db-benchmark and VBench (54 vs 24) vector-db-benchmark and Vector-Arena (54 vs 23)