VectorDBBench and vector-db-benchmark
These are **competitors** — both are standalone benchmarking frameworks designed to evaluate vector database performance, each with their own methodology and set of tested databases, so users would typically choose one or the other based on which databases and metrics they prioritize.
About VectorDBBench
zilliztech/VectorDBBench
Benchmark for vector databases.
Supports comprehensive performance and cost-effectiveness testing across 30+ vector databases and cloud services (Milvus, Pinecone, Weaviate, pgvector, Redis, etc.) using standardized real-world datasets (SIFT, GIST, Cohere embeddings). Implements realistic production workloads including concurrent insertion, serial/concurrent searching, and filtered search scenarios with configurable parameters like dimensionality, dataset size, and concurrency levels. Provides both CLI and web UI for test execution and generates comparative result reports with cost analysis for cloud deployments.
About vector-db-benchmark
qdrant/vector-db-benchmark
Framework for benchmarking vector search engines
Supports benchmarking across multiple vector databases (Qdrant, Weaviate, Milvus, etc.) with pluggable engine implementations and configurable scenarios covering connection, indexing, data upload, and query phases. Uses a distributed server-client architecture with Docker-based engine deployment and Python clients, allowing parameter tuning via JSON configurations and wildcard-based test selection. Integrates datasets automatically via a central registry and produces standardized performance metrics across different hardware setups for comparative analysis.
Related comparisons
Scores updated daily from GitHub, PyPI, and npm data. How scores work