VectorDBBench and vector-db-benchmark
These are **competitors**: both provide independent benchmarking frameworks for evaluating vector database performance, with VectorDBBench offering broader coverage of multiple vector DB systems while MyScale's benchmark is specialized for fully-managed offerings.
About VectorDBBench
zilliztech/VectorDBBench
Benchmark for vector databases.
Supports comprehensive performance and cost-effectiveness testing across 30+ vector databases and cloud services (Milvus, Pinecone, Weaviate, pgvector, Redis, etc.) using standardized real-world datasets (SIFT, GIST, Cohere embeddings). Implements realistic production workloads including concurrent insertion, serial/concurrent searching, and filtered search scenarios with configurable parameters like dimensionality, dataset size, and concurrency levels. Provides both CLI and web UI for test execution and generates comparative result reports with cost analysis for cloud deployments.
About vector-db-benchmark
myscale/vector-db-benchmark
Framework for benchmarking fully-managed vector databases
Measures throughput (queries per second) and cost-performance ratios across fully-managed vector database services, including filtered vector search workloads. The framework executes standardized workloads against cloud-hosted databases and aggregates results into comparative cost-per-100-QPS metrics. Built on extensible benchmarking infrastructure that supports adding new managed database providers through configurable client implementations.
Related comparisons
Scores updated daily from GitHub, PyPI, and npm data. How scores work