lance-format/lance

Built on a Rust core with PyO3 bindings, Lance implements columnar storage with specialized indexing strategies like IVF_PQ for vector search and BM25 for full-text retrieval, enabling hybrid queries across modalities. It natively supports zero-copy blob storage for images, videos, and audio alongside embeddings, with efficient lazy loading for large objects. The format includes ACID transactions and automatic versioning without requiring separate infrastructure, integrating seamlessly with Apache Arrow, Spark, Ray, and analytical query engines like DuckDB and Trino.

6,169 stars. Actively maintained with 158 commits in the last 30 days.