ThoughtRiver/lmdb-embeddings

Leverages Lightning Memory-Mapped Database (LMDB) to enable zero-load-time access to pre-trained embeddings with negligible memory overhead—large models like GloVe-840B require only a few MB versus 4GB traditionally. Supports pluggable serialization backends (pickle, msgpack) and includes an LRU cache for frequently accessed vectors, with transparent compatibility for gensim models and custom embedding iterators.

416 stars. No commits in the last 6 months.