spcl/MRAG

Extracts embeddings from Transformer multi-head attention activations rather than decoder layers, enabling separate vector representations for different semantic aspects within documents. Provides a complete pipeline including synthetic dataset generation, embedding computation, vector database integration (Docker-based), and evaluation framework with visualization tools. Targets complex multi-aspect queries where relevant documents have distant embeddings in traditional embedding spaces.

240 stars.