KarelDO/xmc.dspy

Implements a three-stage pipeline (Infer-Retrieve-Rank) that combines language models with dense retrievers to handle extreme multi-label classification without finetuning. Built on DSPy's modular framework, it enables teacher-student optimization where expensive models (GPT-4) generate demonstrations that improve cheaper student models (Llama-2), with configurable cost-performance tradeoffs. Supports multiple retrievers, LM backends (OpenAI and local via TGI), and custom signatures for domain adaptation, with reproducible cached inference across 10K+ label datasets.

449 stars. No commits in the last 6 months.