avnlp/llm-blender
LLM-Blender: Ensembling framework that maximizes LLM performance via pairwise ranking. Employs PairRanker to rank candidates and GenFuser to merge outputs, generating superior responses by combining the diverse strengths of multiple open-source models.
The framework integrates with Haystack RAG pipelines via a custom `LLMBlenderRanker` component, enabling ensemble ranking within retrieval-augmented generation workflows. PairRanker uses cross-attention encoders (RoBERTa) to perform all-pairs candidate comparisons and rank outputs, while GenFuser employs seq2seq fusion to synthesize top-K candidates into enhanced final responses. Evaluation on BillSum and MixInstruct datasets with modern models (Llama-3, Mistral-7B, Phi-3) demonstrates measurable improvements in BERTScore, BARTScore, and BLEURT metrics over individual model outputs.
Stars
36
Forks
3
Language
Python
License
MIT
Category
Last pushed
Dec 20, 2025
Commits (30d)
0
Get this data via API
curl "https://pt-edge.onrender.com/api/v1/quality/transformers/avnlp/llm-blender"
Open to everyone — 100 requests/day, no key needed. Get a free key for 1,000/day.