vectara/hallucination-leaderboard
Leaderboard Comparing LLM Performance at Producing Hallucinations when Summarizing Short Documents
Leverages Vectara's HHEM-2.3 hallucination evaluation model to score LLMs across a curated, non-public dataset of 7700+ documents spanning news, science, medicine, and legal domains with varying complexity (50–24K words). Rankings track hallucination rate, factual consistency, answer rate, and summary length, with results updated regularly as models evolve. Integrates with Hugging Face for interactive exploration and provides an open-source HHEM-2.1 variant for reproducible research.
3,122 stars. Actively maintained with 9 commits in the last 30 days.
Stars
3,122
Forks
96
Language
Python
License
Apache-2.0
Category
Last pushed
Mar 10, 2026
Commits (30d)
9
Get this data via API
curl "https://pt-edge.onrender.com/api/v1/quality/llm-tools/vectara/hallucination-leaderboard"
Open to everyone — 100 requests/day, no key needed. Get a free key for 1,000/day.
Related tools
PKU-YuanGroup/Hallucination-Attack
Attack to induce LLMs within hallucinations
Amirhosein-gh98/Gnosis
Can LLMs Predict Their Own Failures? Self-Awareness via Internal Circuits
NishilBalar/Awesome-LVLM-Hallucination
up-to-date curated list of state-of-the-art Large vision language models hallucinations...
MemTensor/HaluMem
HaluMem is the first operation level hallucination evaluation benchmark tailored to agent memory systems.
intuit/sac3
Official repo for SAC3: Reliable Hallucination Detection in Black-Box Language Models via...