project-miracl/nomiracl
NoMIRACL: A multilingual hallucination evaluation dataset to evaluate LLM robustness in RAG against first-stage retrieval errors on 18 languages.
The dataset includes both relevant and non-relevant query-passage pairs balanced at 50/50, enabling evaluation of whether LLMs can correctly abstain when no relevant information is retrieved. It provides a unified Python interface supporting multiple inference backends (HuggingFace, vLLM, Cohere, OpenAI, NVIDIA, Azure, Anyscale) and includes four customizable prompt templates (vanilla, role-based, repetition, explanation) for systematic robustness testing across techniques.
No commits in the last 6 months. Available on PyPI.
Stars
26
Forks
4
Language
Python
License
Apache-2.0
Category
Last pushed
Nov 29, 2024
Commits (30d)
0
Dependencies
6
Get this data via API
curl "https://pt-edge.onrender.com/api/v1/quality/rag/project-miracl/nomiracl"
Open to everyone — 100 requests/day, no key needed. Get a free key for 1,000/day.
Higher-rated alternatives
onestardao/WFGY
WFGY: open-source reasoning and debugging infrastructure for RAG and AI agents. Includes the...
KRLabsOrg/verbatim-rag
Hallucination-prevention RAG system with verbatim span extraction. Ensures all generated content...
iMoonLab/Hyper-RAG
"Hyper-RAG: Combating LLM Hallucinations using Hypergraph-Driven Retrieval-Augmented Generation"...
frmoretto/clarity-gate
Stop LLMs from hallucinating your guesses as facts. Clarity Gate is a verification protocol for...
chensyCN/LogicRAG
Source code of LogicRAG at AAAI'26.