GAIR-NLP/BeHonest
BeHonest: Benchmarking Honesty in Large Language Models
This project helps evaluate how "honest" large language models (LLMs) are by checking if they know their limits, avoid lying, and respond consistently. You input a list of questions or prompts for an LLM, and it outputs a report detailing the model's honesty score across various behaviors. This is for researchers, developers, or product managers who build or deploy LLMs and need to ensure their models are trustworthy.
No commits in the last 6 months.
Use this if you need to systematically test an LLM's truthfulness, consistency, and ability to admit when it doesn't know an answer, rather than fabricating information.
Not ideal if you're looking for a general-purpose LLM evaluation tool for tasks like summarization or translation, as this focuses specifically on honesty and consistency.
Stars
34
Forks
—
Language
JavaScript
License
—
Category
Last pushed
Aug 15, 2024
Commits (30d)
0
Get this data via API
curl "https://pt-edge.onrender.com/api/v1/quality/llm-tools/GAIR-NLP/BeHonest"
Open to everyone — 100 requests/day, no key needed. Get a free key for 1,000/day.
Featured in
Higher-rated alternatives
sierra-research/tau2-bench
τ²-Bench: Evaluating Conversational Agents in a Dual-Control Environment
xlang-ai/OSWorld
[NeurIPS 2024] OSWorld: Benchmarking Multimodal Agents for Open-Ended Tasks in Real Computer Environments
bigcode-project/bigcodebench
[ICLR'25] BigCodeBench: Benchmarking Code Generation Towards AGI
scicode-bench/SciCode
A benchmark that challenges language models to code solutions for scientific problems
THUDM/AgentBench
A Comprehensive Benchmark to Evaluate LLMs as Agents (ICLR'24)