rungalileo/agent-leaderboard

Combines synthetic dataset generation with multi-turn conversation simulation across five enterprise domains (banking, healthcare, insurance, telecom, investment) to evaluate agents on task completion and tool-calling accuracy. Introduces Action Completion (AC) and Tool Selection Quality (TSQ) metrics that measure real-world effectiveness—whether agents accomplish all user goals with correct tool selection—rather than isolated capability scores. Hosts a live leaderboard on Hugging Face comparing 17+ models and provides open datasets for reproducible agent evaluation in production scenarios.

217 stars.