OpenBMB/UltraEval-Audio

Provides unified evaluation across 34 authoritative benchmarks spanning speech understanding and generation tasks (ASR, TTS, AST, audio codecs) in 10 languages and 12 task categories. Features isolated inference via IPC to prevent dependency conflicts, automated dataset acquisition and metric binding, and GPU-accelerated parallel evaluation with resume-from-checkpoint capabilities. Integrates with popular audio foundation models (Qwen3-Omni, GLM-4-Voice, VoxCPM, CosyVoice) through standardized replication documentation and one-click evaluation commands.

281 stars.