EvolvingLMMs-Lab/lmms-eval
One-for-All Multimodal Evaluation Toolkit Across Text, Image, Video, and Audio Tasks
3,883 stars and 9,061 monthly downloads. Used by 1 other package. Actively maintained with 30 commits in the last 30 days. Available on PyPI.
Stars
3,883
Forks
539
Language
Python
License
—
Category
Last pushed
Mar 11, 2026
Monthly downloads
9,061
Commits (30d)
30
Dependencies
52
Reverse dependents
1
Get this data via API
curl "https://pt-edge.onrender.com/api/v1/quality/llm-tools/EvolvingLMMs-Lab/lmms-eval"
Open to everyone — 100 requests/day, no key needed. Get a free key for 1,000/day.
Featured in
Related tools
open-compass/VLMEvalKit
Open-source evaluation toolkit of large multi-modality models (LMMs), support 220+ LMMs, 80+ benchmarks
vibrantlabsai/ragas
Supercharge Your LLM Application Evaluations 🚀
Giskard-AI/giskard-oss
🐢 Open-Source Evaluation & Testing library for LLM Agents
evalplus/evalplus
Rigourous evaluation of LLM-synthesized code - NeurIPS 2023 & COLM 2024
EuroEval/EuroEval
The robust European language model benchmark.