megagonlabs/holobench

🫧 Code for Holistic Reasoning with Long-Context LMs: A Benchmark for Database Operations on Massive Textual Data (Maekawa*, Iso* et al.; ICLR 2025)

/ 100

Experimental

No commits in the last 6 months.

Stale 6m No Package No Dependents

Maintenance 0 / 25

Adoption 5 / 25

Maturity 9 / 25

Community 6 / 25

Stars

Forks

Language

Python

License

BSD-3-Clause

Category

Last pushed

Feb 25, 2025

Commits (30d)

Get this data via API

curl "https://pt-edge.onrender.com/api/v1/quality/nlp/megagonlabs/holobench"

Open to everyone — 100 requests/day, no key needed. Get a free key for 1,000/day.

Higher-rated alternatives

google/langfun

OO for LLMs

tanaos/artifex

Small Language Model Inference, Fine-Tuning and Observability. No GPU, no labeled data needed.

vulnerability-lookup/VulnTrain

A tool to generate datasets and models based on vulnerabilities descriptions from @Vulnerability-Lookup.

DataScienceUIBK/HintEval

HintEval💡: A Comprehensive Framework for Hint Generation and Evaluation for Questions

microsoft/LMChallenge

A library & tools to evaluate predictive language models.