google-research/rliable

Implements stratified bootstrap confidence intervals and performance profiles to quantify statistical uncertainty in aggregate metrics like Interquartile Mean (IQM), which is more robust to outlier tasks than mean or median. Provides specialized aggregate metrics (IQM, optimality gap, probability of improvement) alongside visualization tools for performance profiles and sample efficiency curves across multi-task benchmarks. Targets RL evaluation on standard suites like Atari, DeepMind Control, and Procgen with APIs for computing interval estimates across algorithm score matrices.

866 stars. No commits in the last 6 months.