Model Confidence Calibration ML Frameworks

Tools and techniques for calibrating neural network confidence scores and probability predictions, including post-hoc methods, metrics, and frameworks. Does NOT include general uncertainty quantification, Bayesian methods, or confidence intervals unrelated to classifier calibration.

There are 21 model confidence calibration frameworks tracked. 2 score above 50 (established tier). The highest-rated is facebookincubator/MCGrad at 57/100 with 18 stars and 215 monthly downloads.

Get all 21 projects as JSON

curl "https://pt-edge.onrender.com/api/v1/datasets/quality?domain=ml-frameworks&subcategory=model-confidence-calibration&limit=20"

Open to everyone — 100 requests/day, no key needed. Get a free key for 1,000/day.

#	Framework	Score	Tier	Stars	Language
1	facebookincubator/MCGrad MCGrad is a scalable and easy-to-use tool for multicalibration. It ensures...	57	Established	18	Jupyter Notebook
2	dholzmueller/probmetrics Post-hoc calibration methods and metrics for classification	56	Established	53	Python
3	gpleiss/temperature_scaling A simple way to calibrate your neural network.	43	Emerging	1,167	Python
4	yfzhang114/Generalization-Causality 关于domain generalization，domain...	37	Emerging	1,238	—
5	DiTEC-project/DiTEC_WDN_dataset This repository contains parameter generation, simulation, and encapsulation...	35	Emerging	9	Python
6	hollance/reliability-diagrams Reliability diagrams visualize whether a classifier model needs calibration	34	Emerging	167	Jupyter Notebook
7	Affirm/splinator Splinator: probabilistic calibration with regression splines	34	Emerging	24	Python
8	VIDA-NYU/pycalibrate pycalibrate is a Python library to visually analyze model calibration in...	32	Emerging	17	Jupyter Notebook
9	uncbiag/LTS Local Temperature Scaling for Probability Calibration	31	Emerging	22	Python
10	lorenzofamiglini/CalFram Calibration Framework for Machine Learning and Deep Learning	30	Emerging	16	Python
11	by-liu/MbLS Code of our method MbLS (Margin-based Label Smoothing) for network...	30	Emerging	50	Python
12	mdca-loss/MDCA-Calibration [CVPR 2022] Official code for the paper: "A Stitch in Time Saves Nine: A...	29	Experimental	33	Python
13	divelab/LECI The implementation of "Joint Learning of Label and Environment Causal...	26	Experimental	22	Python
14	kirill-vish/Beyond-INet Code for experiments for "ConvNet vs Transformer, Supervised vs CLIP: Beyond...	25	Experimental	102	Python
15	WenjianHuang93/h-Calibration h-calibration: post-hoc calibration for deep learning classifier	22	Experimental	28	Python
16	maheeppurohit/epistemic-weight-engine Epistemic Weight Engine (EWE) — A pre-update gating mechanism for...	22	Experimental	—	Python
17	jhuang265/Calibrating-LLMs-with-Label-Smoothing Code to our ICML 2025 Paper "Calibrated Language Models and How to Find Them...	22	Experimental	3	Python
18	aai-institute/kyle A library for calibrating classifiers and computing calibration metrics	20	Experimental	14	Jupyter Notebook
19	jsbaan/calibration-on-disagreement-data Code accompanying the EMNLP 2022 paper "Stop Measuring Calibration When...	17	Experimental	5	Jupyter Notebook
20	MohammadErfan-Jabbari/CNN-Calibration Investigating calibration in CNNs: Temperature scaling for reliable...	15	Experimental	—	HTML
21	martinferianc/noise Investigation of how noise perturbations impact neural network calibration...	13	Experimental	6	Shell