Fairness Bias Auditing ML Frameworks

Tools and frameworks for measuring, auditing, and mitigating bias and fairness issues in ML models and datasets. Includes bias detection metrics, fairness evaluation pipelines, and debiasing algorithms. Does NOT include general ML ethics resources, responsible AI governance frameworks, or fairness theory papers without implementation code.

There are 55 fairness bias auditing frameworks tracked. 2 score above 70 (verified tier). The highest-rated is fairlearn/fairlearn at 91/100 with 2,213 stars and 170,696 monthly downloads. 2 of the top 10 are actively maintained.

Get all 55 projects as JSON

curl "https://pt-edge.onrender.com/api/v1/datasets/quality?domain=ml-frameworks&subcategory=fairness-bias-auditing&limit=20"

Open to everyone — 100 requests/day, no key needed. Get a free key for 1,000/day.

#	Framework	Score	Tier	Stars	Language
1	fairlearn/fairlearn A Python package to assess and improve fairness of machine learning models.	91	Verified	2,213	Python
2	Trusted-AI/AIF360 A comprehensive set of fairness metrics for datasets and machine learning...	79	Verified	2,763	Python
3	microsoft/responsible-ai-toolbox Responsible AI Toolbox is a suite of tools providing model and data...	64	Established	1,737	TypeScript
4	holistic-ai/holisticai This is an open-source tool to assess and improve the trustworthiness of AI systems.	64	Established	104	Jupyter Notebook
5	EFS-OpenSource/Thetis Service to examine data processing pipelines (e.g., machine learning or deep...	50	Established	6	Python
6	datamllab/awesome-fairness-in-ai A curated list of awesome Fairness in AI resources	49	Emerging	332	—
7	IBM/inFairness PyTorch package to train and audit ML models for Individual Fairness	49	Emerging	66	Python
8	linkedin/LiFT The LinkedIn Fairness Toolkit (LiFT) is a Scala/Spark library that enables...	47	Emerging	173	Scala
9	DataResponsibly/Virny An in-depth performance profiling library for machine learning models	46	Emerging	17	Python
10	ResponsiblyAI/responsibly Toolkit for Auditing and Mitigating Bias and Fairness of Machine Learning Systems 🔎🤖🧰	44	Emerging	100	Python
11	debiai/DebiAI Bias detection and contextual evaluation tool for your AI projects	42	Emerging	30	Vue
12	biaslyze-dev/biaslyze The NLP Bias Identification Toolkit	42	Emerging	39	Jupyter Notebook
13	EqualityAI/EqualityML Evidence-based tools and community collaboration to end algorithmic bias,...	41	Emerging	35	Jupyter Notebook
14	yongkaiwu/FairAI This is a collection of papers and other resources related to fairness.	41	Emerging	95	HTML
15	Libr-AI/fairlib A framework for assessing and improving classification fairness.	40	Emerging	33	Jupyter Notebook
16	pliang279/LM_bias [ICML 2021] Towards Understanding and Mitigating Social Biases in Language Models	39	Emerging	61	Python
17	xuetf/KDD_CUP_2020_Debiasing_Rush Solution to the Debiasing Track of KDD CUP 2020	39	Emerging	160	Jupyter Notebook
18	cylynx/verifyml Open-source toolkit to help companies implement responsible AI workflows.	39	Emerging	23	Python
19	microsoft/responsible-ai-toolbox-mitigations Python library for implementing Responsible AI mitigations.	37	Emerging	69	Jupyter Notebook
20	ys-zong/MEDFAIR [ICLR 2023 spotlight] MEDFAIR: Benchmarking Fairness for Medical Imaging	35	Emerging	73	Python
21	DetoxAI/detoxai Python toolkit for debiasing neural networks in image classification tasks	35	Emerging	81	Python
22	brandeis-machine-learning/awesome-ml-fairness Papers and online resources related to machine learning fairness	35	Emerging	75	—
23	radames/Machine-Learning-Ethics-References List of references about Machine Learning bias and ethics	33	Emerging	64	—
24	adhit-r/fairmind Ethical AI Governance Platform \| Bias Detection \| Compliance \| Fairness...	32	Emerging	7	Python
25	Ali-Alameer/AI_fairness This GitHub repository offers resources to create fair and unbiased AI...	32	Emerging	14	Jupyter Notebook
26	romanlutz/ResponsibleAI A collection of news articles, books, and papers on Responsible AI cases....	31	Emerging	70	—
27	mirianfsilva/ai-fairness Notes, references and materials on AI Fairness that I found useful and...	30	Emerging	9	—
28	aman179102/trust-aware A trust-aware, human-in-the-loop AI decision system that knows when not to...	30	Emerging	9	Python
29	JMitnik/FacialDebiasing Debiasing: Mitigating Algorithmic Facial Bias repository	29	Experimental	7	Jupyter Notebook
30	guldoganozgur/ei_fairness Code for the paper "Equal Improvability: A New Fairness Notion Considering...	28	Experimental	7	Jupyter Notebook
31	nick-jhlee/fair-manifold-pca Fast and Efficient MMD-based Fair PCA via Optimization over Stiefel Manifold...	28	Experimental	11	C++
32	aida-ugent/fairret A fairness library in PyTorch.	27	Experimental	32	Python
33	uiuctml/fair-classification Post-processing for fair classification	26	Experimental	16	Jupyter Notebook
34	aimms/bias-in-ai Shows how machine learning bias can be illustrated by creating an AIMMS...	23	Experimental	1	Python
35	aweawdsdwaofr/agorai_package 🤝 Build fair AI systems with AgorAI, a Python library that leverages...	22	Experimental	—	Python
36	Jatin-sehgal7060/Cognitivelens-AI-Human-Comparison 🤖 Analyze and compare human and AI decision-making to enhance trust,...	22	Experimental	—	Python
37	yuji-roh/fairbatch FairBatch: Batch Selection for Model Fairness (ICLR 2021)	22	Experimental	19	Python
38	Trustworthy-ML-Lab/Training_Trustworthy_LRM_with_Refine A new training framework for Trustworthy Large Reasoning Models	22	Experimental	4	Python
39	MaxPaez/fairness-analysis-pima 📊 Analyze fairness in Machine Learning models using the Pima Diabetes...	22	Experimental	—	Python
40	nmaudet/fairdiv-indivisible-items A collection of simple algorithms for fair division of indivisible items	21	Experimental	3	Jupyter Notebook
41	Doleus/doleus Build confidence in your AI with systematic slice-based testing	20	Experimental	11	Python
42	ArpitaRandive/Fairness-Aware-Reinforcement-Learning-Under-Distribution-Shift This project simulates a public resource allocation system inspired by...	19	Experimental	—	Python
43	FreakyF/Neural-Injustice Experimental research platform for biometric correlation and algorithmic bias.	19	Experimental	—	TypeScript
44	SayMyName-BiasNaming/samyna-tmlr Source code supporting the paper "Say My Name: a Model's Bias Discovery Framework"	16	Experimental	1	—
45	ValentinLafargue/FairnessDetails Fairness is in the details : an auditing pipeline for face datasets	15	Experimental	—	—
46	emanzahrashah/hiring-bias-fairness-analysis An end-to-end data science project analyzing bias and fairness in an...	14	Experimental	—	Jupyter Notebook
47	Shaw1011/AI-Ethics-Auditor Production-grade platform for auditing AI/ML models for fairness,...	14	Experimental	—	Python
48	djvill/SLAC-Fairness Tools to assess fairness and mitigate unfairness in sociolinguistic auto-coding	13	Experimental	—	HTML
49	jmdvinodjmd/SSB_MLHC Official code for the paper “Sample Selection Bias in Machine Learning for...	13	Experimental	2	Jupyter Notebook
50	Aqib121201/WohnFair-cryptographically-auditable-housing-allocation-system α-fair, cryptographically auditable housing allocation system — reproducible...	13	Experimental	2	Go
51	Tonyhrule/Queer-Bias-LLMs This project implements a collaborative agent pipeline to detect and reduce...	12	Experimental	3	Python
52	DIDSR/bias.myti.report A visualization tool to facilitate the comparison of user-implemented bias...	12	Experimental	1	Jupyter Notebook
53	messihjlee/RM-IAT Replication Materials for "Implicit Bias-Like Patterns in Reasoning Models"	12	Experimental	1	Jupyter Notebook
54	CompliancePal/modelcard-action Validate the model card document in a GitHub action	11	Experimental	2	TypeScript
55	Team-Obisidan/Biases-im-Machine-Learning This German scientific poster (developed as HTML5/React web app) shows the...	10	Experimental	1	JavaScript

Comparisons in this category

fairlearn and AIF360 (91 vs 79) AIF360 and awesome-fairness-in-ai (79 vs 49) AIF360 and responsibly (79 vs 44) AIF360 and FairAI (79 vs 41) AIF360 and AI_fairness (79 vs 32) AIF360 and fairmind (79 vs 32) fairlearn and inFairness (91 vs 49) fairlearn and fairlib (91 vs 40) fairlearn and AI_fairness (91 vs 32) fairlearn and fairmind (91 vs 32)