minnesotanlp/Quantifying-Annotation-Disagreement

Official implementation of Wan et al's paper "Everyone's Voice Matters: Quantifying Annotation Disagreement Using Demographic Information" (AAAI 2023)

15
/ 100
Experimental

No commits in the last 6 months.

No License Stale 6m No Package No Dependents
Maintenance 0 / 25
Adoption 4 / 25
Maturity 1 / 25
Community 10 / 25

How are scores calculated?

Stars

6

Forks

1

Language

Jupyter Notebook

License

—

Category

bias-measurement-evaluation

Last pushed

Jan 17, 2023

Commits (30d)

0

GitHub
Bias Measurement Evaluation · 37 tools

Get this data via API

curl "https://pt-edge.onrender.com/api/v1/quality/nlp/minnesotanlp/Quantifying-Annotation-Disagreement"

Open to everyone — 100 requests/day, no key needed. Get a free key for 1,000/day.

Higher-rated alternatives

dccuchile/wefe

WEFE: The Word Embeddings Fairness Evaluation Framework. WEFE is a framework that standardizes...

46

dreji18/Fairness-in-AI

Detecting Bias and ensuring Fairness in AI solutions

36

amazon-science/bold

Dataset associated with "BOLD: Dataset and Metrics for Measuring Biases in Open-Ended Language...

35

dhfbk/variationist

Variationist: Exploring Multifaceted Variation and Bias in Written Language Data (ACL 2024 demo track)

33

soarsmu/BiasFinder

BiasFinder | IEEE TSE | Metamorphic Test Generation to Uncover Bias for Sentiment Analysis Systems

30

Explore NLP Tools

All categories Trending NLP directory Insights