Data as of Sep 29, 2026 · Based on 3,265 AI responses · See how Parse measures this
CrowS-Pairs is a dataset and accompanying codebase designed to quantify stereotypical biases in masked language models. It contains 1,508 sentence-pair examples across nine bias types (e.g., race/color, gender, religion, age, nationality, disability, appearance, socioeconomic status) where each pair contrasts a stereotype about a historically disadvantaged US group with a minimally edited advantaged-group sentence, and includes annotations. The repository provides Python tooling (including metric.py and a requirements.txt) to measure bias in MLMs and notes reliability concerns raised by Blodgett et al. 2021, with data stored in crows_pairs_anonymized.csv.
Hosted on GitHub
0%No change
of AI answers about CrowS-Pairs and its rivals. Week of Sep 21
“CrowS-Pairs / Stereoset : Measure underlying stereotypes regarding race, gender, and religion using conditional sentence pairs.”
AI mentioned CrowS-Pairs in 0% of answers about CrowS-Pairs and its rivals in the week of Sep 21.
