Bias and ratios: reading a number without being fooled — Cours — Académie

The technical core. A ratio should never be taken at face value: you need to know where it comes from, and tell a real bias apart from a statistical mirage.

1The disparity ratio and the small-group trap

The base measure compares, across groups, the rate of the positive class (getting the loan, passing the screen…). The disparity ratio = rate of the most favored group ÷ rate of the least favored. 60% vs 30% → ratio 2.0.

The small-group trap. A dramatic ratio can be carried by a handful of individuals. A rate computed on n = 18 is fragile: two more cases can flip it. Always look at n, never the ratio alone. An extreme max/min on a tiny group is an artifact.

Justifying is not excusing. "That's just what the historical data shows" is not acceptable: history can encode past discrimination. A disparity must be justified (a documented legitimate necessity) or mitigated. And if mappingValidated:false, the bias is "unconfirmed" — you don't sign.

2What to measure, and the notions of fairness

The right rate

You compare, across groups, the rate of the positive class (getting the loan, passing the screen). Not the overall error rate, not income, not headcount — those are characteristics, not the outcome being compared.

Several fairness notions, sometimes incompatible

Demographic parity: equal selection rates across groups.
Equal opportunity: equal true positive rates (not missing the good candidates within a group).

You can't satisfy everything. These notions diverge, and mathematical results show they are incompatible as soon as base rates differ. The expert chooses the relevant notion and justifies it — there's no universal optimum. Choosing isn't the same as being blind: you cross-check several metrics so as not to miss a problem that a single one would mask.

The averaging trap

A high overall accuracy (92%) can mask a failure on a sub-group (60%): average performance says nothing about fairness. Likewise, a balanced overall ratio can hide an intra-group disparity (e.g., among under-30s) — hence examining sub-groups (after checking their headcount).

3Where bias comes from, and how to mitigate it

The sources

Sample not representative of the targeted population.
Labels derived from biased past human decisions (an "evaluation" is not a neutral truth): the model learns the bias.
Proxies: variables correlated with a sensitive attribute that re-encode it.

Masking isn't enough. Removing the sensitive attribute from the dataset doesn't remove the bias: proxies still carry it. "Fairness through unawareness" is insufficient — results must be measured by group (so the attribute must be known, cf. §5).

Families of mitigation

Pre-processing: rebalancing / re-weighting the data.
In-processing: imposing fairness constraints during training.
Post-processing: adjusting decision thresholds by group.

Limit: "mechanically equalizing rates" (forced parity) has perverse effects (reverse discrimination, loss of relevance). You understand the source before correcting.

4Reading a number like an expert

Small sample sizes

A rate on a small group is fragile: it should come with a confidence interval (a wide one), and one should be wary of an extreme ratio that is only an artifact. Compare the robust groups first.

The COMPAS lesson

In the COMPAS case (recidivism), the documented bias was about markedly unequal false positive rates across groups — not overall accuracy, which could appear comparable.

No magic threshold. A real gap on robust groups (e.g., a 1.3 ratio) must be examined: find the cause, then justify it (legitimate factor, non-proxy) or mitigate it, with documentation. It's judgment that decides, not a binary threshold. And never sign on a mappingValidated:false.