Week 4: Fairness in AI
DSAN 5450: Data Ethics and Policy
Spring 2024, Georgetown University
\[ \newcommand{\nimplies}{\;\not\!\!\!\!\implies} \]
Week 3 Recap
“Repetition is the mother of perfection” - Dwayne Michael “Lil Wayne” Carter, Jr.
Baby Steps: A Real-World Confusion Matrix
| Labeled Low-Risk | Labeled High-Risk | |
|---|---|---|
| Didn’t Do More Crimes | True Negative | False Positive |
| Did More Crimes | False Negative | True Positive |
- What kinds of causal connections and/or feedback loops might there be between our decision variable (low vs. high risk) and our outcome variable (did vs. didn’t do more crimes)
- What types of policy implications might this process have, after it “runs” for several “iterations”?
- Why might some segments of society, with some shared ethical framework(s), weigh the “costs” of false negatives and false positives differently from other segments of society with different shared ethical framework(s)?
- (Non-rhetorical questions!)
Categories of Fairness Criteria
Roughly, approaches to fairness/bias in AI can be categorized as follows:
- Single-Threshold Fairness
- Equal Prediction
- Equal Decision
- Fairness via Similarity Metric(s)
- Causal Definitions
- [Today] Context-Free Fairness: Easier to grasp from CS/data science perspective; rooted in “language” of Machine Learning (you already know much of it, given DSAN 5000!)
- But easy-to-grasp notion \(\neq\) “good” notion!
- Your job: push yourself to (a) consider what is getting left out of the context-free definitions, and (b) the loopholes that are thus introduced into them, whereby people/computers can discriminate while remaining “technically fair”
Laws: Often Perfectly “Technically Fair”
Ah, la majestueuse égalité des lois, qui interdit au riche comme au pauvre de coucher sous les ponts, de mendier dans les rues et de voler du pain!
(Ah, the majestic equality of the law, which prohibits rich and poor alike from sleeping under bridges, begging in the streets, and stealing loaves of bread!)
Anatole France, Le Lys Rouge (France 1894)
(No) Fairness Through Unawareness
- HW1: Using tiny sample (\(N < 10K\)) of Florida voter registrations… RandomForestClassifier (default settings, no hyperparameter tuning, no cross-validation, no ensembling with other methods) will predict self-reported race with \(>90\%\) accuracy (in balanced sample) from just surname and county of residence
- Can reach \(70\text{-}75\%\) with just surname or just county of residence
- Also in HW1: Facebook ad matching service provides over 1,000 different user attributes for (micro)targeting
To Make It Even More Concrete…
- Bloomberg analysis of neighborhoods with same-day delivery from Amazon:
Last One I Promise
Last One I Promise
We Can Do (A Bit) Better…
- Use random variables to model inferences made by an algorithm (or a human!)
- \(\implies\) fairness by statistically equalizing loan rejections, error rate, etc. between groups
- Obvious societal drawback: equality does not ameliorate the effects of past injustices (see: police contact vs. trust-in-government plot from last week)
- This one we saw coming, given “context-free” nature!
- Less obvious mathematical drawback: impossibility results (because algebra 😳)
- Roughly: can’t satisfy [more than] two statistical fairness criteria at once; similar to how setting \(\Pr(X) = p\) also determines \(\Pr(\text{not }X) = 1 - p\), or how plugging \(x = 3\) into \(x + y = 5\) leaves only one possibility \(y = 2\)
- BUT, “impossibility” \(\neq\) impossibility: (a) one criteria may be “all you need” in given setting; (b) can derive more robust measures by “relaxing” confusion-matrix measures
Definitions and (Impossibility) Results
(tldr:)
- We have information \(X_i\) about person \(i\), and
- We’re trying to predict a binary outcome \(Y_i\) involving \(i\).
- So, we use ML to learn a risk function \(r: \mathcal{R}_{X_i} \rightarrow \mathbb{R}\), then
- Use this to make a binary decision \(\widehat{Y}_i = \mathbf{1}[r(X_i) > t]\)
Protected/Sensitive Attributes
- Standard across the literature: Random Variable \(A_i\) “encoding” membership in protected/sensitive group. In HW1, for example:
\[ A_i = \begin{cases} 0 &\text{if }i\text{ self-reported ``white''} \\ 1 &\text{if }i\text{ self-reported ``black''} \end{cases} \]
Notice: choice of mapping into \(\{0, 1\}\) here non-arbitrary!
We want our models/criteria to be descriptively but also normatively robust; e.g.:
If (antecedent I hold, though majority in US do not) one believes that ending (much less repairing) centuries of unrelenting white supremacist violence here might require asymmetric race-based policies,
Then our model should allow different normative labels and differential weights on
\[ \begin{align*} \Delta &= (\text{Fairness} \mid A = 1) - (\text{Fairness} \mid A = 0) \\ \nabla &= (\text{Fairness} \mid A = 0) - (\text{Fairness} \mid A = 1) \end{align*} \]
despite the descriptive fact that \(\Delta = -\nabla\).
Where Descriptive and Normative Become Intertwined
- Allowing this asymmetry is precisely what enables bring descriptive facts to bear on normative concerns!
- Mathematically we can always “flip” the mapping from racial labels into \(\{0, 1\}\)…
- But this (in a precise, mathematical sense: namely, isomorphism) implies that we’re treating racial categorization as the same type of phenomenon as driving on left or right side of road (see: prev slides on why we make the descriptive vs. normative distinction)
- (See also: Sweden’s Dagen H!)
“Fairness” Through Equalized Positive Rates (EPR)
\[ \boxed{\Pr(D = 1 \mid A = 0) = \Pr(D = 1 \mid A = 1)} \]
- This works specifically for discrete, binary-valued categories
- For general attributes (whether discrete or continuous!), generalizes to:
\[ \boxed{D \perp A} \iff \Pr(D = d, A = a) = \Pr(D = d)\Pr(A = a) \]
Imagine you learn that a person received a scholarship (\(D = 1\)); [with equalized positive rates], this fact would give you no knowledge about the race (or sex, or class, as desired) \(A\) of the individual in question. (DeDeo 2016)
Achieving Equalized Positive Rates
The good news: if we want this, there is a closed-form solution: take your datapoints \(X_i\) and re-weigh each point to obtain \(\widetilde{X}_i = w_iX_i\), where
\[ w_i = \frac{\Pr(Y_i = 1)}{\Pr(Y_i = 1 \mid A_i = 1)} \]
and use derived dataset \(\widetilde{X}_i\) to learn \(r(X)\) (via ML algorithm)… Why does this work?
Let \(\mathcal{X}_{\text{fair}}\) be the set of all possible reweighted versions of \(X_i\) ensuring \(Y_i \perp A_i\). Then
\[ \widetilde{X}_i = \min_{X_i' \in \mathcal{X}_{\text{fair}}}\textsf{distance}(X_i', X_i) = \min_{X_i' \in \mathcal{X}_{\text{fair}}}\underbrace{KL(X_i' \| X_i)}_{\text{Relative entropy!}} \]
- The bad news: nobody in the fairness in AI community read DeDeo (2016), which proves this using information theory? Idk. It has a total of 22 citations 😐
“Fairness” Through Equalized Error Rates
Equalized positive rates didn’t take outcomes \(Y_i\) into account…
- (Even if \(A_i = 1 \Rightarrow Y_i = 1, A_i = 0 \Rightarrow Y_i = 0\), we’d have to choose \(\widehat{Y}_i = c\))
This time, we consider the outcome \(Y\) that
Equalized False Positive Rate (EFPR):
\[ \Pr(D = 1 \mid Y = 0, A = 0) = \Pr(D = 1 \mid Y = 0, A = 1) \]
- Equalized False Negative Rate (EFNR):
\[ \Pr(D = 0 \mid Y = 1, A = 0) = \Pr(D = 0 \mid Y = 1, A = 1) \]
- For general (non-binary) attributes: \((D \perp A) \mid Y\):
\[ \Pr(D = d, A = a \mid Y = y) = \Pr(D = d \mid Y = y)\Pr(A = a \mid Y = y) \]
⚠️ LESS EQUATIONS PLEASE! 😤
- Depending on your background and/or learning style (say, visual vs. auditory), you may be able to look at equations on previous two slides and “see” what they’re “saying”
- If your brain works similarly to mine, however, your eyes glazed over, you began dissociating, planning an escape route, etc., the moment \(> 2\) variables appeared
- If you’re in the latter group, welcome to the cult of Probabilistic Graphical Models 😈
- Our first measure that 🥳🎉matches a principle of justice in society!!!🕺🪩
- Blackstone’s Ratio: “It is better that ten guilty persons escape, than that one innocent suffers.” (Blackstone 1769)
- (…break time!)
Back to Equalized Error Rates
- Blackstone’s Ratio: “It is better that ten guilty persons escape, than that one innocent suffers.” (Blackstone 1769)
- Mathematically \(\Rightarrow \text{Cost}(FPR) = 10\cdot \text{Cost}(FNR)\)
- Legally \(\Rightarrow\) beyond reasonable doubt standard for conviction
- EFPR \(\iff\) rates of false conviction should be the same for everyone, including members of different racial groups.
- Violated when black people are disproportionately likely to be incorrectly convicted, as if a lower evidentiary standard were applied to black people.
One Final Context-Free Criterion: Calibration
- A risk function \(r(X)\) is calibrated if
\[ \Pr(Y = 1 \mid r(X) = v_r) = v_r \]
- (Sweeping a lot of details under the rug), I see this one as: the risk function “tracks” real-world probabilities
- Then, \(r(X)\) is calibrated by group if
\[ \Pr(Y = y \mid r(X) = v_r, A = a) = v_r \]
Impossibility Results
- tldr: We cannot possibly achieve all three of equalized positive rates (often also termed “anti-classification”), classification parity, and calibration (regardless of base rates)
- More alarmingly: We can’t even achieve both classification parity and calibration, except in the special case of equal base rates
“Impossibility” vs. Impossibility
- Sometimes “impossibility results” are, for all intents and purposes, mathematical curiosities: often there’s some pragmatic way of getting around them
- Example: “Arrow’s Impossibility Theorem”
- [In theory] It is mathematically impossible to aggregate individual preferences into societal preferences
- [The catch] True only if people are restricted to ordinal preferences: “I prefer \(x\) to \(y\).” No more information allowed
- [The way around it] Allow people to indicate the magnitude of their preferences: “I prefer \(x\) 5 times more than \(y\)”
- In this case, though, there are direct and (often) unavoidable real-world barriers that fairness impossibility imposes 😕
Arrow’s Impossibility Theorem
- Aziza, Bogdan, and Charles are competing in a fitness test with four events. Goal: determine who is most fit overall
| Run | Jump | Hurdle | Weights | |
|---|---|---|---|---|
| Aziza | 10.1” | 6.0’ | 40” | 150 lb |
| Bogdan | 9.2” | 5.9’ | 42” | 140 lb |
| Charles | 10.0” | 6.1’ | 39” | 145 lb |
- We can rank unambiguously on individual events: Jump: Charles \(\succ_J\) Aziza \(\succ_J\) Bogdan
- Now, axioms for aggregation:
- \(\text{WP}\) (Weak Pareto Optimality): if \(x \succ_i y\) for all events \(i\), \(x \succ y\)
- \(\text{IIA}\) (Independence of Irrelevant Alternatives): If a fourth competitor enters, but Aziza and Bogdan still have the same relative standing on all events, their relative standing overall should not change
- Long story short: only aggregation that can satisfy these is “dictatorship”: choose one event, give it importance of 100%, the rest have importance 0% 😰
ProPublica vs. Northpointe
- This is… an example with 1000s of books and papers and discussions around it! (A red flag 🚩, since, obsession with one example may conceal much wider range of issues!)
- But, tldr, Northpointe created a ML algorithm called COMPAS, used by court systems all over the US to predict “risk” of arrestees
- In 2016, ProPublica published results from an investigative report documenting COMPAS’s racial discrimination, in the form of violating equal error rates between black and white arrestees
- Northpointe responded that COMPAS does not discriminate, as it satisfies calibration
- People have argued about who is “right” for 8 years, with some progress, but… not a lot
So… What Do We Do?
- One option: argue about which of the two definitions is “better” for the next 100 years (what is the best way to give food to the poor?)
It appears to reveal an unfortunate but inexorable fact about our world: we must choose between two intuitively appealing ways to understand fairness in ML. Many scholars have done just that, defending either ProPublica’s or Northpointe’s definitions against what they see as the misguided alternative. (Simons 2023)
- Another option: study and then work to ameliorate the social conditions which force us into this realm of mathematical impossibility (why do the poor have no food?)
The impossibility result is about much more than math. [It occurs because] the underlying outcome is distributed unevenly in society. This is a fact about society, not mathematics, and requires engaging with a complex, checkered history of systemic racism in the US. Predicting an outcome whose distribution is shaped by this history requires tradeoffs because the inequalities and injustices are encoded in data—in this case, because America has criminalized Blackness for as long as America has existed.
Why Not Both??
- On the one hand: yes, both! On the other hand: fallacy of the “middle ground”
- We’re back at descriptive vs. normative:
- Descriptively, given 100 values \(v_1, \ldots, v_{100}\), their mean may be a good way to summarize, if we have to choose a single number
- But, normatively, imagine that these are opinions that people hold about fairness.
- Now, if it’s the US South in 1860 and \(v_i\) represents person \(i\)’s approval of slavery, from a sample of 100 people, then approx. 97 of the \(v_i\)’s are “does not disapprove” (Rousey 2001) — in this case, normatively, is the mean \(0.97\) the “correct” answer?
- We have another case where, like the “grass is green” vs. “grass ought to be green” example, we cannot just “import” our logical/mathematical tools from the former to solve the latter! (However: this does not mean they are useless! This is the fallacy of the excluded middle, sort of the opposite of the fallacy of the middle ground)
- This is why we have ethical frameworks in the first place! Going back to Rawls: “97% of Americans think black people shouldn’t have rights” \(\nimplies\)“black people shouldn’t have rights”, since rights are a primary good