Week 5: Context-Free Fairness

DSAN 5450: Data Ethics and Policy
Spring 2026, Georgetown University

Jeff Jacobs

jj1088@georgetown.edu

Wednesday, February 11, 2026

Schedule

Today’s Planned Schedule:

	Start	End	Topic
Lecture	6:30pm	7:00pm	Setting the Table: HW1 \(\leadsto\) HW2 →
	7:00pm	7:15pm	Issues with Context-Free Fairness →
	7:15pm	7:30pm	Bringing in Context →
	7:30pm	8:00pm	Similarity-Based Fairness →
Break!	8:00pm	8:10pm
	8:10pm	9:00pm	Causal Fairness Building Blocks →

\[ \DeclareMathOperator*{\argmax}{argmax} \DeclareMathOperator*{\argmin}{argmin} \newcommand{\bigexp}[1]{\exp\mkern-4mu\left[ #1 \right]} \newcommand{\bigexpect}[1]{\mathbb{E}\mkern-4mu \left[ #1 \right]} \newcommand{\definedas}{\overset{\small\text{def}}{=}} \newcommand{\definedalign}{\overset{\phantom{\text{defn}}}{=}} \newcommand{\eqeventual}{\overset{\text{eventually}}{=}} \newcommand{\Err}{\text{Err}} \newcommand{\expect}[1]{\mathbb{E}[#1]} \newcommand{\expectsq}[1]{\mathbb{E}^2[#1]} \newcommand{\fw}[1]{\texttt{#1}} \newcommand{\given}{\mid} \newcommand{\green}[1]{\color{green}{#1}} \newcommand{\heads}{\outcome{heads}} \newcommand{\iid}{\overset{\text{\small{iid}}}{\sim}} \newcommand{\lik}{\mathcal{L}} \newcommand{\loglik}{\ell} \DeclareMathOperator*{\maximize}{maximize} \DeclareMathOperator*{\minimize}{minimize} \newcommand{\mle}{\textsf{ML}} \newcommand{\nimplies}{\;\not\!\!\!\!\implies} \newcommand{\orange}[1]{\color{orange}{#1}} \newcommand{\outcome}[1]{\textsf{#1}} \newcommand{\param}[1]{{\color{purple} #1}} \newcommand{\pgsamplespace}{\{\green{1},\green{2},\green{3},\purp{4},\purp{5},\purp{6}\}} \newcommand{\pedge}[2]{\require{enclose}\enclose{circle}{~{#1}~} \rightarrow \; \enclose{circle}{\kern.01em {#2}~\kern.01em}} \newcommand{\pnode}[1]{\require{enclose}\enclose{circle}{\kern.1em {#1} \kern.1em}} \newcommand{\ponode}[1]{\require{enclose}\enclose{box}[background=lightgray]{{#1}}} \newcommand{\pnodesp}[1]{\require{enclose}\enclose{circle}{~{#1}~}} \newcommand{\purp}[1]{\color{purple}{#1}} \newcommand{\sign}{\text{Sign}} \newcommand{\spacecap}{\; \cap \;} \newcommand{\spacewedge}{\; \wedge \;} \newcommand{\tails}{\outcome{tails}} \newcommand{\Var}[1]{\text{Var}[#1]} \newcommand{\bigVar}[1]{\text{Var}\mkern-4mu \left[ #1 \right]} \]

Context-Free
\(\rightarrow\) Context-Sensitive

First: Immanent Critique

• “Trancendental” Critique: Your assumptions are stupid! You should use these ones instead (then we get the result I want 😈)
• Immanent (\(\neq\) Imminent) Critique: Let’s work within your set of assumptions, and derive something undesirable
• \(\neq\) Proof by Contradiction! More “Proof by Pragmatism”
• Think of geometry: for modeling 2D table, “effective” derivation of results from Euclid’s 5th; for modeling 4D space-time, “effective” derivation of results from Riemannian geometry

Our Case: For Reflective Equilibrium!

Steamboat William (1803), Colorized

• Assume brogrammer Fairness™ Definitions
• See what results we get
• Do they seem 👍/👎 intuitively? And/or on feasibly-intersubjective criteria?
• (Secretly you already did this with fairness through unawareness / anti-classification: dropping race \(\overset{?}{=}\) no racism?)

ProPublica vs. Northpointe

• Example with 1000s of books, papers, discussions around it!
  • 🚩: obsession with one example may conceal wider range of issues
• Northpointe created a ML algorithm called COMPAS, used by court systems all over the US to predict “risk” of arrestees
• In 2016, ProPublica published investigative report documenting COMPAS’s racial discrimination, in violating equal error rates between black and white arrestees
• Northpointe responded that COMPAS does not discriminate, as it satisfies calibration
• People have argued about who is “right” for 8 years, with some progress, but… not a lot

COMPAS Overview

When defendants are booked into jail in Broward County, Florida, they are asked to respond to a COMPAS questionnaire with 137 questions, including “Was one of your parents ever sent to jail or prison?,” “How many of your friends/acquaintances are taking drugs illegally?,” and “How often did you get into fights at school?” Arrestees are also asked to agree or disagree with the statements “A hungry person has the right to steal” and “If people make me angry or I lose my temper, I can be dangerous.”

Answers are fed into the COMPAS model, which generates an individual risk score reported in three buckets: “low risk” (1 to 4), “medium risk” (5 to 7), and “high risk” (8 to 10).

COMPAS Overview

ProPublica accused COMPAS of racism: “There’s software used across the country to predict future criminals. And it’s biased against blacks,” read the subheading on the article.

ProPublica found that COMPAS’s error rates—the rate at which the model got it wrong—were unequal across racial groups. COMPAS’s predictions were more likely to incorrectly label African Americans as high risk and more likely to incorrectly label white Americans as low risk.

“In the criminal justice context,” said Julia Angwin, coauthor of the ProPublica article, “false findings can have far-reaching effects on the lives of the people charged with crimes.”

The “Competing” Fairness Definitions

• Time for a (long) demo!

References