| read_time | dark_mode | male | age | hours | |
|---|---|---|---|---|---|
| 0 | 14.4 | False | 0 | 43.0 | 65.6 |
| 1 | 15.4 | False | 1 | 55.0 | 125.4 |
| 2 | 20.9 | True | 0 | 23.0 | 642.6 |
| 3 | 20.0 | False | 0 | 41.0 | 129.1 |
| 4 | 21.5 | True | 0 | 29.0 | 190.2 |
Week 10: Doubly-Robust Estimation, Text-as-Data
DSAN 5650: Causal Inference for Computational Social Science
Summer 2026, Georgetown University
Wednesday, July 22, 2026
Final Project Huddle
🥳 You’re doing great 🥳
- Reminder: The two options are not mutually exclusive: let the research question drive your trajectory!
- Emerging Theme 1 What is required to take [thing people already study], push it into the realm of causality?
- I can measure change in text property (sentiment, topic) before and after event… how do I know event caused change?
- Emerging Theme 2 I have an outcome (“puzzle”) \(Y\), plus a treatment \(T\) that I think causes it… How do I concretely “connect the dots” from \(T\) to \(Y\)?
- Ex: I think introduction of Fox News Channel caused increased polarization…
- In Both Cases Start project with the associational connections, then explore possibilities of (a) controlling for forks/pipes, (b) existence of colliders, (c) if there’s some “exogenous variation” you can exploit (stand-in for coin flip)
Double Robustness
- Propensity Score Weighting seems so much easier than all the hard work of modeling… why can’t we just propensity score all the things and be done with it!?
- By using doubly-robust estimation methods, you can:
- Carefully develop a covariate adjustment strategy (then use e.g. regression),
- Carefully develop a propensity score strategy, and then
- Be only as wrong as the least-wrong of and !!
With doubly-robust estimation, as long as the answer is either True or False you’re good!
Doubly-Robust Estimation
…So, is this a causal effect? Does dark theme cause users to spend less time reading?
What Does the Data Look Like?
Control-Treatment Balance
Enter Uber’s causal inference library: causalml
And then WeightIt to generate a “love plot”:
Model 1: Propensity Score
Code
def estimate_e(df, X, D, model_e):
e = model_e.fit(df[X], df[D]).predict_proba(df[X])[:,1]
return e
user_df['e'] = estimate_e(user_df, X, "dark_mode", LogisticRegression())
fig, ax = plt.subplots(figsize=(7,2.75))
sns.kdeplot(
x='e', hue='dark_mode', data=user_df,
# bins=30,
#stat='density',
common_norm=False,
fill=True,
ax=ax
);
ax.set_xlabel("$e(X)$");
Double-Robustness to the Rescue!
Wrong regression model:
Code
def compare_estimators(X_e, X_mu, D, y, seed):
df = generate_data(seed=seed)
e = estimate_e(df, X_e, D, LogisticRegression())
mu0, mu1 = estimate_mu(df, X_mu, D, y, LinearRegression())
slearn = mu1 - mu0
ipw = (df[D] / e - (1-df[D]) / (1-e)) * df[y]
aipw = slearn + df[D] / e * (df[y] - mu1) - (1-df[D]) / (1-e) * (df[y] - mu0)
return np.mean((slearn, ipw, aipw), axis=1)
def simulate_estimators(X_e, X_mu, D, y):
r = Parallel(n_jobs=8)(delayed(compare_estimators)(X_e, X_mu, D, y, i) for i in range(100))
df_tau = pd.DataFrame(r, columns=['S-learn', 'IPW', 'AIPW'])
return df_tau
# The actual plots
fig, ax = plt.subplots(figsize=(4,3.5))
wrong_reg_df = simulate_estimators(
X_e=['male', 'age'], X_mu=['hours'], D="dark_mode", y="read_time"
)
wrong_reg_plot = sns.boxplot(
data=pd.melt(wrong_reg_df), x='variable', y='value', hue='variable',
ax=ax,
linewidth=2
);
wrong_reg_plot.set(
title="Distribution of $\hat τ$", xlabel='', ylabel=''
);
ax.axhline(2, c='r', ls=':');
Wrong propensity score model:
Code
fig, ax = plt.subplots(figsize=(4, 3.5))
wrong_ps_df = simulate_estimators(
['age'], ['male', 'hours'], D="dark_mode", y="read_time"
)
wrong_ps_plot = sns.boxplot(
data=pd.melt(wrong_ps_df), x='variable', y='value', hue='variable',
ax=ax,
linewidth=2
);
ax.set_title("Distribution of $\hat τ$");
ax.axhline(2, c='r', ls=':');
plt.show()
Instrumental Variables
If randomization works to obtain causal effects…
…Find something random in the causal system, use e.g. matching to “force” the same scenario!
General form: \(\text{Effect}(D \rightarrow Y) = \frac{\text{Effect}(Z \rightarrow Y)}{\text{Effect}(Z \rightarrow D)}\) (Try “plugging in” \(Z\) = Coin Flip!)
\[ \beta_{\text{IV}}^{\text{Wald}} = \frac{ \mathbb{E}[Y_i \mid Z_i = 1] - \mathbb{E}[Y_i \mid Z_i = 0] }{ \mathbb{E}[D_i \mid Z_i = 1] - \mathbb{E}[D_i \mid Z_i = 0] }, \; \beta_{\text{IV}} = \frac{\text{Cov}[Y, Z]}{\text{Cov}[D,Z]} \]
The Solution? Sample Splitting!
- Machine learning noticed this long ago: the goal is a model that generalizes, not memorizes!
Topic Models as PGMs
From Blei (2012)
…Unlocks a world of social modeling through text!
Influence Over Time
From Barron et al. (2018)
Textual Influence Over Time
