Seminar: Kausalanalyse

Ein Einsteiger-Seminar für Studierende der Politikwissenschaft

Dr. Paul C. Bauer
mail@paulcbauer.de
@p_c_bauer
www.paulcbauer.de
License for original material Licence

Updated: Jul 17, 2024

Session 2
Statistical foundations: Measurement, variables, data (distributions) and models

Univariate distribution of trust (2006)
0	1	2	3	4	5	6	7	8	9	10
303	42	172	270	369	1281	853	1344	1295	353	356

Joint distribution of trust and victimization (2006, N = 6633)
	0	1	2	3	4	5	6	7	8	9	10
no victim	259	36	135	214	320	1142	782	1228	1193	326	331
victim	44	6	37	56	48	139	70	114	101	27	25

Population & sampling method (2)

Sampling: Select subset of units from population to estimate characteristics of that population
Steps: Researcher (we)…
- …define (target) population.
- …create a sampling frame = list of population members to sample from
  - Q: Can you imagine a situation where population $\neq / =$ sampling frame?
- …choose sampling method & units that should be in the sample.
Q: Are the above steps necessary when we work with secondary data (e.g., ALLBUS)?

No.. but we should still evaluate representativeness (statistical inference).

id	gender	age	degree	subject
John	M	25	bachelor	sociology
Petra	F	30	master	physics
Hans	M	29	master	biology

names	id	gender	income	education	happiness	age
Hans	1	male	1000	0	5	30
Peter	2	male	5000	3	10	30
Julia	3	female	500	1	3	30
Andrea	4	female	1600	3	7	30
Feli	5	female	1600	3	7	30

Data in table format
Name	trust2006	threat2006	education2006
Aseela	4	0	8
Dominic	5	1	1
Elshaday	0	0	0
Daniel	5	0	9
Sulaimaan	7	0	4
Peyton	5	0	1
Mudrik	2	0	4
Alexander	7	0	5
..	..	..	..

Univariate distribution of trust (2006, N = 6633)
0	1	2	3	4	5	6	7	8	9	10
303	42	172	270	368	1281	852	1342	1294	353	356

Univariate distribution of threat (2006, N = 6633)
0	1
5966	667

Univariate distribution of education (2006, N = 6633)
0	1	2	3	4	5	6	7	8	9	10
380	806	194	89	2182	324	687	474	195	425	877

Individual causal/treatment effect

Treatment variable $D$ : Aspirin
- $D_{i}$ : Value of $D$ for individual $i$ (1 = aspirin = treatment, 0 = no aspirin = control)
Outcome variable $Y$ : Headache
- $Y_{i}(\color{red}{1})$ , $Y_{i}(\color{blue}{0})$ : Potential outcomes for individual $i$
  - e.g., $Y_{i}(\color{red}{1})$ : Headache Peter would have when taking an aspirin
- $Y_{i}$ : Realized outcome → Headache we observe
Individual-level Treatment Effect (ITE)
- $ITE = Y_{i}(\color{red}{1}) - Y_{i}(\color{blue}{0})$ ( $t$ often omitted)
- $ITE_{Peter} = \text{Headache}_{Peter}(\color{red}{\text{Aspirin}}) - \text{Headache}_{Peter}(\color{blue}{\text{No aspirin}})$

Fundamental problem of causal inference (Holl. 1986)

Only one potential outcome will ever be realized and observed!
- Unobserved potential outcome also called counterfactual outcome

Let’s assume Peter takes an aspirin $D_{Peter} = 1$
- We observe $Y_{Peter}(\color{red}{1})$ but not $Y_{Peter}(\color{blue}{0})$
Then we have the following data:

$\quad i\quad$	$\quad D_i\quad$	$\quad Y_{i}\quad$	$\quad Y_{i}(\color{red}{1})\quad$	$Y_{i}(\color{blue}{0})$
Peter	1	0	0	?

But we can not calculate $ITE_{Peter} = Y_{Peter}(\color{red}{1}) - Y_{Peter}(\color{blue}{0}) = 0 - ?$

Assumptions: Scientific solution

unit homogeneity (Holland 1986, 948): Compare two different units and assume they are the same (e.g., 2 samples of substance in lab)
temporal stability (a) and causal transience (b) (Holland 1986, 948): Measure causal effect by sequential exposure of unit $i$ to control and then to treatment, measuring $Y$ after each exposure
- 1. states that the outcome value for unit $i$ under control, does not depend on when the sequence ‘apply control to unit $i$ then measure $Y$ on $i$ ’ occurs
  - i.e., Peter’s value under control would be the same regardless of when Peter is assigned to the control condition
- 1. states the outcome value of unit $i$ under treatment is not affected by the prior exposure of $i$ to the sequence in (a)
  - Peter’s value under treatment not affected by control condition and measurement of $Y$ beforehand
Scientific solution (exploit assumptions above) vs. statistical solution (average causal effect = expected value of the differences) (Holland 1986, 947)

Quick summary

Definition of causal effect does not require more than one individual (Imbens and Rubin 2015, 8)
- Difference in potential outcomes, same individual, same moment in time post-treatment
- $ITE_{Peter} = \text{Headache}_{Peter}(\color{red}{\text{Aspirin}}) - \text{Headache}_{Peter}(\color{blue}{\text{No aspirin}})$
BUT only one potential outcome realized (and observable)
Estimation: Pursue between-individual comparisons or within-individual comparisons
Individuals = units = can be anything (school classes, firms, governments etc.)
Q: How many potential outcomes do we have for a treatment variable Aspirin (no/yes), Education (primary school/high school/university) and Motivation (lowest/low/high/highest)?

Exercise: Employment and life satisfaction

We want to estimate the causal effect for Peter: $\text{Life Satisfaction}_{Peter}(\color{red}{\text{Unemployed}})$ $- \text{Life Satisfaction}_{Peter}(\color{blue}{\text{Employed}})$ at $t = July$ (yellow point).
- Q: What is shown on the graph below?
- Q: At $t = July$ , do we observe Peter’s outcome under treatment (unemployed) or under control (employed)?
- Q: Which green measurement point would you pick as a comparison for Peter’s yellow measurement? And why?

ATE: Average Treatment Effect

Average Treatement Effect (ATE): The average difference in the pair of potential outcomes (averaged over the entire population of interest)
- $ATE = E[Y_{i}(1) - Y_{i}(0)]$ (time is omitted from the notation)
- Q: Which observations does that concern in the table below?

$Unit$	$D_{i} (Aspirin:Yes/No)\quad$	$Y_{i}(1) (Headache \mid Aspirin)\quad$	$Y_{i}(0) (Headache \mid NoAspirin)$
Simon	1	0	?
Julia	1	1	?
Paul	0	?	1
Trump	0	?	0
Fabrizio	0	?	0
Diego	0	?	0

Again.. can’t estimate ATE without observing both potential outcomes (even with infinite sample size)

Why? moving from ITE to ATE?

$ITE$ is unobservable
Average Treatment Effect ( $ATE$ ) = Diff. in averages
$ATE$ can be estimated relying on less daring assumptions (Holland 1986, 948f)
- e.g., (proper) random assignment (should) partly satisfy them (“statistical solution”)

Important: $ATE = E[Y_{i}(1) - Y_{i}(0)]$ : is the expected value of the unit-level treatment effect under the distribution induced by sampling from the super-population (= ATE in super-population) (Imbens and Rubin 2015, 117)

Estimating Population ATE (PATE) $(ATE_{sp} = \tau_{sp} = \mathbb{E}_{sp})$ from Sample ATE (SATE) $(ATE_{fs} = \tau_{fs})$ requires random sample (Holland 1986, Imai 2008)
- $_{fs}$ = finite sample, $_{sp}$ = super population (= target population)

ATE: Naive Estimate

Naive estimate of ATE: Difference between expected values in treatment and control
- Equate $E[Y_{i}(1)]$ - $E[Y_{i}(0)]$ with $E[Y_{i}|D=1] - E[Y_{i}|D=0]$
- $E[Y_{i}|D=1] - E[Y_{i}|D=0] = ({\color{red}{0+1}})/2 - ({\color{orange}{1+0+0+0}})/4 = 0.5 - 0.25 = 0.25$

$Unit$	$D_{i} (Aspirin: Yes/No)\quad$	$Y_{i} (Headache: Yes/No)\quad$
Simon	1	0
Julia	1	1
Paul	0	1
Trump	0	0
Fabrizio	0	0
Diego	0	0

Q: What is the problem with the naive estimate of the treatment effect? Can we interpret 0.25 as causal effect? Why (not)?

ATE: Decomposition

$Unit$	$D_{i} (Aspirin: Yes/No)\quad$	$Y_{i} (Head.: Yes/No)\quad$	$Y_{i}(1) (Head. \mid YesAspirin)\quad$	$Y_{i}(0) (Head. \mid NoAspirin)$
Simon	1	0	0	?
Julia	1	1	1	?
Paul	0	1	?	1
Trump	0	0	?	0
Fabrizio	0	0	?	0
Diego	0	0	?	0

ATE can be decomposed as a function of 5 quantities (e.g., Keele 2015, 4)
$\underbrace{E[Y_{i}(1) - Y_{i}(0)]}_{\substack{ATE}}$ = ${\color{violet}\pi}$ $(\underbrace{E[\color{red} {Y_{i}(1)|D_{i} = 1}] - E[\color{blue}{Y_{i}(0)|D_{i} = 1}]}_{\substack{ATT}})$ + $(1 - {\color{violet}\pi})(\underbrace{E[\color{green}{Y_{i}(1)|D_{i} = 0}] - E[\color{orange}{Y_{i}(0)|D_{i} = 0}]}_{\substack{ATC}})$
${\color{violet}\pi}$ : proportion of sample assigned to treatment (2/6 = 0.33..)
e.g., $E[\color{red} {Y_{i}(1)|D_{i} = 1}]$ : Average pot. outcome under treatment, given units are in treatment condition

Q: What do the following terms describe? $E[\color{blue}{Y_{i}(0)|D_{i} = 1}]$ ; $E[\color{green}{Y_{i}(1)|D_{i} = 0}]$ ; $E[\color{orange}{Y_{i}(0)|D_{i} = 0}]$
Q: In your own words, which parts do we have and which parts don’t we have?

ATE: Identification Problem

Observed (measured)
- ${\color{violet}\pi}$ using $E[{\color{violet}{D_{i}}}]$
- $E[\color{red} {Y_{i}(1)|D_{i} = 1}]$ using $E[Y_{i}\mid D_{i} = 1]$
- $E[\color{orange}{Y_{i}(0)|D_{i} = 0}]$ using $E[Y_{i}\mid D_{i} = 0]$
Unobserved (not measured)
- $E[\color{blue}{Y_{i}(0)|D_{i} = 1}]$ : Average outcome under control $(Y_{i}(0))$ , given units in treatment condition $(...|(D_{i}=1))$
- $E[\color{green}{Y_{i}(1)|D_{i} = 0}]$ : Average outcome under treatment $(Y_{i}(1))$ , given units in control condition $(...|(D_{i}=0))$
$E[Y_{i}(1) - Y_{i}(0)]$ = 2/6 $\times$ (0.5 - ?) + 4/6 $\times$ (? - 0.25)

	$Y = 1$	$Y = 0$
$D = 1$	0.5	?
$D = 0$	?	0.25

Identification Problem!
- Two quantities are unobservable
- Just as for ITE we must rely on assumptions to identify ATE (fill em in)

Causal estimands & notation (2)

Often the focus is on subpopulations (Imbens and Rubin 2015, 18)
Females (covariate value): $\tau_{fs} (f) = \frac{1}{N(f)} \sum\limits_{i: X_{i} = f}^{N} (Y_{i}(1) - Y_{i}(0))$ (Conditional ATE)
- e.g., average effect of drug only for females
ATT: $\tau_{fs,t} = \frac{1}{N_{t}} \sum\limits_{i: D_{i} = 1}^{N} (Y_{i}(1) - Y_{i}(0))$
- Average effect of the treatment for those who were exposed to it
- Q: How could we write the ATC?
Other subpopulations: Complier Average Treatment Effect; Intent-to-Treat Effect (see overview at Egap)

Causal estimands & notation (3): Exercise

Q: For which units in the Table below would you need to fill in the missing potential outcomes if you were interested in…

…the average treatment effect: $\small ATE_{fs} = \tau_{fs} = \frac{1}{N} \sum_{i=1}^{N} (Y_{i}(1) - Y_{i}(0))$ ?
…the average treatment effect on the treated: $\small ATT_{fs} = \tau_{fs,t} = \frac{1}{N_{t}} \sum\limits_{i: D_{i} = 1}^{N} (Y_{i}(1) - Y_{i}(0))$ ?
…the conditional average treatment effect for males: $\small ATE_{fs,m} = \tau_{fs} (m) = \frac{1}{N(m)} \sum\limits_{i: X_{i} = m}^{N} (Y_{i}(1) - Y_{i}(0))$ ?
…the conditional average treatment effect on the treated for males: $\small ATT_{fs}(m) = \tau_{fs, t} (m) = \frac{1}{N_{t}(m)} \sum\limits_{i: D_{i} = 1, X_{i} = m}^{N} (Y_{i}(1) - Y_{i}(0))$ ?

Causal estimands: Exercise
$Unit$	$D_{i}$	$Y_{i}(1)$	$Y_{i}(0)$
Simon	1	0	?
Julia	1	1	?
Paul	0	?	1
Trump	0	?	0
Fabrizio	0	?	0
Diego	0	?	0

(Key) Assumptions

Often causal identification concerns defending several key assumptions (e.g., Keele 2015)

Causal ordering assumption: Written as $D_{i} \longrightarrow Y_{i}$ (Imai’s notation)
- No reverse causality: $D_{i}\not\longleftarrow Y_{i}$
- No simultaneity: $D_{i}\not\longleftrightarrow Y_{i}$
Independence assumption (IA): also called unconfounded assignment (For other names see Imbens and Rubin 2015, 43)
Stable Unit Treatment Values Assumption (SUTVA): (1) No interference assumption & (2) Consistency assumption

We’ll talk about 2. and 3. now and other ones later

Assumptions: Independence Assumption (IA)

IA = Treatment status is independent of potential outcomes $(Y_{i}(1), Y_{i}(0) \perp D_{i})$
- i.e., assignment status unrelated to potential outcomes…
- …whether you take aspirin or not is independent of what outcome you would have under treatment/control
- Violation: e.g., experimenter knows how patients respond to aspirin and assigns accordingly
Insight: Under IA “expectation(s) of the unobserved potential outcomes is equal to the conditional expectations of the observed outcomes conditional on treatment assignment” (Keele 2015, 5)
- IA allows us to connect unobservable potential outcomes to observable quantities in the data
- IA is linked to the “assignment mechanism” (see (Imbens and Rubin 2015, 14) for a recent discussion)
Why does the indpendence assumption (+ SUTVA below) identify causal effect? (next slide)

Assumptions: Independence Assumption (IA)

Under the IA: ATE = $\underbrace{E[Y_{i}(1) - Y_{i}(0)]}_{\substack{unobserved}} = E[Y_{i}(1)] - E[Y_{i}(0)] = \underbrace{E[Y_{i}|D_{i} = 1] - E[Y_{i}|D_{i} = 0]}_{\substack{observed}}$
We can replaces potential outcomes with observed outcomes

$Unit$	$D_{i}\quad$	$Y_{i}\quad$	$Y_{i}(1)\quad$	$Y_{i}(0)$
Simon	1	0	0	?
Julia	1	1	1	?
Paul	0	1	?	1
Trump	0	0	?	0
Fabrizio	0	0	?	0
Diego	0	0	?	0

The IA allows us to equate the expected value of the whole column $E[Y_{i}(1)]$ (red and green values) with the observed red values, i.e. $E[Y_{i}|D_{i} = 1]$ (same for column $E[Y_{i}(1)]$ ).
$E[Y_{i}|D_{i} = 1] \stackrel{1}{=} E[(Y_{i}(0) + D_{i}(Y_{i}(1) - Y_{i}(0))| D_{i} = 1] \stackrel{2}{=} E[Y_{i}(1)|D_{i} = 1] \stackrel{3}{=} E[Y_{i}(1)]$
See next slide for explanation of step $\stackrel{1}{=}$ , $\stackrel{2}{=}$ and $\stackrel{3}{=}$
- Same logic for $E[Y_{i}\mid D_{i} = 0] = E[Y_{i}(0)]$

Assumptions: Independence Assumption - Steps

Step $\stackrel{1}{=}$ : In causal inference, the observed outcome $Y_{i}$ can be expressed as a combination of potential outcomes $Y_{i}(0)$ and $Y_{i}(1)$ based on the treatment indicator $D_{i}$ (Observed Outcome Definition). Specifically, $Y_{i} = Y_{i}(0) + D_{i} (Y_{i}(1) - Y_{i}(0))$ .
- This means that if the individual $i$ does not receive the treatment ( $D_{i} = 0$ ), their outcome is $Y_{i}(0)$ . If they do receive the treatment ( $D_{i} = 1$ ), their outcome is $Y_{i}(1)$ .
Step $\stackrel{2}{=}$ : If $D_{i} = 1$ , $(Y_{i}(0) + 1\times Y_{i}(1) - 1\times Y_{i}(0))$ so $Y_{i}(0)$ cancels out and we end up with $E[Y_{i}(1)\mid D_{i} = 1]$
Step $\stackrel{3}{=}$ : Because $Y_{i}(1)$ is independent of $D_{i}$ (independence assumption) we can replace $E[Y_{i}(1)\mid D_{i} = 1]$ with $E[Y_{i}(1)]$
Longer explanation
- To estimate the ATE we would need to calculate the expected value of the differences between potential outcomes in column $Y_{i}(1)$ and column $Y_{i}(0)$ . In other words, we would have to observe both treatment and control units in their counterfactual states (e.g., observe what the value of control units would be if they had been treated). However, for the units that were assigned to control $D_{i} = 0$ we do not observe $Y_{i}(1)$ and the other way round.
- Starting with the column $Y_{i}(1)$ , the independence assumption simply means that the expected value of the whole column $E[Y_{i}(1)]$ (red and green values) can be equated with the expected value of the first two rows of the column, namely $E[Y_{i}(1)\mid D_{i} = 1]$ (the red values). And that is what we actually observe. Hence, through this assumption there is no need to observe the missing green values any more. The same logic applies to column $Y_{i}(0)$ . The IA allows us to equate the expected value of the whole column $E[Y_{i}(0)]$ (blue and orange values) with the orange values, i.e. $E[Y_{i}(0)\mid D_{i} = 0]$ .

Independence assumption & random assignment

Q: Imagine the students in our class were randomly split into two groups, a treatment group and a control group. What would be your expectation regarding the distribution of … across the two groups?
- Observable characteristics (Gender, age, skin color)
- Unobservable characteristics (blood pressure, happiness)
- Possible responses to experimental treatment (reaction to aspirin)

Random assignment
- = a statistical solution (Holland 1986, 948f)
- Units randomly assigned to treatment/control have identical distributions of covariates/potential outcomes in both groups (long run!)
“if the physical randomization is carried out correctly, then it is plausible that S [= treatment status D] is independent of $Y_{t}$ [ $Y_{i}(1)$ ], and $Y_{c}$ [ $Y_{i}(0)$ ], and all other variables over U [the super population]. This is the independence assumption.” (Holland 1986, 948).
- Random assignment induces independence between treatment status and potential outcomes
Q: What does in the long run mean?

Assumptions: SUTVA

SUTVA: Stable Unit Treatment Values Assumption
-1- No interference & -2- Consistency: No hidden variations of treatment

The potential outcomes for any unit do not vary with the treatments assigned to other units

A subject’s potential outcome is not affected by other subjects’ exposure to the treatment
Imai’s notation: $Y_{i}(D_{1},D_{2},...,D_{n})=Y_{i}(D_{i})$

For each unit, there are no different forms or versions of each treatment level, which lead to different potential outcomes (Imbens and Rubin 2015, 10; Keele 2015, 5)

Unambiguous definition of exposure, e.g. “15 min of exercise” (Keele 2015, 5)
No hidden multiple versions/different administration of treatment
Imai’s notation: $Y_{i}=Y_{i}(d)\text{ whenever } D_{i}=d$

Assumptions: SUTVA Exercise (2)

Q: Below you find examples for planned experiments. Discuss in groups of 2 or 3.
- What is the unit in these examples? How could the independence assumption and the SUTVA assumption be violated when those experiments are implemented?

Private lessons $(D)$ → School performance $(Y)$
- In a Mannheim school, half of the 10% worst performing pupils in each class are randomly assigned to receiving private lessons.
In-person teaching $(D)$ → student satisfaction $(Y)$
- To test the reception of online-teaching, a unversity decides to randomly assign half of the seminars to in-person teaching, the other half to online teaching.
Job-training programme $(D)$ → Unemployment $(Y)$
- Half of Mannheim’s unemployed are randomly assigned to a job training program.
COVID19 drug $(D)$ → survival $(Y)$
- 50% of the COVID19 patients at a Mannheim hospital are randomly assigned a new, promising drug in a placebo trial.

Assumptions example: Campaign Advertisement and Vote Choice (1)

Research question: What is the effect of a campaign advertisement on an individual’s vote choice (Party A vs. B)?
- Treatment $D_{i}$ : Exposure to the campaign advertisement
- Outcome $Y_{i}$ : Individual’s vote choice.
Causal Ordering Assumption $D_{i} \longrightarrow Y_{i}$ : implies that treatment causes outcome and not the other way around
- No reverse causality $D_{i}\not\longleftarrow Y_{i}$ : implies that an individual’s vote choice does not influence whether or not they were exposed to the campaign advertisement
- No simultaneity: $D_{i}\not\longleftrightarrow Y_{i}$ : implies that treatment and outcome do not occur simultaneously, i.e., exposure to advertisement $D$ occurs before the vote choice $Y$ is made
- Practice: ensure this by timing exposure to advertisement $D$ before voting choice $Y$ is made and by randomizing advertisement exposure to avoid selection bias where more likely voters for Party A are more exposed to the advertisement

Assumptions example: Campaign Advertisement and Vote Choice (2)

Independence Assumption $(Y_{i}(1), Y_{i}(0) \perp D_{i})$ : states that treatment assignment is independent of potential outcomes
- i.e., the exposure to the advertisement should not be correlated with other factors that might influence the vote choice which can be achieved through randomization
Stable Unit Treatment Value Assumption (SUTVA)
- No Interference: treatment of one individual does not affect the outcomes of another individual
  - e.g., one person’s exposure to the advertisement does not influence another person’s vote choice
- Consistency Assumption: For each unit, there are no different forms or versions of each treatment level, which lead to different potential outcomes (unambiguous definition of exposure)
  - e.g., if an individual is exposed to the advertisement, their vote choice reflects the effect of seeing that specific advertisement which is the same for other treated individuals (analogue for individuals in control)
- SUTVA violated if individuals discuss the advertisement with each other, thereby indirectly affecting each other’s vote choices or if advertisement is different across treated individuals

Experimental vs. observational studies

Assignment mechanism: “process that determines which units receive which treatments, hence which potential outcomes are realized and thus can be observed [and which are missing]” (see Imbens and Rubin 2015, 31)
Q: What is the difference between experimental (experiments) and observational studies?

Experiments (experimental studies inlcude lab & field experiments)
- Assignment mechanism is both known and controlled by the researcher
- Researcher randomly assigns units to treatment and control
- We know the function of $Pr(\mathbf{D}|\mathbf{X},\mathbf{Y}(0),\mathbf{Y}(1))$ !
Observational studies (e.g., Imbens and Rubin 2015, 41)
- Assignment mechanism is not known to, or not under the control of, the researcher
- Researcher can not randomly assign units to treatment and control
- We don’t know the functional form

We are leaving the realm of experimental studies and enter the realm of observational studies!

Observational studies (1)

Many questions cannot be answered using experiments (Q: Any examples?)
Imbens and Rubin (2015) show that under certain assumptions assignment mechanism within subpopulations of units with the same value for the covariates (Q?) can be interpreted as if it was completely randomized experiment (see Imbens and Rubin 2015, 257)
- Albeit an experiment with unknown assignment probabilities for the units

Assumptions (see also Imbens and Rubin 2015, 257, 262, 43)
- 1. Unconfounded assignment or uncounfoundedness (= conditional independence): Assignment free from dependence on the potential outcomes
- 1. Probabilistic assignment: Probability of receiving any level of the treatment is strictly between zero and one for all units
- 1. Individualistic assignment: Probability for unit $i$ is essentially a function of the pre-treatment variables for unit $i$ only, free of dependence on the values of pre-treatment variables for other units (Q: What is difference to SUTVA?³)
- Assumption (2) & (3) often glossed over (see Imbens and Rubin 2015, 43)

Observational studies (2)

I & R call assignment mechanisms that fulfill those assumptions regular assignment mechanism
Given assumptions 1-3 the probability of receiving the treatment is equal to $e(x)=N_{t}(x)/(N_{c}(x)+N_{t}(x))$ for all units with $X_{i}=x$ conditional on the number of treated and control units composing such a subpopulation
- $N_{t}(x)$ or $N_{c}(x)$ are the number of units in treatment and control groups with pre-treatment value $X_{i}=x$
- $e(x)$ is also called the Propensity score⁴
We don’t know a priori assignment probabilities for units, but know that units with the same pre-treatment covariate values have same $e(x)$ , i.e., the same prob. of getting the treatment
This insight still suggests feasible strategies (e.g., focus on subsample)!
Q: What might be the problem if we have many distinct values of covariates?

Exercise 1: Conditioning on covariates (1)

Q: Which persons would you compare (use) in the table below if you want to condition on gender, i.e., hold the value of the covariate gender constant ( $X_{i}=1$ , $X_{i}=0$ ) and focus on the respective subsamples (subsetting!)?

Table: Holding covariate values constant
$Unit$	$D_{i}$	$X_{i}$	$Y_{i}$	$Y_{i}(1)$	$Y_{i}(0)$
Simon	1	0	0	0	?
Julia	1	1	1	1	?
Paul	0	0	1	?	1
Sarah	0	1	0	?	0
Fabrizio	0	0	0	?	0
Diego	0	0	0	?	0

Exercise 1: Conditioning on covariates (2)

Below we colored individuals for which $(\color{#984ea3}{X_{i} = 1})$ and $(\color{#fb9a99}{X_{i} = 0})$ .

Table: Holding covariate values constant
$Unit$	$D_{i}$	$X_{i}$	$Y_{i}$	$Y_{i}(1)$	$Y_{i}(0)$
Simon	1	0	0	0	?
Julia	1	1	1	1	?
Paul	0	0	1	?	1
Sarah	0	1	0	?	0
Fabrizio	0	0	0	?	0
Diego	0	0	0	?	0

Q: Imagine we study the causal effect of education (MA sociology degree: Yes, no) on income (in Euros): Do you think the assumption of unconfoundedness/conditional independence is realistic when we only condition on gender?

Exercise 2: Conditioning on covariates (1)

Q: On the left you find the joint distribution of outcome Trust (Y), treatment Victimization (D) and covariate Education (X).
- If victimization had been randomly assigned, would estimating the causal effect require conditioning for education (= holding education constant)?
- Which conditional distribution do we focus on if we were to hold Education constant at $X_{i}=6$ ? And $X_{i}=2$ ?
- You assume cond. independence/unconfoundedness for subpopulations of values of the Education variable (i.e., victimization should be as good as random). Within which partitions of the sample would you then compare victims to non-victims?

Exercise 2: Conditioning on covariates (2)

Conditional distributions are colored in orange: $(\color{orange}{X_{i}=6})$ and $(\color{orange}{X_{i}=2})$ .
Important notion: Condition/control = filtering (Pearl et al. 2016, 8)
- when we condition on X, we filter the data into subsets based on values of X (Pearl et al. 2016, 8) [hold values of X constant]
- Subsequently, we would estimate the treatment effect in those subsets and average it (in some way)

Exercise: Covariates & bias

Q: Discuss in groups: A friend of yours wants to investigate the causal effect of having studied (yes vs. no) at $t_0$ on individuals’ income 20 years later $t_1$ . Your friend wants to control (condition on) different covariates but is nervous because she heard that one might introduce different biases. She is considering the covariates below. Please indicate whether one should control for the following variables (“Yes” vs. “No”) in this concrete example and what kind of bias they may introduce or avoid.
- 1. Marital status at the age of 35
- 1. Parents’ educational level
- 1. Piano lessons during childhood
- 1. Intelligence test in school
- 1. Work experience at the age of 16
- 1. Job skill-level (low vs. high) in his/her forties

Model-based imputation (2)

Data: $\small N_{t}$ =42; $\small N_{c}$ =150 (subset visualized)
Education: Mean 2.81 for treated and 5.01 for control units
- Q: Is the covariate distribution of Education (X) across treatment and control balanced?
LM: $\small y_{i} = \underbrace{\color{magenta}{\beta_{0}} + \color{orange}{\beta _{1}} \times d_{i} + \color{orange}{\beta _{2}} \times x_{i}}_{\text{Modell} = \color{green}{\widehat{y}}_{i} = \text{Predicted values}} + \underbrace{\color{red}{\varepsilon}_{i}}_{\color{red}{Error}} = \color{green}{\widehat{y}}_{i} + \color{red}{\varepsilon}_{i}$
- Estimates: $\small \color{magenta}{\beta_{0}}$ =6.02; $\small \color{orange}{\beta _{1}}$ = -0.51; $\small \color{orange}{\beta _{2}}$ = 0.06;
Q: What do we mean by “extrapolation”?
- Model makes predictions for regions where we don’t have observations (e.g., victims with education 7 or higher)
Pre-processing data makes sense!
- Prune units without equivalent units in treatment or control to balance data

Balancing scores & Propensity score (1)

Idea: find lower-dimensional functions of covariates that suffice for removing bias associated with differences in the pre-treatment variables (Imbens and Rubin 2015, 266f)
- e.g., can we replace our relevant covariates (e.g., education, gender, age) with a balancing score that represents them?
Formally: balancing score is a function of the covariates such that the probability of receiving the active treatment given the covariates is free of dependence on the covariates given the balancing score: $D_{i} \perp \!\!\! \perp X_{i} | b(X_{i})$
Important property: if assignment to treatment is unconfounded given the full set of covariates, then assignment is also unconfounded conditioning only on a balancing score

Balancing scores & Propensity score (2)

Propensity score is a scalar balancing score
- = conditional probability of receiving the treatment given $X_{i}=x$
- $D_{i} \perp \!\!\! \perp X_{i} | e(X_{i})$
Estimating the propensity score
- A science in itself… (see Imbens and Rubin 2015, Ch. 13)
- Practice: Estimate a logistic regression in which treatment variable D is predicted by covariates X
  - Yields propensity score (probability that someone receives treatment given covariates) as predicted by X
- However we still need to make the unconfoundeness assumption

Data + propensity score
Name	x_education	x_age	d_victim	y_trust	pr_score
Alissa	4	75	0	8	0.14
Damaris	0	17	1	7	0.50
Juan	0	18	0	9	0.50
Rosa	6	62	0	5	0.11
Janeth	4	62	0	6	0.17
Yeimi	2	51	1	9	0.28
Jacob	4	31	0	5	0.24
Monica	4	38	1	8	0.22
Cesar	6	44	1	3	0.14
Marcos	0	18	1	5	0.50

Reminder: Classical experiment

The “Classical” Experiment
- Randomly split units into two groups
- Expose one to the treatment but not the other (control group)
- After randomization independence holds: $Y_{i}(0), Y_{i}(1), X_{i} \perp D_{i}\text{ for all i = 1, ..., N}$
Difference-in-Means: $\hat{\tau} = E[Y_{i}(1)|D_{i} = 1] - E[Y_{i}(0)|D_{i} = 0]$ …

…as an unbiased estimate of the…
…Average treatment effect: $E[\hat{\tau}] = E[Y_{i}(1)] - E[Y_{i}(0)]$
Randomized experiments considered “gold standard” but often not feasible…

Table 1: Wide format
unit	trust.2006	trust.2007	Victimization.2006	Victimization.2007
Peter	1	1	0	1
Julia	5	5	0	1
Pedro	6	6	1	0

Table 2: Long format
unit	time	trust	Victimization
Julia	2006	5	0
Julia	2007	5	1
Pedro	2006	6	1
Pedro	2007	6	0
Peter	2006	1	0
Peter	2007	1	1

DiD Setup (1)

Two groups of units (e.g., restaurants, individuals)
Outcome Y: Observed twice, before and after treatment (2 timepoints/periods $t_{0}$ and $t_{1}$ )
Treatment D: happens between $t_{0}$ and $t_{1}$
- Treatment status at $t_{0}$ and $t_{1}$ is known (= 0 for all at $t_{0}$ )
Covariates X: Observed at $t_{0}$ or before
Graph (left) shows average outcome (across time and groups)
Naive strategies:
- $\scriptsize E[Y_{i}|D_{i} = 1; T_{i} = 1]$ $\scriptsize- E[Y_{i}|D_{i} = 1; T_{i} = 0]$
- $\scriptsize E[Y_{i}|D_{i} = 1; T_{i} = 1]$ $\scriptsize- E[Y_{i}|D_{i} = 0; T_{i} = 1]$
- Q: What would this be in the graph on the left?
Problem: Bias due to pre-existing trends/pre-existing differences (Q: ?)
Solution/Idea: Take trend of control group as counterfactual for unobserved trend in treatment group

DiD Setup (2)

We absorb unit-specific pre-treatment levels in Y through differencing within units
- $E[Y_{i}(d)|D_{i} = d; T_{i} = 1] - E[Y_{i}(d)|D_{i} = d; T_{i} = 0]\text{ for }d \in {0,1}$
- i.e., difference between averages of individual changes between $t_{0}$ and $t_{1}$ both for treatment and control units (only change is left over)
- Q: Where would we find those averages in the graph on the preceeding slide?
We absorb pre-existing trends by differencing within-differences across units
- $\underbrace{E[Y_{i}(1)|D_{i} = 1; T_{i} = 1] - E[Y_{i}(1)|D_{i} = 1; T_{i} = 0]}_{\text{Trend in treatment group}}$ - $\{\underbrace{E[Y_{i}(0)|D_{i} = 0; T_{i} = 1] - E[Y_{i}(0)|D_{i} = 0; T_{i} = 0]}_{\text{Trend in control group}}\}$

Table 1 provides a short example where $\Delta Y$ reflects differencing within units. Subsequently, we would calculate $(-20 + (- 30))/2 - (-10 + 0)/2 = -15$ (differencing within-differences).

Table 1: Causal estimands: Exercise

$Unit$	$D_{i}$	$Y_{i, t = 0}$	$Y_{i, t = 1}$	$\Delta Y$
Restaurant 1	1	40	20	-20
Restaurant 2	1	60	30	-30
Restaurant 3	0	50	40	-10
Restaurant 4	0	30	30	0

DiD: Assumptions (1)

What we want to know:
- $\underbrace{E[Y_{i}(1)|D_{i} = 1; T_{i} = 1] - E[Y_{i}(1)|D_{i} = 1; T_{i} = 0]}_{\text{Trend under treatment for treated units}}$ - $\{\underbrace{E[Y_{i}(0)|D_{i} = 1; T_{i} = 1] - E[Y_{i}(0)|D_{i} = 1; T_{i} = 0]}_{\text{Trend under control for treated units}}\}$
What we do know:
- $\underbrace{E[Y_{i}(1)|D_{i} = 1; T_{i} = 1] - E[Y_{i}(1)|D_{i} = 1; T_{i} = 0]}_{\text{Trend in treatment group}}$ - $\{\underbrace{E[Y_{i}(0)|D_{i} = 0; T_{i} = 1] - E[Y_{i}(0)|D_{i} = 0; T_{i} = 0]}_{\text{Trend in control group}}\}$
Q: So which assumption do we make here?
Q: How do we know that the trend in the control group adequately captures the counterfactual trend in the treatment group in absence of the treatment?

DiD: Regression estimation (1)

We can easily estimate our DiD effect via regression.. but how?
We want to estimate:
- $\hat{\tau} = \underbrace{E[Y_{i}(1)|D_{i} = 1; T_{i} = 1]}_{\text{Treatment group; posttreatment}} - \underbrace{E[Y_{i}(1)|D_{i} = 1; T_{i} = 0]}_{\text{Treatment group; pretreatment}} -$ $\{\underbrace{E[Y_{i}(0)|D_{i} = 0; T_{i} = 1]}_{\text{Controlgroup; posttreatment}} - \underbrace{E[Y_{i}(0)|D_{i} = 0; T_{i} = 0}_{\text{Controlgroup; pretreatment}}]\}$
So what do we need?
- a pre-/post-period indicator, $T_{i} \in \{0,1\}$ (time variable)
- a treatment/control group indicator, $D_{i} \in \{0,1\}$ (treatment variable)
- an interaction of the two to capture the four combinations of $D_{i}$ and $T_{i}$

DiD: Regression estimation (2)

Table 1: Longformat data (toy data!)
name	time T	treated D	outcome Y
Restaurant_1	0	1	15.00
Restaurant_3	0	1	24.00
Restaurant_5	0	1	15.00
Restaurant_2	0	0	40.50
Restaurant_4	0	0	13.75
Restaurant_6	0	0	8.50
Restaurant_1	1	1	27.00
Restaurant_3	1	1	23.00
Restaurant_5	1	1	21.50
Restaurant_2	1	0	24.00
Restaurant_4	1	0	11.50
Restaurant_6	1	0	10.50

Table 2: Aggregated data (means)
time T	treated D	mean_Y
0	0	20.92
0	1	18.00
1	0	15.33
1	1	23.83

Q: Try to write down the DID estimator in potential outcomes notation (previous slide!).
The DID estimator can be implemented through the following linear model:
- $\text{outcome Y}_{i} =$ $\beta_{1} + \beta_{2}\text{time T}_{i} + \beta_{3}\text{treated D}_{i} +$ $\beta_{4}\text{time T}_{i}\times \text{treated D}_{i}+ \epsilon_{i}$
Q: Please explain…
- …how we get from Table 1 to Table 2.
- …to which rows (mean_Y) in Table 2 the terms in the DiD estimator (previous slide) correspond.
- …how (combinations of) the terms in the model (above) may yield estimates of each of the four components (mean_Y values in Table 2).

DiD: Regression estimation (3)


	Dependent variable:

	`outcome Y`

`treated D`	-2.92
	(8.02)

`time T`	-5.58
	(8.02)

I(`treated D` * `time T`)	11.42
	(11.35)

Constant	20.92^***
	(5.67)


Observations	12
R²	0.14
Adjusted R²	-0.19
Residual Std. Error	9.83 (df = 8)
F Statistic	0.42 (df = 3; 8)

Note:	p<0.1; p<0.05; p<0.01

Equation: $\text{outcome Y}_{i} =$ $\beta_{1} + \beta_{2}\text{time T}_{i} + \beta_{3}\text{treated D}_{i} +$ $\beta_{4}\text{time T}_{i}\times \text{treated D}_{i}+ \epsilon_{i}$
We can estimate the model for the data above as follows (see left):

lm(`outcome Y` ~ `treated D` + `time T` + 
                 I(`treated D`*`time T`), 
                data= data_long)

Below we calculate the coefficicents from our group $\times$ time averages
- $\beta_{1}$ = 20.92
- $\beta_{2}$ = 15.33 - 20.92 = -5.59
- $\beta_{3}$ = 18.00 - 20.92 = -2.92
- $\beta_{4}$ = Difference between trends = (23.83 - 18.00) - (15.33 - 20.92) = 11.42 (our causal estimate!)

DiD: Regression estimation (4)

Alternatively, we could difference the outcome beforehand
- Outcome is now $y_{i,t1} - y_{i,t0}$

Wide format
name	D	Y (T = 0)	Y (T = 1)	Y_diff
Restaurant_1	1	15.00	27.0	12.00
Restaurant_2	0	40.50	24.0	-16.50
Restaurant_3	1	24.00	23.0	-1.00
Restaurant_4	0	13.75	11.5	-2.25
Restaurant_5	1	15.00	21.5	6.50
Restaurant_6	0	8.50	10.5	2.00

Estimation with wide-format data
- $y_{i,t1} - y_{i,t0} = \beta_{0} + \beta _{1} D_{i} + \beta X_{i} + \varepsilon_{i}$
Summing up: The kind of model we estimate in R depends on the data structure
- Long-format: lm(Y ~ D + T + D*T + X, data= data_long)
- Wide-format: lm(Y_diff ~ D + X, data= data_long)
- …where X are covariates that we can add to make the parallel trends assumption more realistic

Covariates & aggregate units & standard error

Condition on covariates
- Account for possibility that samples have systematically different characteristics (and would not display parallel trends)
- Choose covariates that were measured at $t_{0}$ or before
DiD can be used with individuals and aggregate units (e.g., cities)
- Be aware that composition of aggregate units may change (drop-outs?)
Sometimes necessary to adjust standard errors as not to understate uncertainty (e.g., if there is geographic closeness)

Basics (2)

$X_{i}$ : Score with $c$ as a known cutoff
- also called running variable, forcing variable or index
$Z_{i}$ = Assignment variable; $1 \text{ if } X_{i} \geq c, 0\text{ otherwise}$
$D_{i}$ : Treatment actually received
Q: Can you give an example of $X_{i}$ , $Z_{i}$ and $D_{i}$ for a concrete person, e.g., thinking of access to a Bachelor program?

We observe outcome under control condition for units below cutoff and outcome under treatment condition for units above cutoff

Basics (3): Sharp vs. Fuzzy RDD

Important: Assignment to treatment $Z_{i}$ (e.g., being offered scholarship) not necessarily $=$ receiving (complying with) treatment $D_{i}$ (e.g., taking the scholarship)
Sharp RD: $Z_{i} = D_{i}$ , all units comply with treatment condition they have been assigned to
Fuzzy RD: $Z_{i} \neq D_{i}$ , some units fail to receive treatment despite having a score above cutoff (and vice versa)

Q: Explain sharp and fuzzy RD in the case of a Bachelor program were cutoff is a certain grade (1.4)?

Basics (4)

Observed average outcome given the score & potential outcomes

$\mathbb{E}[Y_{i}|X_{i}] = \begin{cases} \mathbb{E}[Y_{i}(0)|X_{i}] & \quad \text{if } X_{i} < c\\ \mathbb{E}[Y_{i}(1)|X_{i}] & \quad \text{if } X_{i} \geq c \end{cases}$

Sharp RD design exhibits extreme case of lack of common support on $X$
- Units in the control and treatment groups cannot have the same value of score $X$
- This highlights that RD analysis fundamentally relies on extrapolation towards cutoff point $c$
Central goal of empirical RD analysis
- Adequately perform (local) extrapolation in order to compare control and treatment units
Basic assumption of comparability
- Units with very similar values of the score but on opposite sides of the cutoff are comparable

Basics (5)

Fund. problem of causal inference: Only observe the outcome under control, $Y_{i}(0)$ , for those units whose score is below the cutoff $c$ , and the outcome under treatment, $Y_{i}(1)$ , for those units whose score is above the cutoff $c$
Fig. 3 plots the average potential outcomes given the score, $E[Y_{i}(1)|X_{i} = x]$ and $E[Y_{i}(0)|X_{i} = x]$ , against the score
We can estimate the regression function $E[Y_{i}(1)|X_{i}]$ for values of the score to the right of the cutoff because we observe $Y_{i}(1)$ , for every $i$ when $X \geq c$ (solid red line)
- And $E[Y_{i}(0)|X_{i}]$ for values left to the cutoff (solid blue line)
Sharp RD treatment effect: $\tau_{SRD} = \mathbb{E}[Y_{i}(1) - Y_{i}(0)|X_{i} = c]$
- Conceptually: what would be the average outcome change for units with score level $X_{i} = c$ if we switched their status from control to treated?

Example: Meyersson (2014) (1)

Meyersson (2014): Islamic Rule and the Empowerment of the Poor and Pious
- Research question: What is the impact of having a mayor from an Islamic party on women’s rights (educational attainment of young women)?
- $\text{Units }i$ : Municipalities
- $Y_{i}$ : educational attainment of women, measured as the percentage of women aged 15 to 20 in 2000 who had completed high school by 2000
- $X_{i}$ : vote margin obtained by the Islamic party in the 1994 Turkish mayoral elections, measured as the vote percentage obtained by the Islamic party minus the vote percentage obtained by its strongest secular party opponent
- $D_{i}$ ( $T$ in the paper): electoral victory of the Islamic party in 1994, equal to 1 if the Islamic party won the mayoral election and 0 otherwise $(D_{i} = Z_{i})$
Methodological challenge: Municipalities where support for Islamic parties is high enough to result in the election of an Islamic mayor (of one of two islamic parties) may differ systematically from municipalities where this is not the case (secular mayor)
- e.g., religiosity (conservative) is a confounder that may affect both treatment (islamic mayor) and outcome (educational attainment of women)

Basics (2)

$X_{i}$ : Score with $c$ as a known cutoff
- also called running variable, forcing variable or index
$Z_{i}$ = Assignment variable; $1 \text{ if } X_{i} \geq c, 0\text{ otherwise}$
$D_{i}$ : Treatment actually received
Q: Can you give an example of $X_{i}$ , $Z_{i}$ and $D_{i}$ for a concrete person, e.g., thinking of access to a Bachelor program?

We observe outcome under control condition for units below cutoff and outcome under treatment condition for units above cutoff

Basics (3): Sharp vs. Fuzzy RDD

Important: Assignment to treatment $Z_{i}$ (e.g., being offered scholarship) not necessarily $=$ receiving (complying with) treatment $D_{i}$ (e.g., taking the scholarship)
Sharp RD: $Z_{i} = D_{i}$ , all units comply with treatment condition they have been assigned to
Fuzzy RD: $Z_{i} \neq D_{i}$ , some units fail to receive treatment despite having a score above cutoff (and vice versa)

Q: Explain sharp and fuzzy RD in the case of a Bachelor program were cutoff is a certain grade (1.4)?

Basics (4)

Observed average outcome given the score & potential outcomes

$\mathbb{E}[Y_{i}|X_{i}] = \begin{cases} \mathbb{E}[Y_{i}(0)|X_{i}] & \quad \text{if } X_{i} < c\\ \mathbb{E}[Y_{i}(1)|X_{i}] & \quad \text{if } X_{i} \geq c \end{cases}$

Sharp RD design exhibits extreme case of lack of common support on $X$
- Units in the control and treatment groups cannot have the same value of score $X$
- This highlights that RD analysis fundamentally relies on extrapolation towards cutoff point $c$
Central goal of empirical RD analysis
- Adequately perform (local) extrapolation in order to compare control and treatment units
Basic assumption of comparability
- Units with very similar values of the score but on opposite sides of the cutoff are comparable

Basics (5)

Fund. problem of causal inference: Only observe the outcome under control, $Y_{i}(0)$ , for those units whose score is below the cutoff $c$ , and the outcome under treatment, $Y_{i}(1)$ , for those units whose score is above the cutoff $c$
Fig. 3 plots the average potential outcomes given the score, $E[Y_{i}(1)|X_{i} = x]$ and $E[Y_{i}(0)|X_{i} = x]$ , against the score
We can estimate the regression function $E[Y_{i}(1)|X_{i}]$ for values of the score to the right of the cutoff because we observe $Y_{i}(1)$ , for every $i$ when $X \geq c$ (solid red line)
- And $E[Y_{i}(0)|X_{i}]$ for values left to the cutoff (solid blue line)
Sharp RD treatment effect: $\tau_{SRD} = \mathbb{E}[Y_{i}(1) - Y_{i}(0)|X_{i} = c]$
- Conceptually: what would be the average outcome change for units with score level $X_{i} = c$ if we switched their status from control to treated?

Example: Meyersson (2014) (1)

Meyersson (2014): Islamic Rule and the Empowerment of the Poor and Pious
- Research question: What is the impact of having a mayor from an Islamic party on women’s rights (educational attainment of young women)?
- $\text{Units }i$ : Municipalities
- $Y_{i}$ : educational attainment of women, measured as the percentage of women aged 15 to 20 in 2000 who had completed high school by 2000
- $X_{i}$ : vote margin obtained by the Islamic party in the 1994 Turkish mayoral elections, measured as the vote percentage obtained by the Islamic party minus the vote percentage obtained by its strongest secular party opponent
- $D_{i}$ ( $T$ in the paper): electoral victory of the Islamic party in 1994, equal to 1 if the Islamic party won the mayoral election and 0 otherwise $(D_{i} = Z_{i})$
Methodological challenge: Municipalities where support for Islamic parties is high enough to result in the election of an Islamic mayor (of one of two islamic parties) may differ systematically from municipalities where this is not the case (secular mayor)
- e.g., religiosity (conservative) is a confounder that may affect both treatment (islamic mayor) and outcome (educational attainment of women)

Local nature of RD effects (1)

Sharp RD parameter = captures average difference in potential outcomes under treatment versus control
- This average difference is calculated at a single point $(c)$ on the support of a continuous random variable (the score $X_{i}$ )
- Captures a causal effect that is local in nature in contrast to other causal parameters in the potential outcomes framework
RD treatment effect has limited external validity
- is (often) not representative of treatment effects that would occur for units with scores away from the cutoff
Meyersson example: Lack of external validity reflected in the focus on close and not all elections
- Figure 3(a) [next slide]: Educational attainment of women seems higher in municipalities where Islamic party looses
- Figure 3(b) [next slide]: Close to $c$ educ. attainment seems higher in municipalities where Islamic barely wins (zoom in + fourth-orderpolynomial fit)
- By definition: sample of municipalities near the cutoff comprises subset of constituencies where Islamic party is very competitive (localness!)
  - …raises the question of how representative they are of wider sample

Estimation: Continuity-Based Approach (1)

Two approaches to estimation: randomization-based vs. continuity-based approach (we focus on the latter!)
Assumption of comparability formalized by Hahn et al. (2001) using continuity assumptions
- $\mathbb{E}[Y_{i}(1) - Y_{i}(0)|X_{i} = c] = \lim_{x\downarrow c} \mathbb{E}[Y_{i}|X_{i} = x] - \lim_{x\uparrow c} \mathbb{E}[Y_{i}|X_{i} = x]$
- They show (in words): If the average potential outcomes are continuous functions of the score at $c$ (cf. Fig. 4, previous slide), the difference between the limits of the treated and control average observed outcomes – as the score converges to the cutoff – is equal to the average treatment effect at the cutoff
Continuity: As score $x$ gets closer to cutoff $c$ , the average potential outcome function $\mathbb{E}[Y_{i}(0)|X_{i} = x]$ gets closer to its value at the cutoff $\mathbb{E}[Y_{i}(0)|X_{i} = c]$ (same for $\mathbb{E}[Y_{i}(1)|X_{i} = x]$ )
- “key RD identifying assumption is the continuity (or lack of abrupt changes) of the regression functions for treatment and control units at the cutoff in the absence of the treatment” (Cattaneo et al. 2019, 100)
- Continuity: Formal justification for estimating Sharp RD effect by focusing on small neighborhood around cutoff $c$
In contrast, randomization-based approach explicitly assumes that RDD induces randomized experiment in window near the cutoff (local randomization assumption)

Estimation: Continuity-Based Approach (2)

Local polynomial point estimation: Estimate regression functions within bandwith $h$ around score $c$
- This estimation approach is nonparametric because it does not assume a particular parametric form of the regression functions (we just fit a curve to the data)
Challenge: the unknown regression functions $\mathbb{E}[Y_{i}(0)|X_{i}=x]$ and $\mathbb{E}[Y_{i}(1)|X_{i}=x]$ have to be approximated by a polynomial function of the score
Local polynomial methods implement linear regression fits using only observations near the cutoff point, separately for control and treatment units
- We localize polynomial fit to the cutoff (discarding observations sufficiently far away) and usually employ a low-order polynomial approximation (usually linear or quadratic)
- We only use observations between $c − h$ and $c + h$ , where $h > 0$ is so-called bandwidth, i.e., $h$ determines size of neighborhood around cutoff
- We commonly adopt weighting scheme within bandwidth to ensure observations closer to $c$ receive more weight (weights determined by a kernel function $K(·)$ )
Statistical properties of local polynomial estimation and inference depend crucially on the accuracy of the approximation near the cutoff (bandwidth matters!)

Steps: Local Polynomial Point Estimation

Choose a polynomial order $p$ and a kernel function $K(\cdot)$ .
- $K(\cdot)$ assigns weights to each transformed observation $(K(\frac{X_{i}−c}{h}))$
- Choice of $p$ is more consequential $(p$ = 0 has undesirable properties; higher $p$ improves accuracy of approximation but increases variabiltiy of estimator + danger of overfitting; preference for local linear RD with $p=1)$
Choose a bandwidth $h$ around $c$ (see Fig 14, Cattaneo et al. 2019, 46).
For observations above the cutoff, fit a weighted least squares regression of the outcome $Y_{i}$ on a constant and $(X_{i}−c),(X_{i}−c)^{2},...,(X_{i}−c)^{p}$ , where $p$ is the chosen polynomial order, with weight $K(\frac{X_{i}−c}{h})$ for each observation. The estimated intercept from this local weighted regression, $\hat{\mu}_{+}$ , is an estimate of the point $\mu_{+}=\mathbb{E}[Y_{i}(1)|X_{i}=c]$ .
- Similar for observations below the cutoff (cf. see Cattaneo et al. 2019, 42 also for equation)
Calculate the Sharp RD point estimate: $\hat{\tau}_{SRD} = \hat{\mu}_{+} − \hat{\mu}_{-}$

Graphically illustrated in Figure 12 on the next slide.

Estimation: Local Polynomial Point Estimation

Example Figure 12
- Polynomial of order one $(p = 1)$ is fit within bandwidth $h_{1}$
  - Observations outside $h_{1}$ are not used
- RD effect is $\tau_{SRD}=\mu_{+} − \mu_{-}$ and local polynomial estimator of this effect is $\hat{\tau}_{SRD} = \hat{\mu}_{+} − \hat{\mu}_{-}$
- Important: Graph shows binned observations!
Implementation requires choice of three main ingredients (Cattaneo et al. 2019, 43-51, Fig. 13/14)
- The kernel function $K(·)$ , the order of the polynomial $p$ , and the bandwidth $h$
- Using local linear estimation with bandwidth $h$ and uniform kernel is equivalent to estimating simple linear regression without weights for observations within $h$

Test: Placebo cutoffs (3)

Key RD identifying assumption continuity (or lack of abrupt changes) at the cutoff in the absence of treatment
- Test whether regression functions for control and treatment units are continuous at points other than the cutoff $c$
- Discontinuities away from the cutoff would cast doubt on the RDD
Q: When would the data fail this test and what could it mean substantively? (Cattaneo et al. 2019, 103, Fig. 20)

Further examples

Do Harsher Prison Conditions Reduce Recidivism? (Chen and Shapiro 2007)
- Treatment $D$ : Prison conditions
- Outcome $Y$ : Recidivism rates
- Score $X$ : “Upon entry to the federal prison system, an inmate is processed […] individual’s security custody score. The score is intended to predict prisoner misconduct and therefore to measure the supervision needs of individuals.” (Chen and Shapiro 2007, 5)
- Cutoff $c$ : Particular security custody score
Racial Profiling and Use of Force in Police Stops: How Local Events Trigger Periods of Increased Discrimination (Legewie 2016)
- “I argue that racial bias in the use of force increases after relevant events such as the shooting of a police officer by a black suspect. To examine this argument, I design a quasi experiment using data from 3.9 million time and geocoded pedestrian stops in New York City. The findings show that two fatal shootings of police officers by black suspects increased the use of police force against blacks substantially in the days after the shootings.”
- Treatment $D$ : Fatal shooting of police officer by black suspect
- Outcome $Y$ : Racial bias in the use of force
- Score $X$ : Time
- Cutoff $c$ : Time of the shooting (particular point in time)
Criticism & warnings: Gelman & Imbens (2019); Review of political science research by Stommes, Aronow, Sävje, 2021

First-difference (FD) estimator

First-difference (FD) estimator (see e.g. Gangl 2010, 33-35)
- $\Delta y_{it} = y_{it} - y_{it-1} = \theta \Delta d_{it} + \beta \Delta x_{it} + \Delta \varepsilon_{it}$
  - $i$ is the index for individuals, $t$ is the index for the time points $(t_{0}, t_{1},t_{2}\text{, etc.})$
  - $\Delta y_{it}$ is the differenced outcome variable (between $t_{0}$ and $t_{1}$ etc.)
  - $\theta$ is the treatment variable coefficient (estimate of the “causal” effect)
  - $\Delta d_{it}$ is the differenced treatment variable $d$
  - $\beta \Delta x_{it}$ one differenced covariate $\Delta x_{it}$ and its effect/coefficient $\beta$
    - …with more covariates we would have a matrix $\Delta X_{it}$ of differenced covariates and a vector of $\beta$ s
  - $\Delta \varepsilon_{it}$ is the differenced error term
- Constant covariates drop out through differencing (cf. top two equations here)
  - e.g., if $\Delta x_{it}$ is constant, difference is 0 and $\beta \Delta x_{it}$ is 0

Fixed-effects (FE) estimator

Fixed-effects (FE) estimator (see e.g. Gangl 2010, 33-35)
- $y_{it}-{\overline {y_{i}}} = \theta \left(d_{it} - {\overline {d_{i}}}\right) + \beta \left(x_{it} - {\overline {x_{i}}}\right) + \left(\varepsilon_{it} - {\overline {\varepsilon_{i}}}\right)$
  - $i$ is the index for individuals, $t$ is the index for the time points $(t_{0}, t_{1}, t_{2}$ etc.)
  - $y_{it}-{\overline {y_{i}}}$ is the de-meaned outcome variable $(\overline {y_{i}}$ = individual mean across time)
  - $\theta \left(d_{it} - {\overline {d_{i}}}\right)$ is the de-meaned treatment variable $\left(d_{it} - {\overline {d_{i}}}\right)$ and its coefficient $\theta$ (estimated “causal effect”)
  - $\beta \left(x_{it} - {\overline {x_{i}}}\right)$ is one de-meaned covariate $\left(x_{it} - {\overline {x_{i}}}\right)$ and its effect $\beta$
    - …with more covariates we have a matrix of de-meaned covariates and a vector of $\beta$ s
  - $\left(\varepsilon_{it} - {\overline {\varepsilon_{i}}}\right)$ is a de-meaned error term
    - …sometimes error terms for unobserved constant variables are added to the equation to show that they drop out (e.g., see example here).
  - Q: What happens to covariates that are constant across time?

Understanding first-differences and de-meaning

Q: How would you calculate the $\Delta y_{it}$ , $\Delta d_{it}$ (first differences), $y_{it}-{\overline {y_{i}}}$ and $d_{it} - {\overline {d_{i}}}$ (de-meaned) for Brittany’s trust in the Table below? (same logic for covariates!)

Some wide format panel data
Name	victim.2006	victim.2007	victim.2008	trust.2006	trust.2007	trust.2008
Brittany	0	0	1	4	4	2
Ethan	1	1	0	5	6	4
Kyle	0	0	0	0	7	5
Jacob	0	1	1	5	3	6
Jessica	0	0	0	7	9	4

Q: What are the theoretically possible min. and max. values of the differenced outcome variable Y (0-10) and treatment variable D (0,1)?
Q: What happens to variables that are stable over time? (Think of the difference!)
Q: What are we comparing here between treatment and control as opposed to cross-sectional data (Hint: Change)?
Q: What should we control for here since stable covariates drop out of the equation?
Q: If we look at change for both treatment & outcome between $t_{0}$ and $t_{1}$ , what additional assumptions are we making (Hint: Temporal order)?

First-differenced and de-meaned data

Data is simply the differences (change scores) between time points (see below): trust.06.07 = trust.2007 - trust.2006
First-differenced: Individual’s change from $t_{0}$ to $t_{1}$ etc. for treatment, outcome and covariates

Data change scores/first differences (wide-format)
Name	trust.06.07	trust.07.08	victim.06.07	victim.07.08
Brittany	0	-2	0	1
Ethan	1	-2	0	-1
Kyle	7	-2	0	0
Jacob	-2	3	1	0
Jessica	2	-5	0	0

Data is de-meaned for fixed effects approach (see below): trust.2006.dem = trust.2006 - trust.mean
De-meaned = individual’s deviation for each time point from its mean across time points (same for covariates)

Data de-meaned (wide-format)
Name	trust.2006.dem	trust.2007.dem	trust.2008.dem	victim.2006.dem	victim.2007.dem	victim.2008.dem
Brittany	0.67	0.67	-1.33	-0.33	-0.33	0.67
Ethan	0.00	1.00	-1.00	0.33	0.33	-0.67
Kyle	-4.00	3.00	1.00	0.00	0.00	0.00
Jacob	0.33	-1.67	1.33	-0.67	0.33	0.33
Jessica	0.33	2.33	-2.67	0.00	0.00	0.00

Assumptions (FD and FE models)

Advantage: Stable, i.e., time invariant (unobserved & observed) confounders drop out
Assumptions (standard): Selection only on observable time-variant covariates/confounders (Q: ?)
- FE estimator identifies average treatment effect on the treated if exogeneity of time-varying idiosyncratic errors can be maintained (Gangl 2010, 34)
- $+$ linearity and additivity of treatment effect (e.g. Keele 2015, 10) (examples for linearity, additivity etc.)
Focus on within-unit variation may dramatically change outcome variable and variation we are looking at: explore variation in Y after transformation! (Mummolo and Peterson 2018)
- Preprocessing data: e.g., omit observations in which units change from treatment to control
FE and FD should be equivalent for $T = 2$ (e.g., see here)
BUT causal inference with panel data + within-comparisons is an ongoing research field (Imai & Kim 2019)
- Assumptions necessary for causal inference not entirely clear
- Results often misinterpreted (checklist for interpretation!)

Applications

Imai et al. (2020) motivate their study using two applications
Acemoglu, D., Naidu, S., Restrepo, P., & Robinson, J. A. (2019). Democracy does cause growth. Journal of Political Economy, 127(1), 47-100.
- “We provide evidence that democracy has a positive effect on GDP per capita. Our dynamic panel strategy controls for country fixed effects and the rich dynamics of GDP, which otherwise confound the effect of democracy. To reduce measurement error, we introduce a new indicator of democracy that consolidates previous measures. Our baseline results show that democratizations increase GDP per capita by about 20 percent in the long run. We find similar effects using a propensity score reweighting strategy as well as an instrumental-variables strategy using regional waves of democratization. The effects are similar across different levels of development and appear to be driven by greater investments in capital, schooling, and health.”
- $X_{it}$ (= $D_{it}$ ): 1 = “Free” or “Partially Free” in Freedom House + positive Polity IV index score, 0 = otherwise, $Y_{it}$ : GDP per capita (logged)
Scheve, Kenneth, and David Stasavage. “Democracy, war, and wealth: lessons from two centuries of inheritance taxation.” American Political Science Review (2012): 81-102.
- “In this article we use an original data set to provide the first empirical analysis of the political economy of inherited wealth taxation that covers a significant number of countries and a long time frame (1816–2000). Our goal is to understand why, if inheritance taxes are often very old taxes, the implementation of inheritance tax rates significant enough to affect wealth inequality is a much more recent phenomenon. We hypothesize alternatively that significant taxation of inherited wealth depended on (1) the extension of the suffrage and (2) political conditions created by mass mobilization for war. Using a difference-in-differences framework for identification, we find little evidence for the suffrage hypothesis but very strong evidence for the mass mobilization hypothesis. Our study has implications for understanding the evolution of wealth inequality and the political conditions under which countries are likely to implement policies that significantly redistribute wealth and income.”
- $X_{it}$ (= $D_{it}$ ): 1 = inter-state war, 0 = no war, $Y_{it}$ : Top rate of inheritance taxation

Leads & lags specification

Specifying causal quantity of interest requires specifying leads and lags
Leads: Choose the number of leads $F$
- $F = 0$ represents the contemporaneous effect
- $F = 2$ implies the treatment effect on the outcome two time periods after the treatment is administered
- Specifying F > 0 allows for examining long-term (cumulative) effect
Lags: Specify how many previous time periods $L$ one wants to adjust (match) for
- e.g., match on the $L = 2$ lags history or $L = 5$ lags history
- Trade-off: Greater value improves credibility of causal identification (assumptions next week!) but reduces efficiency of estimates by reducing the number of potential matches
  - Q: Why? Example?
After selecting $F$ and $L$ we can specify quantity of interest

Causal quantity of interest (ATT)

Average treatment effect of policy (= treatment) change among the treated (ATT) (Imai et al. 2020, 11)
$\delta(F,L)=$ $E\{Y_{i,t+F}(X_{it}=1,X_{i,t−1} = 0,\{X_{i,t−\ell}\}^{L}_{\ell=2})$ $− Y_{i,t+F}(X_{it} = 0, X_{i,t−1}= 0,\{X_{i,t−\ell}\}^{L}_{\ell=2})|X_{it}= 1,X_{i,t−1}= 0\}$
- Treatment change: from $X_{i,t−1}$ = 0 to $X_{it} = 1$ $(X = D)$
- $Y_{i,t+F}(X_{it} = 1,X_{i,t−1} = 0,\{X_{i,t−\ell}\}^{L}_{\ell=2})$ : Potential outcome under treatment/policy change
- $Y_{i,t+F}(X_{it} = 0, X_{i,t−1}= 0,\{X_{i,t−\ell}\}^{L}_{\ell=2})$ : Potential outcome without treatment/policy change
- $\{X_{i,t−\ell}\}^{L}_{\ell=2} = \{X_{i,t−2}, ...,X_{i,t−L}\}$ : Rest of the treatment history that is set to realized history
  - $t−\ell\text{ with } \ell=2$ : Treatment history backwards from t-2 (two time points before)
- Example: $\delta(1,5)$ represents the average causal effect of policy change on the outcome one time period after the treatment while assuming that the potential outcome only depends on the treatment history up to five time periods back (Q: And $\delta(2,3)$ = ?)
The above causal quantity allows for a future treatment reversal, i.e., treatment status could go back to the control condition before the outcome is measured, i.e., $X_{i,t+\ell}= 0$ for some with $1 \leq \ell \leq F$ (other definition possible)

Identification assumption(s) & choice of L, F

Identification assumptions
- Absence of spill-over effects (analogue to SUTVA)
- Limited carryover effect assumption (limited possibility that past treatments affect future outcomes)
- Parallel trend assumption after conditioning on the treatment, outcome, and covariate histories (see Imai et al. 2020, 12)
How should researchers choose the values of L and F?
Large value of $L$ improves the credibility of the limited carryover effect assumption
- …because it allows a greater number of past treatments (i.e., those up to time $t−L$ ) to affect the outcome of interest (i.e., $Y_{i,t+F}$ ) [i.e., matched set is picked accordingly]
- Q: What might be problematic about large values of $L$ ?
  - It may reduce the number of matches and yield less precise estimates!
  - Practice: Motivate choice of $L$ with substantive knowledge and examine sensitivity of estimates to different choices of $L$
$F$ should be motivated by interest in short-term or long-term causal effects:
- Q: What may be problematic about large values of $F$ ?
  - Large values of $F$ potentially problematic because units may switch treatment status during the $F$ lead time periods

Step 1: Constructing the matched sets

Q: What does the graph illustrate (left panel!)? Explain picking one or several of the highlighted treated observations, e.g., $(i,t) = (2,5)$
Q: What would the matches for observations $(i,t) = (3,5)$ and $(2,6)$ be in the left panel $[\delta(F = 0,L = 3)]$ ?
Q: What is shown in the right panel? What’s the problem with $(i,t) = (3,6)$ ?
Q: What is/are the matched set(s)?
Q: Non-matching (on treatment history) observations are discarded: What should we probably do after such an analysis? (analogue to matching & target population)

Step 2: Refining the Matched Sets I

In Step 1 we adjust for the treatment history.
To make the parallel trends assumption credible we should also adjust for other confounders such as past outcomes and (possibly time-varying) covariates
We (can) apply various matching methods (see Imai et al. 2020, 14; cf. Lecture 5 & 6)
- Calculation of distance measures may be slightly more complicated
  - e.g., Mahalanobis distance between the treated observation and each control observation over time (see Imai et al. 2020, 14)
  - e.g., Propensity score: Treatment assignment model (e.g. logistic regression) within data subsets that consist of all treated observations and their matched control observations from same year
  - e.g., exact matching (currently only for stable Covs in PanelMatch)
- Once distance measure is computed for all control units in the matched set → refine matched set by selecting up to $J$ most similar control units that satisfy a caliper constraint $C$ (give zero weight to other matched control units)

Step 3: Difference-in-Differences Estimator

Given refined matched sets, we estimate the ATT of treatment change in two steps

We estimate the counterfactual outcome $Y_{i,t+F}(X_{it}=0,X_{i,t−1}=0,X_{i,t−2},...,X_{i,t−L})$ using the weighted average of the control units in the refined matched set, for each treated observation $(i,t)$
Compute difference-in-differences estimate of the ATT for each treated observation and average it across all treated observations

See Imai et al. (2020, 16) for formal notation of estimator

Specifying the future treatment sequence (I)

Sometimes we are interested in a non-contemporaneous treatment effect (i.e., $F>0$ )
- What happens to treatment status between treatment and outcome measurement?
ATT as defined above (Imai et al. 2021, Eq 8, p.7) does not specify future treatment sequence
- Treated (and control) units may change their treatment status after time $t$ but before the outcome is measured at time $t+F$
Focus could be ATT of stable policy (treatment)
- Counterfactual scenario is that treated unit does not receive the treatment before the outcome is measured
- Modify ATT by specifying the future treatment sequence (with respect to counterfactual scenario of interest)
  - Treated (matched control) observations are those who remain under the treatment (control) condition throughout F time periods after the administration of the treatment

Specifying the future treatment sequence (III)

Assume estimation of ATT of stable policy treatment: Which units would be suitable control for observations for observation $(i,t) = (3,4)$ (treated at $t = 4$ ) with $\delta(F = 1,L = 2)$ ?
Unit 1 was not in control at $t = 4$ ❌
Unit 2, 4 and 5 were all in control at $t = 4$ ✅
Unit 2, 4 and 5 all share the same treatment history for $L = 2$ ✅
Unit 2 changes treatment status from $t = 4$ to $t = 5$ ❌
Unit 4,5 remain in control until $t = 5$ ✅
Q: Which units would be suitable control for the unit $i = 3$ treated at $t = 4$ with $\delta(F = 1,L = 3)$ ?

Identification assumption(s): Update Lousdal 2018

Lousdal (2018, 2-4) provides one of the best summaries of IV assumptions (+ lucid explanations)
- Three basic assumptions (A1-3) and a fourth one (A4) [ $X$ replaced with $D$ on slides ]
A variable $Z$ is an instrument if it meets the following three assumptions:
- The relevance assumption (A1): The instrument $Z$ has a causal effect on $D$ .
- The exclusion restriction assumption (A2): $Z$ affects the outcome $Y$ only through $D$ .
- The exchangeability assumption (A3): $Z$ does not share common causes with the outcome $Y$
  - Also called independence assumption, ignorable treatment assignment, no confounding for effect of $Z$ on $Y$
The monotonicity or no defiers assumption (A4): There are no defiers (cf. Lousdal 2018, 3-4; Dunning 2009, 2704)
- If no defiers exist (A4), then the only subpopulation in which $Z$ can affect $D$ is the compliers, i.e., effect of $Z$ on $Y$ will only stem from the group of compliers
- A1-4 allow us to estimate local average treatment effect (LATE) also called complier average causal effect (CACE)
- Problem that remains (for decision makers): We can not identify the compliers individually (differentiate from always/never takers)

Estimation: Two stages

First stage: $\hat{D}_{i} = \alpha + \delta Z_{i} + \mathbf{X}_{i}\mathbf{\beta}$
- Regress treatment $D$ on instrument $Z$
Second stage: $y_{i} = \gamma + \theta \hat{D}_{i} + \mathbf{X}_{i}\mathbf{\beta} + \epsilon_{i}$
- Regress outcome $Y$ on the predicted values of the first regression
Conceptually:
- predicted values reflect the variation in treatment $D$ that is explained by the instrument $Z$
- this left-over variation is exogenous so we can estimate it’s causal effect on $Y$
Stages often implemented in single function, e.g. ivreg() in R

Limitations (and Strengths) (1)

Requires strong assumptions (can be partially validated with data)
As-if random (independence) IV assumption
- Show $Z$ to be uncorrelated with pretreatment covariates
- Use a priori reasoning and detailed knowledge of empirical context
- Observational studies: Often matter of opinion (“less plausible” to “more plausible”)
Additional issues with use of multiple regression models
- Concern about endogeneity of single treatment variable (counfounding)
  - Researchers add covariates to condition but forget to discuss their potential endogeneity
  - Recommendation: also report regression without covariates (the latter can be harmful)
- Truly random instruments may not show strong correlation with (endogenous) treatment
  - Small-sample bias can arise and its recommended to report “reduced-form” results (i.e., ITT effect namely $D$ on $Z$ )
- Generally, be careful in extrapolating results and validate the linear regression model itself

Limitations (and Strengths) (2)

Important!
- Neither core criteria for a valid instrumental variable, i.e., …
  - … $Z$ is statistically independent of unobserved causes of the dependent variable and…
  - …affects the dependent variable only through its effect on the endogenous treatment $D$ ,…
- …are directly testable from data
We have to defend these assumptions using evidence and reasoning but be careful especially outside of experimental studies!

IV questions

Why might we want to use IV? (e.g., vs. matching + conditional independence assumption)
- Kern & Hainmueller 2009 Example
  - West German television exposure $D$ $\rightarrow$ support for East German regime $Y$
  - Instrument: Living in Dresden $Z$
- Reverse causation $D \leftarrow Y$
  - e.g., people do or do not seek access to West German television because of their low or high support of the East German regime
- Confounding $D \leftarrow X \rightarrow Y$
  - e.g., having relatives in West Germany $X$ might affect both one’s West German television exposure $D$ and support for East German regime $Y$ (we can think of different mechanisms)

Identification assumption(s): Update Lousdal 2018

Lousdal (2018, 2-4) provides one of the best summaries of IV assumptions (+ lucid explanations)
- Three basic assumptions (A1-3) and a fourth one (A4) [ $X$ replaced with $D$ on slides ]
A variable $Z$ is an instruments if it meets the following three assumptions:
- The relevance assumption (A1): The instrument $Z$ has a causal effect on $D$ .
- The exclusion restriction assumption (A2): $Z$ affects the outcome $Y$ only through $D$ .
- The exchangeability assumption (A3): $Z$ does not share common causes with the outcome $Y$
  - Also called independence assumption, ignorable treatment assignment, no confounding for effect of $Z$ on $Y$
The monotonicity or no defiers assumption (A4): There are no defiers (cf. Lousdal 2018, 3-4; Dunning 2009, 2704)
- If no defiers exist (A4), then the only subpopulation in which $Z$ can affect $D$ is the compliers, i.e., effect of $Z$ on $Y$ will only stem from the group of compliers
- A1-4 allow us to estimate local average treatment effect (LATE) also called complier average causal effect (CACE)
- Problem that remains (for decision makers): We can not identify the compliers individually (differentiate from always/never takers)

Identification assumption(s): Examples

The relevance assumption (A1): The instrument $Z$ has a causal effect on $D$
- “Given that most people in the Dresden district were cut off from West German television broadcasts, not living in the Dresden district is highly correlated with exposure to West German television.” (Kern & Hainmueller 2009, 385)
The exclusion restriction assumption (A2): $Z$ affects the outcome $Y$ only through $D$ .
- Threat: Alternative mechanisms/paths, e.g., Living in Dresden $Z$ $\rightarrow$ Living conditions $X$ $\rightarrow$ Political support $Y$
- See Kern & Hainmueller (2009, Section 3.2.2) for arguments why assumption holds condition on covariates. (highlight similarity of Dresden and control districts in Fig.2)
The exchangeability assumption (A3): $Z$ does not share common causes with the outcome $Y$
- Threat: “East Germans who desired to watch West German television might have moved away from the Dresden district. If interest in West German television was correlated with regime support, which seems likely, this kind of sorting behavior would invalidate our instrument” (Kern & Hainmueller 2009, 386)
- See Kern & Hainmueller (2009, Section 3.2.3) for arguments why assumption holds condition on covariates (highlight low mobility).
The monotonicity or no defiers assumption (A4): There are no defiers (cf. Lousdal 2018, 3-4; Dunning 2009, 2704)
- “It is highly unlikely that there were East Germans who would have watched West German television if they had lived in Dresden but who would not have watched West German television if they had not lived in Dresden.” (Kern & Hainmueller 2009, 384)

Homework (1): Screening & Breast cancer

1960s: Health Insurance Plan (HIP) clinical trial studied effects of screening $D$ for breast cancer $Y$ (Dunning 2009)
- Assigned-to-treatment group (or treatment group): ~31,000 women between 40 and 64 years invited for annual clinical visits and mammographies (X-rays designed to detect breast cancer)
- Control group: ~31,000 women received status quo health care
Instrument: Invitation for screening $Z$ issued at random
- Women in assigned-to-treatment group were just like the women who were not, up to random error
Table 1 (next slide) shows death rates from breast cancer 5 years after trial
Homework: Read through Dunning’s (2009) example and explain the different comparisons we can make and the logic of IV analysis using Table 1 below.
- Exemplary calculation assuming presence of Compliers and Never Takers (but absence of Always Takers/Defiers)

Homework (3): Screening & Breast cancer

Naive comparison (red): Compare women who received screening with those that refused (Est.: -0.34)
- Problem: Self-selection into treatment, i.e., women who screened systematically different (richer, better-educated)
- Problem: richer, better-educated more prone to breast cancer [Explanation: fewer children]
Correct, experimental comparison (orange): Compare women randomly invited (whether or not they were actually screened) to the whole control group
- Intention-to-treat analysis: Shows strong effect in relative terms (effect: -0.77)
  - 1.26 (assignment-to-treatment group) - 2.03 (control group)
  - Probably understates effect because one third (10800) in treatment group not screened
What was the effect of screening on women in the treatment group who accepted screening (green)?
- Control group (total): Can not differentiate compliers/never takers (neither receives treatment when assiged to control) [we assume only the two groups - compliers & never takers are present]
- Treatment group: ~2/3 (20200) accepted screening $\rightarrow$ because of random assignment mix of never takers and compliers should be same in control group (~2/3 would have accepted screening in both)
- IV analysis compares death rates of compliers in treatment [1, green, observable!] and control [2]
- Calculate [2]: Rate of NTs same in both groups (blue circle, 16) -> 63 - 16 = 47 dead compliers in control group
  - Calculate death rates of compliers in treatment and control: (23/20200)*1000 = 1.14; (47/20200)*1000 = 2.33
  - LATE/CATE: 1.14 - 2.33 = -1.19
Assumption(s): No Always Takers, i.e., no women screened in the control group (we could adjust for double cross-over!) and absence of defiers

IV questions

Why might we want to use IV? (e.g., vs. matching + conditional independence assumption)
- Kern & Hainmueller 2009 Example
  - West German television exposure $D$ $\rightarrow$ support for East German regime $Y$
  - Instrument: Living in Dresden $Z$
- Reverse causation $D \leftarrow Y$
  - e.g., people do or do not seek access to West German television because of their low or high support of the East German regime
- Confounding $D \leftarrow X \rightarrow Y$
  - e.g., having relatives in West Germany $X$ might affect both one’s West German television exposure $D$ and support for East German regime $Y$ (we can think of different mechanisms)

Identification assumption(s): Update Lousdal 2018

Lousdal (2018, 2-4) provides one of the best summaries of IV assumptions (+ lucid explanations)
- Three basic assumptions (A1-3) and a fourth one (A4) [ $X$ replaced with $D$ on slides ]
A variable $Z$ is an instruments if it meets the following three assumptions:
- The relevance assumption (A1): The instrument $Z$ has a causal effect on $D$ .
- The exclusion restriction assumption (A2): $Z$ affects the outcome $Y$ only through $D$ .
- The exchangeability assumption (A3): $Z$ does not share common causes with the outcome $Y$
  - Also called independence assumption, ignorable treatment assignment, no confounding for effect of $Z$ on $Y$
The monotonicity or no defiers assumption (A4): There are no defiers (cf. Lousdal 2018, 3-4; Dunning 2009, 2704)
- If no defiers exist (A4), then the only subpopulation in which $Z$ can affect $D$ is the compliers, i.e., effect of $Z$ on $Y$ will only stem from the group of compliers
- A1-4 allow us to estimate local average treatment effect (LATE) also called complier average causal effect (CACE)
- Problem that remains (for decision makers): We can not identify the compliers individually (differentiate from always/never takers)

Identification assumption(s): Examples

The relevance assumption (A1): The instrument $Z$ has a causal effect on $D$
- “Given that most people in the Dresden district were cut off from West German television broadcasts, not living in the Dresden district is highly correlated with exposure to West German television.” (Kern & Hainmueller 2009, 385)
The exclusion restriction assumption (A2): $Z$ affects the outcome $Y$ only through $D$ .
- Threat: Alternative mechanisms/paths, e.g., Living in Dresden $Z$ $\rightarrow$ Living conditions $X$ $\rightarrow$ Political support $Y$
- See Kern & Hainmueller (2009, Section 3.2.2) for arguments why assumption holds condition on covariates. (highlight similarity of Dresden and control districts in Fig.2)
The exchangeability assumption (A3): $Z$ does not share common causes with the outcome $Y$
- Threat: “East Germans who desired to watch West German television might have moved away from the Dresden district. If interest in West German television was correlated with regime support, which seems likely, this kind of sorting behavior would invalidate our instrument” (Kern & Hainmueller 2009, 386)
- See Kern & Hainmueller (2009, Section 3.2.3) for arguments why assumption holds condition on covariates (highlight low mobility).
The monotonicity or no defiers assumption (A4): There are no defiers (cf. Lousdal 2018, 3-4; Dunning 2009, 2704)
- “It is highly unlikely that there were East Germans who would have watched West German television if they had lived in Dresden but who would not have watched West German television if they had not lived in Dresden.” (Kern & Hainmueller 2009, 384)

Homework (1): Screening & Breast cancer

1960s: Health Insurance Plan (HIP) clinical trial studied effects of screening $D$ for breast cancer $Y$ (Dunning 2009)
- Assigned-to-treatment group (or treatment group): ~31,000 women between 40 and 64 years invited for annual clinical visits and mammographies (X-rays designed to detect breast cancer)
- Control group: ~31,000 women received status quo health care
Instrument: Invitation for screening $Z$ issued at random
- Women in assigned-to-treatment group were just like the women who were not, up to random error
Table 1 (next slide) shows death rates from breast cancer 5 years after trial
Homework: Read through Dunning’s (2009) example and explain the different comparisons we can make and the logic of IV analysis using Table 1 below.
- Exemplary calculation assuming presence of Compliers and Never Takers (but absence of Always Takers/Defiers)

Homework (3): Screening & Breast cancer

Naive comparison (red): Compare women who received screening with those that refused (Est.: -0.34)
- Problem: Self-selection into treatment, i.e., women who screened systematically different (richer, better-educated)
- Problem: richer, better-educated more prone to breast cancer [Explanation: fewer children]
Correct, experimental comparison (orange): Compare women randomly invited (whether or not they were actually screened) to the whole control group
- Intention-to-treat analysis: Shows strong effect in relative terms (effect: -0.77)
  - 1.26 (assignment-to-treatment group) - 2.03 (control group)
  - Probably understates effect because one third (10800) in treatment group not screened
What was the effect of screening on women in the treatment group who accepted screening (green)?
- Control group (total): Can not differentiate compliers/never takers (neither receives treatment when assiged to control) [we assume only the two groups - compliers & never takers are present]
- Treatment group: ~2/3 (20200) accepted screening $\rightarrow$ because of random assignment mix of never takers and compliers should be same in control group (~2/3 would have accepted screening in both)
- IV analysis compares death rates of compliers in treatment [1, green, observable!] and control [2]
- Calculate [2]: Rate of NTs same in both groups (blue circle, 16) -> 63 - 16 = 47 dead compliers in control group
  - Calculate death rates of compliers in treatment and control: (23/20200)*1000 = 1.14; (47/20200)*1000 = 2.33
  - LATE/CATE: 1.14 - 2.33 = -1.19
Assumption(s): No Always Takers, i.e., no women screened in the control group (we could adjust for double cross-over!) and absence of defiers

Effect size (1)

Effect size: In causal research effect often = difference in means of $Y$ between treatment and control group
- Q: Is it easier to judge difference (effect size) on income scale (in euros) or on a trust scale (4 levels)?

–

Yes, income!
Examples (in my own research):
- Victimization (0,1) on generalized trust (0-10): ~0.6 (naive estimate)
- Fake vs. real source (Nachricten 360/Tagesschau) on belief (0-6): 0.62 (comparable to similar studies)
“Vague” scales
- Approach 1: What effect sizes did previous studies with the same outcome scale identify?
- Approach 2: What are the biggest differences we can observe on that outcome scale?
  - …between units (e.g., males and females)
  - …within units (e.g., individual’s drop in life satisfaction scale after child’s death)

Effect uncertainty (1)

Objective: estimate causal effect $\theta$ , e.g., effect of education on income in a (target) population $P$
- $P$ could be Mannheim students, German citizens, immigrants etc.
- Steps: Define population $P$ → collect sample $S$ from population → produce estimate $\hat{\theta}$
- Estimate comes with uncertainty, i.e., we don’t know how far $\hat{\theta}$ is from true population parameter $\theta$
Dominant perspective: Uncertainty about population parameters is induced by random sampling from the population
- Typically assume that sample comprises only a small fraction of the population of interest
Sampling perspective…
- …makes sense when data can be regarded as small random subset of population of interest (e.g., ALLBUS data)
- …makes less sense when we have data on the whole population (e.g., comparing countries): then we know our estimates with certainty!

Effect uncertainty (2)

Frequentist paradigm: In theory, we draw repeated samples $(S_{1}, S_{2}, S_{3}, ...)$ from population $P$
In each sample we calculate the quantity of interest resulting in a vector of estimates: $(\hat{\theta}_{S1}, \hat{\theta}_{S2}, \hat{\theta}_{S3}, ...)$
- The distribution of these estimates is called sampling distribution…
- …and is the distribution of the $\hat{\theta}'s$ that we get across these different samples
Central limit theorem (CLT): Roughly states that sampling distribution (given certain conditions) approaches a normal distribution
In reality, we normally only observe one sample but we can still assume (CLT!) that the sampling distribution would look like a normal distribution (or a t-distribution)
Classic standard errors (SEs) [and also confidence intervals]: are based on this frequentist logic and are designed to capture sampling variation
- e.g., “sample mean’s standard error is the standard deviation of the set of means that would be found by drawing an infinite number of repeated samples from the population and computing a mean for each sample” (Wikipedia)

Effect uncertainty (3): Standard errors

Standard error (SE): the standard deviation (SD) of the sampling distribution of a statistic, e.g., of our causal effect of interest
- Standard deviation (SD) is a measure of the amount of variation or dispersion of a set of values
Sample mean as ‘simple’ example: Mean $\bar{x}$ is quantity of interest and the samples are uncorrelated
- Standard error of the sample mean (also called SEM): SE = $\sigma_{\bar{x}}={\frac{\sigma }{{\sqrt{n}}}}$
  - $\sigma$ $(=\sqrt{\sigma^{2}})$ ): standard deviation of the population distribution of $x$
  - $n$ : number of units in the sample (sample size)
- $\sigma$ seldomly known so we replace it with the sample standard deviation $s = \sqrt{\frac{1}{n} \sum_{i=1}^{n} (x_{i} - \bar{x})^2}$
- Implication of formula $(\sqrt{n})$ : Sample size must be multiplied by 4 to achieve half the SE (cost-benefit tradeoffs)
If sampling distribution is normally distributed, sample mean, standard error, and quantiles of the normal distribution can be used to calculate confidence intervals for the true population mean
Same logic applies to other quantities of interest, e.g., difference in means (causal effect etc.), estimates from linear regression model etc.

Uncertainty: Sampling-based vs. design-based (1)

Sampling-based uncertainty: stems from the fact that we only observe a subset of the population (Abadie et al. 2020)
Table I: finite population consisting of $n$ units with each unit characterized by a pair of variables $Y_{i}$ and $Z_{i}$ , with inclusion of unit $i$ in a sample encoded by the binary variable $R_{i} \in \{0,1\}$

Sampling-based inference uses information about the process that determines the sampling indicators $R_{1}, ..., R_{n}$ to assess the variability of estimators across different samples (Table I shows three samples)

Uncertainty: Sampling-based vs. design-based (2)

Design-based uncertainty: “arises when the parameter of interest is defined in terms of the unobserved outcomes that some units would attain under a certain intervention” (Abadie et al. 2020, 266)
Table II: scenario in which we observe, for each unit in the population, the value of one of two potential outcome variables, either $Y^{∗}_{i}(1)$ or $Y^{∗}_{i}(0)$ ; $X_{i}\in\{0,1\}$ indicates which potential outcome we observe

Design-based inference uses information about the process that determines the assignments $X_{1}, ..., X_{n}$ to assess the variability of estimators across different samples (Table II shows three samples)

Internal vs. external validity

We can now also return to the concepts of internal vs. external validity
Abadie et al. (2020, 271)’s distinction between sampling-based and design-based uncertainty suggests a definition of these concepts
“Internal validity bears on the question of whether $E[\theta|\mathbf{R},N_{1},N_{0}]$ is equal to $\theta^{causal,sample}$ . This relies on random assignment of the treatment. Whether or not the sampling is random is irrelevant for this question because $\theta^{causal,sample}$ conditions on which units were sampled.” (Abadie et al. 2020, 271)
- $\mathbf{R}$ : Population $n$ -vector with $i$ th element equal to $R_{i}$ (units with $R_{i}=1$ are included in the sample)
- $N_{1},N_{0}$ : Number of units in treatment and control
“External validity bears on the question of whether $E[\theta^{causal,sample}|N_{1},N_{0}]$ equal to $\theta^{causal}$ [population!]. This relies on the random sampling assumption and does not require that the assignment is random” (Abadie et al. 2020, 271)
“However, for $\hat{\theta}$ to be a good estimator of $\theta^{causal}$ , which is often the most interesting estimand, we need both internal and external validity, and thus both random assignment and random sampling.” (Abadie et al. 2020, 271)

Effect Mediation & causal mechanisms (1)

Whether variable $A$ has a causal effect on variable $B$ vs. explain how causal relationship between $A$ and $B$ arises
Causal mechanism (CM): “a process in which a causal variable of interest, i.e., a treatment variable, influences an outcome” (Imai et al. 2011, 765)
- Identification of CM requires specification of an intermediate variable or a mediator $M$ that lies on the causal pathway between treatment $T$ (= $D$ ) and outcome variable $Y$
Q: Can you think of any examples of mediators $M$ that mediate a causal relationship between $D$ and $Y$ ?
Contributions (Imai et al. 2011, 766): “commonly used statistical methods [cf. Baron & Kenny 1986] rely upon untestable assumptions and are often inappropriate even under those assumptions” (Imai et al. 2011, 764)
- Present minimum set of assumptions required to quantify causal mediation effect under standard designs (experimental, observational) (and provide methods in R)
- Develop method of assessing the sensitivity of conclusions to potential violations of (untestable) key assumptions (-Sensitivity analysis formally quantifies degree to which empirical findings rely upon the key assumptions-)
- Offer alternative research designs that enable identification of causal mechanisms under less stringent assumptions

Effect Mediation & causal mechanisms (2)

Causal mechanism: “process whereby one variable $T$ [= $D$ ] causally affects another $Y$ through an intermediate variable or a mediator $M$ that operationalizes the hypothesized mechanism” (Imai et al. 2011, 768)
- e.g., Causal effect of $\text{media framing }(T)$ (neg. immigration vs. neutral story) on $\text{attitudes toward immigration }(Y)$ is transmitted by respondents’ $\text{anxiety }(M)$ (High vs. low) (Brader et al. 2008)
- Media effects may operate through changes in anxiety OR NOT
Inferential goal: Decompose the causal effect of $\text{media framing }(T)$ into the indirect effect, which represents the hypothesized causal mechanism, and the direct effect, which represents all the other mechanisms

Q: Where do we see indirect and direct effect in Fig. 1a on the right? What do Fig. 1b/c show?

Effect Mediation & causal mechanisms (3)

Example of Brader et al. (2008)
- $T$ : media framing; $t = \text{{0/neutral immigration story, 1/neg. immigration story}}$
- $M$ : level of anxiety; $m = \text{{0/low, 1/high}}$
- $Y$ : immigration attitude
$M_{i}(t)$ : potential value of mediator for unit $i$ under the treatment status $T_{i}=t$ (Q: $M_{i}(1)=?$ )
$Y_{i}(t,m)$ : potential outcome that would result if the treatment and mediating variables equal $t$ and $m$
- e.g., $Y_{i}(1,1)$ : potential outcome/opinion of individual $i$ given she has high anxiety and previously got the neg. immigration story (Q: $Y_{i}(0,1)=?$ )
Difference to usual situation
- The observed outcome $Y_{i}$ now depends on two things.. the treatment status and level of mediator observed under treatment status, i.e., $Y_{i}(T_{i},M_{i}(T_{i}))$

Effect Mediation & causal mechanisms (4)

Total unit treatment effects
- $\tau_{i}=Y_{i}(1,M_{i}(1))−Y_{i}(0,M_{i}(0))$ :
  - e.g., effect of media framing $T$ on immigration attitudes $Y$ (“ignoring” $M_{i}$ )
  - BUT we want to divide this into direct and indirect effect
Direct effects of the treatment for each unit $i$ and each treatment status $t=0,1$
- $\zeta_{i}(t)\equiv Y_{i}(1,M_{i}(t))−Y_{i}(0,M_{i}(t))$
Example: $\zeta_{i}(1)$ represents the difference between immigration opinions under treatment $(t = 1 = \text{0/neg. immigration story})$ and control $(t = \text{0/neutral immigration story})$ holding $M (\text{anxiety level})$ constant at the level that would be realized under treatment $(t = 1)$
- The direct effect $\zeta_{i}(t)$ equals the causal effect of the treatment $T$ on the outcome $Y$ that is not transmitted by the hypothesized mediator $M$
Q: What is $\zeta_{i}(0)$ ?

Effect Mediation & causal mechanisms (5)

Indirect causal mediation effects for each unit $i$ and each treatment status $t=0,1$
- $\delta_{i}(t)\equiv Y_{i}(t,M_{i}(1))−Y_{i}(t,M_{i}(0))$ :
Example: $\delta_{i}(1) = Y_{i}(1,M_{i}(1)) - Y_{i}(1,M_{i}(0))$ represents the difference between the two potential immigration opinions for subject $i$ under $t = 1 = \text{0/neg. immigration story}$ but changing the mediator $M$
- $Y_{i}(1,M_{i}(1))$ : is the (observable) immigration opinion if she views the neg. immigration news story
- $Y_{i}(1,M_{i}(0))$ : is her immigration opinion under the counterfactual scenario where subject $i$ still viewed the neg. immigration story but her anxiety level is as if the subject viewed the control (neutral) news story
- Difference between these two potential outcomes $(Y_{i}(1,M_{i}(1)) \text{ and } Y_{i}(1,M_{i}(0)))$ represents the effect of the change in the mediator that would be induced by the treatment, while the direct impact of the treatment is suppressed holding its value constant (here at $t = 1$ )
Q: $\delta_{i}(0)=?$

Effect Mediation: Estimation

Aim: Estimate…
- …average (indirect) causal mediation effects (ACME) $\bar\delta(t)$
- …average direct effects (ADE) $\bar\zeta(t)$
- ATE $\bar\tau$ equals the sum of ACME and ADE
- Goal is to decompose the ATE into the ACME and ADE and then assess the relative importance of the hypothesized mechanism
Key insight
- Both the direct and indirect effects contain a potential outcome that would never be realized under these designs
- Therefore neither quantity can be identified even in randomized experiments (where we randomize both $T$ and $M$ ), let alone observational studies
  - Under these designs the ATE is identified, but the ACME and ADE are not
- We need additional assumptions!

Effect Mediation: Identification assumptions (1)

Assumption 1: sequential ignorability (required under standard designs $\ast)$
- $\ast:$ Treatment assignment is either randomized or assumed to be random given the pretreatment covariates
- See Imai et al. (2011, 770) for formula of sequential ignorability assumption
- $X_{i}$ : Observed pretreatment confounders for unit $i$ (e.g., respondent’s gender and race in the media framing study)
sequential ignorability makes two ignorability assumptions sequentially
- 1. Given the observed pretreatment confounders, the treatment assignment is assumed to be statistically independent (ignorable) of potential outcomes and potential mediators
  - Assumption normally holds in experiments as in the Brader et al. (2008) example discussed here
  - Also called no-omitted-variable bias, unconfoundedness, etc.
- 1. Observed mediator is ignorable given the actual treatment status and pretreatment confounders
  - Example on next slide!

Effect Mediation: Identification assumptions (2)

Example for sequential ignorability (1) and (2)
- 1. because news stories are randomly assigned to subjects, the first part of Assumption 1 will hold even without conditioning on any pretreatment covariates $X_{i}$
- 1. implies that there are no unmeasured pretreatment or posttreatment covariates that confound the relationship between the levels of anxiety $M$ and immigration opinions $Y$
  - That is a strong assumption even in randomized experiments..
  - Would require measurement of complete set of covariates that affect both anxiety and immigration opinions, and they all must not be affected by the treatment
  - Violation, e.g., if both anxiety and immigration opinions are affected by fear disposition or ideology
If sequential ignorability holds we can identify ACME and ADE [+ Assumption 2: consistency assumption, see discussion in Imai et al. (2011, 782)]
Imai et al. (2011) also provide R code to conduct causal mediation analysis
- See package on github: https://github.com/kosukeimai/mediation

Appendix: Only relevant when discussed during the lecture

Effect Mediation: Estimation

Aim: Estimate…
- …average (indirect) causal mediation effects (ACME) $\bar\delta(t)$
- …average direct effects (ADE) $\bar\zeta(t)$
- ATE $\bar\tau$ equals the sum of ACME and ADE
- Goal is to decompose the ATE into the ACME and ADE and then assess the relative importance of the hypothesized mechanism
Key insight
- Both the direct and indirect effects contain a potential outcome that would never be realized under these designs
- Therefore neither quantity can be identified even in randomized experiments (where we randomize both $T$ and $M$ ), let alone observational studies
  - Under these designs the ATE is identified, but the ACME and ADE are not
- We need additional assumptions!

Effect Mediation: Identification assumptions (1)

Assumption 1: sequential ignorability (required under standard designs $\ast)$
- $\ast:$ Treatment assignment is either randomized or assumed to be random given the pretreatment covariates
- See Imai et al. (2011, 770) for formula of sequential ignorability assumption
- $X_{i}$ : Observed pretreatment confounders for unit $i$ (e.g., respondent’s gender and race in the media framing study)
sequential ignorability makes two ignorability assumptions sequentially
- 1. Given the observed pretreatment confounders, the treatment assignment is assumed to be statistically independent (ignorable) of potential outcomes and potential mediators
  - Assumption normally holds in experiments as in the Brader et al. (2008) example discussed here
  - Also called no-omitted-variable bias, unconfoundedness, etc.
- 1. Observed mediator is ignorable given the actual treatment status and pretreatment confounders
  - Example on next slide!

Effect Mediation: Identification assumptions (2)

Example for sequential ignorability (1) and (2)
- 1. because news stories are randomly assigned to subjects, the first part of Assumption 1 will hold even without conditioning on any pretreatment covariates $X_{i}$
- 1. implies that there are no unmeasured pretreatment or posttreatment covariates that confound the relationship between the levels of anxiety $M$ and immigration opinions $Y$
  - That is a strong assumption even in randomized experiments..
  - Would require measurement of complete set of covariates that affect both anxiety and immigration opinions, and they all must not be affected by the treatment
  - Violation, e.g., if both anxiety and immigration opinions are affected by fear disposition or ideology
If sequential ignorability holds we can identify ACME and ADE [+ Assumption 2: consistency assumption, see discussion in Imai et al. (2011, 782)]
Imai et al. (2011) also provide R code to conduct causal mediation analysis
- See package on github: https://github.com/kosukeimai/mediation

Effect Mediation: Estimation

Aim: Estimate…
- …average (indirect) causal mediation effects (ACME) $\bar\delta(t)$
- …average direct effects (ADE) $\bar\zeta(t)$
- ATE $\bar\tau$ equals the sum of ACME and ADE
- Goal is to decompose the ATE into the ACME and ADE and then assess the relative importance of the hypothesized mechanism
Key insight
- Both the direct and indirect effects contain a potential outcome that would never be realized under these designs
- Therefore neither quantity can be identified even in randomized experiments (where we randomize both $T$ and $M$ ), let alone observational studies
  - Under these designs the ATE is identified, but the ACME and ADE are not
- We need additional assumptions!

Effect Mediation: Identification assumptions (1)

Assumption 1: sequential ignorability (required under standard designs $\ast)$
- $\ast:$ Treatment assignment is either randomized or assumed to be random given the pretreatment covariates
- See Imai et al. (2011, 770) for formula of sequential ignorability assumption
- $X_{i}$ : Observed pretreatment confounders for unit $i$ (e.g., respondent’s gender and race in the media framing study)
sequential ignorability makes two ignorability assumptions sequentially
- 1. Given the observed pretreatment confounders, the treatment assignment is assumed to be statistically independent (ignorable) of potential outcomes and potential mediators
  - Assumption normally holds in experiments as in the Brader et al. (2008) example discussed here
  - Also called no-omitted-variable bias, unconfoundedness, etc.
- 1. Observed mediator is ignorable given the actual treatment status and pretreatment confounders
  - Example on next slide!

Effect Mediation: Identification assumptions (2)

Example for sequential ignorability (1) and (2)
- 1. because news stories are randomly assigned to subjects, the first part of Assumption 1 will hold even without conditioning on any pretreatment covariates $X_{i}$
- 1. implies that there are no unmeasured pretreatment or posttreatment covariates that confound the relationship between the levels of anxiety $M$ and immigration opinions $Y$
  - That is a strong assumption even in randomized experiments..
  - Would require measurement of complete set of covariates that affect both anxiety and immigration opinions, and they all must not be affected by the treatment
  - Violation, e.g., if both anxiety and immigration opinions are affected by fear disposition or ideology
If sequential ignorability holds we can identify ACME and ADE [+ Assumption 2: consistency assumption, see discussion in Imai et al. (2011, 782)]
Imai et al. (2011) also provide R code to conduct causal mediation analysis
- See package on github: https://github.com/kosukeimai/mediation

Assignment mechanism (1)

Key component of causal analysis (Imbens and Rubin 2015)
“process that determines which units receive which treatments, hence which potential outcomes are realized and thus can be observed [and which are missing]” (Imbens and Rubin 2015, 31)
“describes, as a function of all covariates and of all potential outcomes, the probability of any vector of assignments” (Imbens and Rubin 2015, 31)
$Pr(\mathbf{D}|\mathbf{X},\mathbf{Y}(0),\mathbf{Y}(1))$ : Function that assigns probabilities to all possible values of vector of assignments $\mathbf{D}$ ( $\mathbf{D}$ = $\mathbf{W}$ in Imbens and Rubin (2015))
- e.g., Bernoulli trial with 4 people
  - N = 4 gives $2^{N}$ possible assignment vectors and $Pr(...) = 0.5^{N}$
  - $\mathbf{D}=\left\{0000, 0001, 0011, ...\right\}$ (Q: How do we get to 16 vectors?)
$\neq$ unit-level assignment probability $p_{i}(\mathbf{X},\mathbf{Y}(0),\mathbf{Y}(1))$ )

Assignment mechanism (2)

Imbens and Rubin (2015, 31f) define assignment mechanism and provide a systematic outline of the underlying causal assumptions
In part, they introduce new terms (called restrictions: Individualist assignment, probabilistic assignment, unconfounded assignment)
Since, these terms reflect the assumptions we discussed so far (independence, SUTVA etc.) we will stick to the latter terms
In experiments we randomize $\rightarrow$ some ways are better than others
Imbens and Rubin (2015, 47f) provide a very insightful taxonomy of classical randomized experiments

Taxonomy (1): Bernoulli trials

Bernoulli experiment tosses a fair coin for each unit
- coin = heads = assigned to treatment; coin = tails = assigned to control
- Coin = fair so unit-level probabilities are all 0.5 (and independent)
- Assignment mechanism: $Pr(\mathbf{D}|\mathbf{X},\mathbf{Y}(0),\mathbf{Y}(1)) = 0.5^{N}$ [probability of any $\mathbf{D}$ = product of individual probabilities] (Imbens and Rubin 2015, 48)
Often used in online survey experiments (tossing “digital coin” as people enter survey)
Disadvantage
- Because of independence of assignment across all units, there is a positive probability (even if small) that all units will receive the same treatment (e.g., $\mathbf{D}=\left\{0000, 1111\right\}$ )
- Q: What is the problem then?

In such situations there is no evidence in the data about potential outcome values under non-observed treatment values

Bernoulli trials: Assignment vectors

Q: Below you have all 16 possible assignment vectors for a Bernoulli experiment with 4 persons (N = 4). Please compare the different vectors. Which one(s) would you prefer? What is the probability of one of them occuring?

Possible assignment vectors (Bernoulli trial, N = 4)
unit	D2	D3	D4	D5	D6	D7	D8	D9	D10	D11	D12	D13	D14	D15	D16
Simon	1	0	1	0	1	0	1	0	1	0	1	0	1	0	1
Julia	0	1	1	0	0	1	1	0	0	1	1	0	0	1	1
Claudia	0	0	0	1	1	1	1	0	0	0	0	1	1	1	1
Diego	0	0	0	0	0	0	0	1	1	1	1	1	1	1	1

$Pr(\mathbf{D}|\mathbf{X},\mathbf{Y}(0),\mathbf{Y}(1)) = 0.5^{N} = 0.5^4 = \frac{1}{16} = 0.0625$
- Each assignment vector has a $\frac{1}{16}$ probability to occur
Other designs ensure equal size (or enough) treated and control units

Taxonomy (2): Completely randomized experiments

Take an even number of units and divide them at random in two groups, with exactly one-half of the sample receiving treatment and remaining receiving control
- e.g., put labels for N units in an urn and draw $N_{t}=N/2$ and treat them

$Pr(\mathbf{D}|\mathbf{X},\mathbf{Y}(0),\mathbf{Y}(1)) = \begin{cases} \left( \begin{array}{c} N \\ N_{t} \end{array} \right)^{-1} & \quad \sum_{i=1}^{N} D_{i}=N_{t},\\ 0 & \quad otherwise \end{cases}$

Advantage: Treatment/control groups have size we choose (e.g., equal size)
Disadvantage: Covariates may still be associated with potential outcomes
Example: Study with N = 20 units, 10 men, 10 women
- Potential treatment and control outcomes vary strongly with sex
- Design assures that 10 get randomly treated but they could be all men (or women)
- Then av. differences would be due to sex rather than treatment
$Pr(\mathbf{D}|\mathbf{X},\mathbf{Y}(0),\mathbf{Y}(1))$ : Function (assignment mechanism) that assigns probabilities to all possible assignment vectors $\mathbf{D}$

$Pr(\mathbf{D}|\mathbf{X},\mathbf{Y}(0),\mathbf{Y}(1))$ =

$\begin{cases}\text{Assignment vectors with} \sum_{i=1}^{N} D_{i}=N_{t} \text{occur with probability of} \left(\begin{array}{c}N \\N_{t}\end{array}\right),\\ \text{All other assignment vectors occur with probability} \end{cases}$

$\sum_{i=1}^{N} D_{i}=N_{t}$ : Assignment vectors where the sum (= sum of 1s) equals $N_{t}$ that is the number of treated units
$N_{t}$ : Number of treated units (does not have to equal $N/2$ )

Completely randomized exp.: Assignment vectors

Q: With $N = 4$ and $N_{t} = 2$ we have 6 possible assignment vectors. Which one(s) would you prefer? What is the probability of one of them occuring?

Possible assignment vectors (N = 4)
unit	D1	D2	D3	D4	D5	D6
Simon	1	1	0	1	0	0
Julia	1	0	1	0	1	0
Claudia	0	1	1	0	0	1
Diego	0	0	0	1	1	1

$\left(\begin{array}{c}N \\ N_{t}\end{array}\right)^{-1}$ = $\left(\frac{N!}{N_{t}!\,(N-N_{t})!}\right)^{-1}$ = $\left(\begin{array}{c}4\\ 2\end{array}\right)^{-1}$ = $\left(\frac{4!}{2!\,(4-2)!}\right)^{-1}$ = $\left(\frac{4\cdot3\cdot2\cdot1}{2\cdot1 \times2\cdot1!}\right)^{-1}$ = $\left(\frac{24}{4}\right)^{-1}$ = $\left(6\right)^{-1}$ = $\frac{1}{6}$
Q: How many assignment vectors are there if $N = 4$ and $N_{t} = 3$ ? What is their probability of occuring?

Yes, 4 assignment vectors and probability 1/4!
…number of assignment vectors is reduced as compared to Bernoulli trial!

Taxonomy (3): Stratified randomized experiments

Population of units in the study is first partitioned into blocks or strata
- Units within each block similar with respect to some (functions of) covariates thought to be predictive of potential outcomes
Within each block, we conduct a completely randomized experiment, with assignments independent across blocks
Example: 2 blocks, e.g., males and females, where independent completely randomized (block) experiments are conducted for each group/block
Assignment mechanism (see Imbens and Rubin 2015, 52): Same formula as completely randomized experiments but replacing $N$ and $N_{t}$ with $N(m)$ and $N_{t}(m)$ for males and $N(f)$ and $N_{t}(f)$ for females

Probability distributions (2)

Probability distributions (Everitt 2010, 338) (Examples)
- “a mathematical formula that gives the probability of each value of the variable” (discrete random variable)
- “a curve described by a mathematical formula which specifies, by way of areas under the curve, the probability that the variable falls within a particular interval” (continuous random variable)
- Uni- and multivariate (e.g., multivariate normal, Q: What is the 3rd dimension?)
Also called theoretical distributions (as opposed to empirical distributions of data) and often invented by mathematicians (Q: Most famous one?)
Bernoulli distribution (simplest discrete distribution): e.g., flipping a fair coin
- PMF (Q?) = $\scriptsize Pr(k) = \begin{cases} q = 1-p & \quad \text{for k} = 0,\\ p & \quad \text{for k} = 1\end{cases}$ (e.g., with p = 0.5)
- Empirical distribution (N = 5), e.g., $\scriptsize \{0, 1, 1, 0, 1\}$
- R: as.numeric(simDAG::rbernoulli(5, 0.5)) = 1, 0, 0, 0, 1

Probability distributions (3)

Q: Why do probability distributions facilitate our work?

We use probability distributions for statistical inference
Frequentism/Frequentist inference
- Any given experiment (dataset) can be considered as one of an infinite sequence of possible repetitions of the same experiment
- RQ: What is the average happiness/effect of gender on happiness among Mannheim students?
- Sample of students in class (dataset) = one of infinite sequence of samples we could draw from Mannheim Univ. students
- Sample → Calculate means/mean diff./regression coefficient → Sampling distribution of statistic $\approx$ probability distribution
We’ll talk more about uncertainty later on

Footnotes

Generalizability of a study’s empirical findings to new environments, settings or populations (Pearl and Bareinboim 2014)
Outcome measurement might become a treatment in itself. Make sure that everyone is equally exposed, i.e., that exposure to outcome measurement is constant across units.
Individualistic assignment and SUTVA are both critical assumptions in causal inference but serve different purposes. Individualistic assignment ensures that the treatment assignment for each unit is based solely on its own characteristics, while SUTVA ensures that the potential outcomes for each unit are not influenced by the treatment assignments of other units and that observed outcomes match potential outcomes for the received treatment. Both assumptions help in simplifying the analysis and interpretation of causal effects, but they operate in distinct areas of the causal inference framework. Dependence on Pre-Treatment Variables: Individualistic Assignment focuses on the independence of the treatment assignment probability from the pre-treatment variables of other units. It ensures that the assignment mechanism for each unit is based solely on its own pre-treatment characteristics. SUTVA does not explicitly address the assignment mechanism. Instead, it focuses on the independence of the potential outcomes from the treatment assignments of other units (no interference) and the consistency of observed and potential outcomes. Scope of assumptions: Individualistic Assignment pertains specifically to the mechanism by which treatments are assigned to units, ensuring that this mechanism is individualistic. SUTVA pertains to the nature of potential outcomes and their relationship with treatment assignments, ensuring no interference and consistency. Implications for Analysis: Individualistic Assignment ensures that when modeling the probability of treatment assignment, one needs to consider only the pre-treatment variables of the specific unit. SUTVA ensures that one can interpret potential outcomes meaningfully without worrying about spillover effects from other units’ treatments, simplifying causal inference by focusing on each unit independently.
e.g., $e(x)=40/(60+40)=0.4$
Small summary: Same propensity score ≠ Same covariate values; Prop. score matching mimics completeley randomised experiment vs. other matching methods that mimic blocked/stratified randomised experiments

1 / 299

Seminar: Kausalanalyse Ein Einsteiger-Seminar für Studierende der Politikwissenschaft Dr. Paul C. Bauer mail@paulcbauer.de @p_c_bauer www.paulcbauer.de License for original material Updated: Jul 17, 2024