Difference-in-Differences in the Presence of Unknown Interference
Abstract
The stable unit treatment value (SUTVA) is a crucial assumption in the Difference-in-Differences (DiD) research design. It rules out hidden versions of treatment and any sort of interference and spillover effects across units. Even if this is a strong assumption, it has not received much attention from DiD practitioners and, in many cases, it is not even explicitly stated as an assumption, especially the no-interference assumption. In this technical note, we investigate what the DiD estimand identifies in the presence of unknown interference. We show that the DiD estimand identifies a contrast of causal effects, but it is not informative on any of these causal effects separately, without invoking further assumptions. Then, we explore different sets of assumptions under which the DiD estimand becomes informative about specific causal effects. We illustrate these results by revisiting the seminal paper on minimum wages and employment by Card and Krueger (1994).
Summary
This paper addresses a critical but often overlooked assumption in Difference-in-Differences (DiD) analysis: the Stable Unit Treatment Value Assumption (SUTVA), specifically the no-interference component. The authors investigate what the DiD estimand actually identifies when interference (spillover effects between units) is present and potentially unknown. They demonstrate that under such conditions, the DiD estimand no longer estimates the Average Treatment Effect on the Treated (ATT), but rather a *difference* between the Total Average Treatment Effect on the Treated (TATT) and the Average Spillover Effect on the Control (ASC) group. This means the DiD estimate is not informative about the individual causal effects without further assumptions. The paper then explores several alternative sets of assumptions, such as constraints on the unobserved trends of the outcome in the absence of treatment, or assumptions on the magnitude or sign of the spillover effects, under which the DiD estimand can be used to partially identify or bound the TATT or ASC. The authors illustrate their theoretical results by revisiting the Card and Krueger (1994) study on minimum wages and employment, showing how the presence of interference could alter the interpretation of their findings. This work is important because it highlights the limitations of standard DiD analysis when interference is a concern and provides a framework for researchers to consider and address this issue.
Key Insights
- •The core finding is that under unknown interference, the standard DiD estimand identifies `τ_1 - τ_0`, where `τ_1` is the Total Average Treatment Effect on the Treated (TATT) and `τ_0` is the Average Spillover Effect on the Control (ASC). It does *not* identify the ATT.
- •The paper highlights that even if the DiD estimand is zero, it does not imply that the treatment had no effect. It could be that both the TATT and the ASC are zero, or that they are equal in magnitude and direction.
- •The authors propose alternative assumptions to (partially) identify the TATT and ASC separately. These include assumptions on the trend of the potential outcome in the absence of treatment (Assumption 8 and 9), or assumptions on the sign (Assumption 10) or magnitude (Assumption 11) of the spillover effect.
- •Proposition 5 shows how to partially identify `τ_g` (either TATT or ASC) under Assumption 9 (Range of the time trend of outcome in the absence of treatment): `τ_g ∈ [E[Y_i1 - Y_i0 | G_i = g] - k, E[Y_i1 - Y_i0 | G_i = g] + k]`, where `k` is a non-negative constant representing the range of the trend.
- •Proposition 7 demonstrates that if `|τ_1| ≥ |τ_0|` (Assumption 11), then `sgn(τ_1) = sgn(DiD)`. This allows for identifying the sign of the TATT even if the magnitude cannot be identified.
- •The paper revisits Card and Krueger (1994) and argues that if the no-interference assumption is violated (e.g., due to cross-border commuting), their conclusion that the minimum wage increase increased employment may not be valid.
- •The authors maintain agnosticism about the specific form of interference, allowing for arbitrary patterns of spillovers. This distinguishes their work from other studies that explicitly model interference structures (e.g., spatial or network spillovers).
Practical Implications
- •Researchers using DiD should explicitly state and justify the no-interference assumption. If this assumption is questionable, they should consider alternative assumptions and identification strategies as outlined in the paper.
- •The results caution against over-interpreting DiD estimates as causal effects in the presence of potential spillover effects. The DiD estimand should be interpreted as a *difference* in effects between treated and control groups.
- •Practitioners can use the alternative assumptions proposed in the paper (e.g., on the sign or magnitude of spillover effects) to conduct sensitivity analyses and assess the robustness of their DiD findings. For example, bounding the treatment effect by considering different possible values for *k* in Assumption 9.
- •The paper opens up avenues for future research, including developing more sophisticated methods for identifying and estimating causal effects in DiD settings with complex interference structures. It also encourages researchers to incorporate domain knowledge and prior beliefs about spillover effects into their analyses.