Thoughts on SUTVA (Part II) 

  Alexis Diamond , guest blogger 

 In part I (yesterday), I introduced the subject of SUTVA (the stable unit treatment value assumption), an assumption associated with Rubin's causal model.  Well, why have SUTVA in the first place? What work is it actually doing?  What does it require?  "The two most common ways in which SUTVA can be violated appear to occur when (b) there are versions of each treatment varying in effectiveness or (b) there exists interference between units" (Rubin 1990, p. 282).*  But this two-step SUTVA shorthand is frequently implausible in the context of many important and interesting causal questions. 


 SUTVA allows for a precise definition of causal effects for each unit.  When SUTVA obtains, the inference under investigation relates to the difference between what would have been observed in a world in which units received the treatment and what would have been observed in a world in which treatment did not exist.  SUTVA makes the inference, the causal question under investigation, crystal clear. 

 But SUTVA is not necessary to perform inference in the context of Rubin's causal model--what is necessary is to precisely define causal effects of interest in terms of potential outcomes and to adhere to the principle that for every set of allowable treatment allocations across units, there is a corresponding set of fixed (non-stochastic) potential outcomes that would be observed. In my peacekeeping analysis, I define units as country-episodes; each unit is an episode during which a country experienced civil war and was either treated/not-treated by a UN peacekeeping mission. 

 I define my causal effects precisely: I am interested in causal effects for treated units, and I define the causal effect for each treated unit as the difference between the observed outcome and what would have been observed had that unit's treatment been turned-off and peacekeeping had not occurred.  There are many other potential outcomes one could contemplate and utilize to make other causal inferences; these others are beyond the scope of my investigation.  I don't need SUTVA or other exclusion restrictions to exclude them.  I exclude them in the way I pose my causal question. 

 I am not claiming that all peacekeeping missions are exactly the same—that would be silly.  I also do not claim non-interference across units—after all, how could this be true, or even approximately true? History matters.  Peacekeeping missions affect subsequent facts on the ground within and across countries.  So SUTVA is going to be violated.  But what allows me to proceed with an attempt at analysis is that my causal question is, nevertheless, well-defined.  Clearly, I mean only one thing when referring to the "estimated effect of peacekeeping": the difference between the observed outcome for each and every treated unit and what would have been observed for each unit under the control regime of no-peacekeeping.  I define the average effect for the treated (ATT), my ultimate estimand of interest, to be the average of these estimated unit-level effects. 

 Three caveats apply: (1) I am not claiming this ATT represents what it does under SUTVA, namely the average difference in potential outcomes that would have been observed given all selected units experiencing treatment vs. all experiencing control; (2) I must assume there is only one version of the control intervention; (3) estimation will require additional assumptions, and if estimating treatment effects under exogeneity (eg., via matching), one must still make the case for ignorable assignment.  This last caveat is very different from, and subsequent to, the others, in the sense that estimation and analysis via matching (or any other method) only makes sense if the first two caveats obtain and the causal question is well-defined. 

 As social science moves increasingly toward adoption of the Rubin causal model, I predict that political scientists (and social scientists more generally) will frame their SUTVA-like assumptions and inferential questions in this way.  I think this is consistent with what Gary King and his coauthors were doing in Epstein et al. (2005)**, when they asked about the effect of war on Supreme Court decision-making.  They were not claiming that occurrences of treatment (war) had no effect on subsequent Supreme Court decisions; they were asking about what would have happened if each episode of treatment had been turned off, one at a time.  And in many cases, this is the only kind of question there is any hope of answering—the only kind of question close enough to the data to allow for plausible inference.  As long as these causal questions themselves are interesting, this general approach seems to me to be a coherent and sensible way forward. 

 *Rubin, Donald B. Formal Modes of Statistical Inference For Causal Effects. Journal of Statistical Planning and Inference. 25 (1990), 279-292. 

 ** Epstein, Lee; Daniel E. Ho; Gary King; and Jeffrey A. Segal. The Supreme Court During Crisis: How War Affects only Non-War Cases, New York University Law Review, Vol. 80, No. 1 (April, 2005): 1-116.