Experimental prudence in political science (Part I) 

 We've talked a fair bit on the blog about the use of experimental data to make causal inferences.   While the inferential benefits of experimental research are clear, experiments raise prudential questions that we rarely face in observational research; they require "manipulation" in more than one sense of that word.  As someone who is an interested observer of the experimental literature rather than an active participant, I wonder how well the institutional mechanisms for oversight have adapted to field experimentation in the social sciences in general (and political science in particular).  In medical experiments, the ability in principle to obtain informed consent from subjects is critical in determining what is ethically acceptable, but this is often not possible in a political context; external validity may depend on concealing the experimental nature of the manipulation from the "subjects."   Moreover, the effects of the manipulation may be large enough to change large-scale political outcomes, thus affecting individuals outside of the nominal pool of subjects. 
 


 As an example, consider the turnout experiments I discussed  here  and  here .  The large-scale phone experiments in Iowa and Michigan are typical in that they involve non-partisan GOTV (get out the vote) efforts.  Treated voters are contacted by phone (or by mail, or in person) and urged to vote, while control voters are not contacted; neither group, as far as I can tell, know that they are experimental subjects.  Such a design is possible because the act of voting is a matter of public record, and thus the cooperation of the subjects is not required to obtain the relevant data. 

 While the effects of such manipulations may provide some insight for political scientists as to the causes of voter turnout, their practical significance is a bit hard to measure; there are not that many genuinely non-partisan groups out there with both the means and the motivation to conduct large-scale voter mobilization efforts.  There have been some recent efforts to study partisan voter mobilization strategies using field experiments.  David Nickerson, Ryan Friedrichs, and David King have a  forthcoming article  reporting on an experiment in the 2002 Michigan gubernatorial campaign, in which a youth organization of the Michigan Democratic Party agreed to randomize their partisan GOTV efforts aimed at voters believed to be Democrats or independents.  The authors find positive effects for all three of the common GOTV manipulations (direct literature, phone calls, and face-to-face canvassing).  In the abstract, obtaining data from manipulations that are clearly relevant in the real world is good for the discipline.   I have no doubt that both party activists and party scholars would love to do more such research, but it all makes me slightly uncomfortable.  As researchers, should we be in a position where we are (potentially) influencing political outcomes not only through arguments based on the evidence that we collect, but through the process of collecting evidence as well?