Remembering the Baldus Study, Part II 

 In a previous post, I summarized the Baldus Study of the role of race in Georgia’s system of capital sentencing during the 1970s.  To review, the Study concluded that the race of the victim, but not that of the defendant, was an important factor in determining whether a capital defendant received the death penalty.  The Study was a pioneering effort to apply what were then cutting-edge statistical techniques (logistic regression) to questions of race discrimination, and it came within a single justice of rendering Georgia’s capital sentencing system constitutionally invalid. 

 As part of my dissertation research, which focuses on applying a potential outcomes understanding of causation to perceptions of immutable characteristics, I am reexamining the Baldus Study data.    With the benefit of 25+ years of hindsight, I have reluctantly concluded that the Study’s findings are questionable (which is different from wrong).  The Study authors collected no data on cases resulting in acquittals or convictions of crimes of severity less than voluntary manslaughter, or indeed on cases that were initially charged as murders but in which charges were reduced prior to trial.  The sampling scheme, a complicated one involving some stratification on the outcome variable (imposition of the death penalty), renders calculation of standard errors difficult, and the method the Study authors used to address this problem depends on asymptotics despite (in some cases) a small number of units. 

 Which leads me to my big questions.  Assume for the moment that I’m right about the idea that modern thinking suggests that the conclusions of the Baldus Study are questionable.  What if the Supreme Court had accepted the Study and struck down Georgia’s capital sentencing system?  Would we now think such a decision was based on questionable science?  Or should courts accept the best statistical evidence available at the time, even if later researchers believe it questionable, because there are also costs to inaction?  (After all, at most, I can “prove” is that Baldus et al. did not prove their case, not that their conclusions were wrong.)  And what will happen to my own “conclusions” in 25 years?