The Balance Test Fallacy in Matching Methods for Causal Inference 

 We've talked a lot on this blog about evaluating the quality of matching solutions when applying these matching for preprocessing, and in some of these discussions I've previewed and referenced arguments from a paper I was working on with Kosuke Imai and Liz Stuart.  We have finally finished the paper.  For anyone interested, you can get a copy  here .  The abstract follows.  Comments welcome! 

 
Matching methods are widely used to adjust for nonrandom treatment assignment when making causal inferences.  In numerous articles across a diverse variety of academic fields that use matching, researchers evaluate the success of the procedure by conducting hypothesis tests, most commonly the  t -test for the mean difference of each of the observed covariates between the matched treated and control groups. We demonstrate that these hypothesis tests are fallacious and discuss better alternatives.