Open Season on the Messenger 

 In a previous post, Mike quoted Alan Greenspan, "I suspect greater payoffs will come from more data than from more technique."  Not an uncommon opinion. But there are more and less flattering ways of reading such statements.   

 For what’s behind the sentiment, I sometimes suspect (I’m not picking fights with the Maestro), is not just the desire for better data but a distrust of advanced statistical methods. There’s this perception that more complicated math necessitates more assumptions, ergo less robust results.  By this logic, the simpler the method, the more credible the conclusion.  Crosstabs rule, ANOVA passes muster.  The truth, of course, is the opposite: simple stats in observational data analysis usually require more assumptions. As we move from crosstabs to OLS to GEE for a given analytical goal we are usually trying to relax assumptions. Tragically, the presence of said assumptions often becomes obvious only after the author points them out.  And then it’s open season on the messenger.   
 


 I witnessed this sort of thinking recently when I reviewed a paper for a leading sociological journal.  The author pointed out some serious methodological flaws in one strand of comparative welfare state research, then proposed an alternative to one well regarded analysis by relaxing some offending assumptions.  Boom, did he get slammed by one reviewer for allegedly making the very assumptions he had exposed in the first place.  The paper was rejected in the first round. (This is sort of a pet peeve of mine, and I might vent again.)