Fun with R 2  

 This semester, I have been one of the TFs for Gov 2000 (the introductory statistics course for Ph.D. students in the Government Department).  It the first time that I've been on the teaching staff for a course, and it  has been quite an experience so far.  We've spent the past month or so introducing the basic linear model.  Along the way, Ryan Moore (the other TF) and I have had some fun sharing the best quotes that we've come across about everyone's favorite regression output, R 2 : 

  Nothing in the CR model requires that R 2  be high.  Hence a high R 2  is not evidence in favor of the model, and a low R 2  is not evidence against it.  Nevertheless, in empirical research reports, one often reads statements to the effect that "I have a high R 2 , so my theory is good," or "My R 2  is higher than yours, so my theory is better than yours." (Arthur Goldberger,  A Course in Econometrics , 1991) 

  Thus R 2  measures directly neither causal strength nor goodness of fit.  It is instead a Mulligan Stew composed of each of them plus the variance of the independent variable.  Its use is best restricted to description of the shape of the point cloud with causal strength measured by the slopes and goodness of fit captured by the standard error of the regression. (Chris Achen,   Interpreting and Using Regression , 1982) 

  Q:  But do you really want me to stop using R 2 ?  After all, my R 2  is higher than all of my friends and higher than those in all the articles in the last issue of APSR!
  A:  If your goal is to get a big R 2 , then your goal is not the same as that for which regression analysis was designed.  The purpose of regression analysis and all of parametric statistical analyses is to estimate interesting population parameters....       If the goal is just to get a big R 2 , then even though that is unlikely to be relevant to any political science research question, here is some "advice": Include independent variables that are very similar to the dependent variable.  The "best" choice is the dependent variable; your R 2  will be 1.0. (Gary King, " How not to lie with statistics ," AJPS , 1986).  

 So this is old news, right?  Maybe not.  Quite possibly the thing that has surprised me the most so far is just how much students want R 2  to tell them how good their model is.  You could almost see the anguish in their faces as we read these quotes to them, particularly among those who have taken some statistics in the past.  The question I want to throw out is, why is R 2  such an attractive number?  Why do we want to believe it?  Maybe our cognitive science colleagues have some insight....