# Social Science Statistics Blog

February 2009
Sun | Mon | Tue | Wed | Thu | Fri | Sat
--- | --- | --- | --- | --- | --- | ---
1 | 2 | 3 | 4 | 5 | 6 | 7
8 | 9 | 10 | 11 | 12 | 13 | 14
15 | 16 | 17 | 18 | 19 | 20 | 21
22 | 23 | 24 | 25 | 26 | 27 | 28

### Authors' Committee

#### Chair:

Matt Blackwell (Gov)

#### Members:

Martin Andersen (HealthPol)
Kevin Bartz (Stats)
Deirdre Bloome (Social Policy)
John Graves (HealthPol)
Rich Nielsen (Gov)
Maya Sen (Gov)
Gary King (Gov)

### Weekly Research Workshop Sponsors

Alberto Abadie, Lee Fleming, Adam Glynn, Guido Imbens, Gary King, Arthur Spirling, Jamie Robins, Don Rubin, Chris Winship

Weekly Workshop Schedule

Search this site:

### Recent Comments

- Re: App Stats: Glynn on "Using Post-Treatment Variables to Establish Upper Bounds on Causal Effects: Assessing Executive Selection Procedures in New Democracies"
  By: Office Furniture Warehouse
- Re: App Stats: Bahar on "International Knowledge Diffusion and the Comparative Advantage of Nations"
  By: Christa Herzog
- Re: App Stats: Bahar on "International Knowledge Diffusion and the Comparative Advantage of Nations"
  By: Frigler.de
- Re: App Stats: Bahar on "International Knowledge Diffusion and the Comparative Advantage of Nations"
  By: sri
- Re: App Stats: Bahar on "International Knowledge Diffusion and the Comparative Advantage of Nations"
  By: sri
- Re: App Stats: Bahar on "International Knowledge Diffusion and the Comparative Advantage of Nations"
  By: W7midia
- Re: App Stats: Titiunik on "Using Regression Discontinuity to Uncover the Personal Incumbency Advantage"
  By: SEO Services Canada
- Re: App Stats: Yamamoto on "A Multinomial Response Model for Varying Choice Sets, with Application to Partially Contested Multiparty Elections"
  By: Robert Rowshan
- Re: App Stats: Yamamoto on "A Multinomial Response Model for Varying Choice Sets, with Application to Partially Contested Multiparty Elections"
  By: KLLD
- Re: App Stats: Pfister on "Visual Computing in Biology"
  By: Biology education

### Recent Entries

- LATEs when they look odd
- Missingness Maps and Cross Country Data
- Richardson on ``Analysis of the Binary Instrumental Variable Model"
- My Basketball Friend
- Uncertainty Estimates and the Current Population Survey
- Is Height Contagious? Detecting Implausible Social Network Effects
- Social pressure and biased refereeing in Italian soccer
- Garip presenting "How Do Network Externalities Lead to Intergroup Inequality?"
- How many people could improved COBRA cover?
- Bayesian Propensity Score Matching

### Categories

- Regular Post

### Blogroll

SMR Blog
Brad DeLong
Cognitive Daily
Complexity & Social Networks
Developing Intelligence
EconLog
The Education Wonks
Empirical Legal Studies
Free Exchange
Freakonomics
Health Care Economist
Junk Charts
Language Log
Law & Econ Prof Blog
Machine Learning (Theory)
Marginal Revolution
Mixing Memory
Mystery Pollster
New Economist
Political Arithmetik
Political Science Methods
Pure Pedantry
Science & Law Blog
Simon Jackman
Social Science++
Statistical modeling, causal inference, and social science

### Archives

- May 2012
- April 2012
- March 2012
- February 2012
- January 2012
- November 2011
- October 2011
- September 2011
- July 2011
- June 2011
- May 2011
- April 2011
- March 2011
- February 2011
- January 2011
- December 2010
- November 2010
- October 2010
- September 2010
- August 2010
- May 2010
- April 2010
- March 2010
- February 2010
- January 2010
- December 2009
- November 2009
- October 2009
- September 2009
- August 2009
- June 2009
- May 2009
- April 2009
- March 2009
- February 2009
- January 2009
- December 2008
- November 2008
- October 2008
- September 2008
- August 2008
- June 2008
- May 2008
- April 2008
- March 2008
- February 2008
- January 2008
- December 2007
- November 2007
- October 2007
- September 2007
- June 2007
- May 2007
- April 2007
- March 2007
- February 2007
- January 2007
- December 2006
- November 2006
- October 2006
- September 2006
- August 2006
- July 2006
- June 2006
- May 2006
- April 2006
- March 2006
- February 2006
- January 2006
- December 2005
- November 2005
- October 2005
- September 2005

### Notification

Enter e-mail to [un]subscribe:
Powered by
Movable Type 4.24-en
Syndicate this site (XML)
[an error occurred while processing this directive]

« January 2009 | Main | March 2009 »

26 February 2009

### LATEs when they look odd

A few weeks ago I came across this paper by Mark Duggan and Fiona Scott Morton in the QJE looking at how government procurement can influence prices--in this case Medicaid procurement of prescription drugs. Basically, Medicaid receives mandatory rebates if drug prices go up too fast, so there is an incentive for companies to develop new versions of existing drugs (which can then enter the market at a higher price) and to price drugs too high initially if they expect to see a large amount of Medicaid volume. In this context, that policy could also harm other purchasers if prices levels would have been lower in the absence of the Medicaid procurement rules.

The authors exploits the large variation in the share of prescriptions accounted for by Medicaid enrollees to generate their estimates. These estimates indicate that Medicaid market share can increase prices by fairly large amounts (a 10% increase in Medicaid market share leads to a nearly 20% increase in drug prices in 1997 and 2002). But, because drug prices are pretty skewed (even after a log transform), they also look at the effect of Medicaid market share on the price rank of a drug--here the results indicate that a 10% increase in Medicaid market share increases the price rank of a drug by about 4 slots. A difficulty with this result is that the Medicaid market share may be endogenous, thus they also present results using instrumental variables and here is where things get interesting.

It is well established that instrumental variables (IV) estimates are local average treatment effects (LATE)--basically the average effect of treatment on a subgroup of units whose behavior is altered by the instrument. In other words, for every unit there is an associated treatment effect and the LATE is the average over a special subset of units. Sometimes this matters, sometimes it doesn't--in this case I think it is interesting to look at and readers can decide it matters.

The instrument that Duggan and Scott Morton uses exploits variations in the number of people with different conditions and the specificity of drugs to treating certain conditions (i.e. a statin doesn't treat schizophrenia). Hence, they compute the fraction of individuals with diagnoses that the relevant drug can treat who are insured by Medicaid (call it Medicaid patient share to distinguish it from Medicaid market share).

Using this instrument, they compute the predicted Medicaid market share and subsequent parameter estimates. With the two years of data (1997 and 2002) they develop two different predicted Medicaid market shares that are the same regardless of the dependent variable. One would hope that the parameter estimates wouldn't change (much) or at least when there are two dependent variables they should move in the same direction. In this case, the estimates using log price as the dependent variable get smaller indicating that a 10% increase in Medicaid market share results in 11% and 18% increases in prices in 1997 and 2002, so the result doesn't change much for the 2002 regression, but the estimate in 1997 does change by quite a bit and in both years the estimates go down. So one would expect that the price rank regression should show similar results, right? Surprisingly, this is not the case, in 1997 the IV estimate indicates an increase of 2.4 ranks in 1997 and 9.2 ranks in 2002.

To me this suggests that the IV estimates are LATEs over a particularly quirky set of "treatment effects" since the effect on prices are still fairly large in both cases while the effect on ranks is small in one case and very large in the other. Does this alter their conclusions in any way? No, and it probably shouldn't, but it is interesting to see the effect and think about what it means for interpreting their data.

Posted by Martin Andersen at 1:53 PM

25 February 2009

### Missingness Maps and Cross Country Data

I've been doing some work on diagnostics for missing data issues and one that I have found particularly useful and enlightening has been what I've been calling a "missingness map." In the last few days, I used it on some World Bank data I downloaded to see what missingness looks like in a typical comparative political economy dataset.

View image

The y-axis here are country-years and the x-axis are variables. We draw a red square where the country-year-variable cell is missing and a light green square where the cell is observed. We can see immediately that a whole set of variables in the middle columns are almost always unobserved. These are variables measuring income inequality and they are known to have extremely poor coverage. This plot very quickly shows us how listwise deletion will affect our analyzed sample and how the patterns of missingness occur in our data. For example, in these data, it seems that if GDP is missing, then many of the other variables, such as imports and exports are also missing. I think this is a neat way to get a quick, broad view of missingness.

(Another map and some questions after the jump...)

We can also change the ordering of the rows to give a better sense of missingness. For the World Bank data, it is wise to resort the data by time and see how missingness changes over time.

View image

A clear pattern emerges that the World Bank has better and better data as we move forward in time (the map becomes more "clear"). This is not surprising, but it is an important point when, say, deciding the population under study in a comparative study. Clearly, listwise deletion will radically change the sample we analyze (the answers will be biased toward more recent data, at the very least). The standard statistical advice of imputation or data augmentation is tricky as well here because we need to choose what to impute. Should we carry forth with imputation given that income inequality measures seem to be completely unavailable before 1985? If we remove observations before this, how do we qualify our findings?

Any input on the missingness map would be amazing, as I am trying to add as a diagnostic it to a new version of Amelia. What would make these plots better?

Posted by Matt Blackwell at 2:58 PM

23 February 2009

### Richardson on ``Analysis of the Binary Instrumental Variable Model"

Please join us this Wednesday, when Thomas Richardson--Department of Statistics, University of Washington--will present "Analysis of the Binary Instrumental Variable Model", work that is joint with Jamie Robins, Harvard School of Public Health. Thomas provided the following abstract:

In this talk I consider an instrumental variable potential outcomes model in which the instrument (Z), treatment (X) and response (Y) are all binary. It is well known that this model is not identified by the observed joint distribution p(x,y,z). Consequently many statistical analyses impose additional untestable assumptions or change the causal estimand of interest. Here we take a different approach, directly characterizing and graphically displaying the set of distributions over potential outcomes that correspond to a given population distribution p(x,y,z). This provides insights into the variation dependence between the partially identified average causal effects for various compliance groups. The analysis also leads directly to re-parametrization that may be used for Bayesian inference and the development of models that incorporate baseline covariates.

The Applied Statistics Workshop meets each Wednesday at 12 noon in K-354 CGIS-Knafel (1737 Cambridge St). The workshop begins with a light lunch and presentations usually start around 1215 and last until about 130 pm.

Posted by Justin Grimmer at 5:24 PM

21 February 2009

### My Basketball Friend

I met one of my friends on basketball court. This is selection. I select him as my friend because he plays good basketball and is an avid player. We have been friends for almost three years. When either of us wants to play, most times we will call each other and meet on the court. I think without knowing him, I will still play basketball, but not that many times. So we influence each other. Sometimes we eat Vietnamese noodles together at Le's right after game. Contextual factors matter, but it is him who makes me eat more times of noodles than I would have by myself. Probably, our friendship has some impacts on both of our weights and may make them change more synchronously. Similarly, if you are a runner, you will surely like running with your friends and may run more because you get a runner as friend. So the empirical question is whether you indeed play more basketball when you get a friend who likes playing basketball and run more if you get a runner friend. It is also possible that because you play more or run more, you eat more, which offsets the weight loss due to those extra exercises.

Given only observational data, it is hard to disentangle the effects of selection, induction and contextual factors on weight changes. We have to assign you friends (roommates) randomly and check if you and your friends gain/lose weight together, possibly because you two play more basketball, run more, eat similar things, have similar living styles, share similar standards about what consists of a normal weight, etc.

It is interesting to see that the effects of friendship seem to be directional or asymmetric. Only people you think as friend can induce you to lose weight. You can not induce a person who does not think you are his friend to lose weight, although you think he is your friend. This is kind of opinion leader effect.

The directionality of friendship effects also counters the challenging of contextual factors hypothesis, because if contextual factors matter, you would expect friends' weight changes correlate without directionality. Also, if they matter, you would expect your neighbors' weight changes synchronize with yours and the weight of your friend who lives hundreds of miles away should not correlate with yours. But neither is corroborated by data.

Hence selection should be the largest concern in this case. Now the questions are whether using weight changes or obese status changes will remove the selection effect and how we could control it better.

One of my friends told me two weeks ago that, he did not buy the points in "The Spread of Obesity in a Large Social Network over 32 Years" until he read the real paper. I confessed, "Same here." Read the real paper, not the popular press. But you are absolutely not obligated to buy the points. Here are more.

K.P. Smith and N.A. Christakis, "Social Networks and Health," Annual Review of Sociology 34: 405-429 (August 2008)

Journal of Health Economics, Volume 27, Issue 5, September 2008

Ethan Cohen-Cole, Jason M. Fletcher, "Is obesity contagious? Social networks vs. environmental factors in the obesity epidemic", Pages 1382-1387.

Justin G. Trogdon, James Nonnemaker, Joanne Pais, "Peer effects in adolescent overweight", Pages 1388-1399.

J.H. Fowler, N.A. Christakis, "Estimating peer effects on health in social networks: A response to Cohen-Cole and Fletcher; and Trogdon, Nonnemaker, and Pais", Pages 1400-1405.

P.s. My friend and I have successfully induced several of our friends who originally do not play basketball to play more. But hopefully they can gain some weight rather than losing weight so that we can play more strongly and better.

Posted by Weihua An at 9:01 AM

19 February 2009

### Uncertainty Estimates and the Current Population Survey

The Annual Social and Economic Supplement (ASEC) to the Current Population Survey (CPS) is among the most widely used and influential data sets in the social sciences and in policymaking. For example, the much-cited figure of 45 million uninsured is a CPS estimate; Title I education funding is allocated using the CPS; and state outlays for the State Children's Health Insurance Program are also determined using the survey.

From the perspective of the social scientist, the CPS is a key research tool because of its large sample size (roughly 60,000 households) and because it is is typically released publicly about 5-6 months after the survey is initially fielded. However, one major drawback is that, unlike other major national surveys (the SIPP, the MEPS, and the NHIS to name a few), the public release of the CPS data does not include variables that must be used to get the correct standard errors for the complex survey design. Rather, the CPS releases a series of adjustment factors for specific population subgroups (e.g. by race, income group, state, etc.) that can be applied to uncertainty estimates. However, this approach is obviously problematic in the case of regression -- which adjustment factors does one use if the regression contains a rich array of covariates? As a result, much research using the CPS (which appears quite often in economics and health services research journals) proceeds either under the assumption of simple random sampling, or using robust standard errors. These studies therefore likely have understated uncertainty estimates, casting some doubt on the conclusions of this work.

So what is the applied researcher to do? One simple method of approximation (suggested to me once by Alan Zaslavsky) is to exploit the fact that the CPS uses monthly rotation groups that effectively replicate the CPS survey design. That is, one could produce separate estimates for each monthly rotation group and combine these estimates to come up with an estimate of the uncertainty from the survey design.

An alternative method (described in Davern, et. al Inquiry 43 (3) 2006), is to construct synthetic stratum and primary sampling unit (PSU) variables using available information in the survey (e.g. metropolitan statistical area, state, and household identifiers). In the above article, the authors compared this synthetic method to the internal census files (which obviously do have the complex survey design variables) and computed the ratio of the synthetic method to the standard error from the internal census file. In general, the ratios were on the order of 0.75 to 0.85, bringing the uncertainty estimates closer to the internal estimates than the ratios of about 0.5-0.6 they found under the assumption of simple random sampling (i.e. making no adjustment for survey design) and using robust standard errors.

Posted by John Graves at 8:23 AM

18 February 2009

### Is Height Contagious? Detecting Implausible Social Network Effects

Some of you may be familiar with the recent work on social network effects in public health, where several studies have found significant networks effects on outcomes such as obesity, smoking, and alcohol use. We have blogged about some of this work here and here. One key question with these findings is whether the observed relationship between one's own health outcome and the health status of other individuals in one's reference group is indeed causal or driven primarily by selection effects. Many have argued that confounding seems like a serious concern given that one's friends are not chosen at random. But at the end of the day it remains an empirical question whether the study design is able to account for these selection effects or not.

In "Detecting implausible social network effects in acne, height, and headaches: longitudinal analysis" Ethan Cohen-Cole and Jason Fletcher add to this debate with a series of interesting placebo studies. They demonstrate that the same empirical specification used in previous studies (ie. logistic regression of own health ~ friends' health + X) also "detects" significant and fairly large network effects for implausible outcomes such as acne, height, and headaches. For example, having a friend with headache problems increases the respondent's chances of headache problems by about 47% on average. These implausible placebo findings suggest that previous findings may have been driven by confounding. Similar placebo tests have been used in a variety of papers such as DiNardo and Pishke (1997), Rosenbaum (2002), Abadie and Gardeazabal (2003), Angrist and Krueger (1999), and Auld and Grootendorst (2004) to name just a few, but this study is another great example that demonstrates the power of such tests for social science research. I will use this as a teaching example I think.

Interestingly, Cohen-Cole and Fletcher also show that their implausible effects go away once they augment the standard model by adjusting for environmental confounders that may affect both an individual and her friends simultaneously. They conclude that "There is a need for caution when attributing causality to correlations in health outcomes between friends using non-experimental data."

I wonder how this debate will evolve. The ultimate test to disentangle correlation and causation would be to find a good natural experiment or to run field a experiment where social ties are exogenously assigned. Does anybody know of ongoing research that does this? It seems difficult to get something like this approved by research review boards of course.

Posted by Jens Hainmueller at 9:50 AM

17 February 2009

### Social pressure and biased refereeing in Italian soccer

I recently came across a paper by Per Pettersson-Lidbom and Mikael Priks that uses a neat natural experiment in Italian soccer to estimate the effect of stadium crowds on referees' decisions. After a bout of hooliganism in early February, 2007, the Italian government began requiring soccer stadiums to fulfill certain security regulations; those stadiums that did not meet the requirements would have to hold their games without spectators. As a result, 25 games were played in empty stadiums that month allowing Petterson-Lidbom and Priks to examine game stats (like this) and see whether referees were more disposed toward the home team when the bleachers were filled with fans than when the stadium was empty. Looking at fouls, yellow cards, and read cards, the authors find that referees were indeed more likely to penalize the home team (and less likely to penalize the away team) in an empty stadium. There does not appear to be any effect of the crowd on players' performance, which suggests that fans were reacting to the crowd and not the players (and that fans should save their energy for haranguing the refs).

One of the interesting things in the results is that refs showed no favoritism toward the home team in games with spectators -- they handed out about the same number of fouls and cards to the home and away teams in those games. The bias shows up in games without spectators, where they hand out more fouls and cards to the home team. (The difference is not statistically significant in games with spectators but is in games with spectators.) If we are to interpret the empty stadium games as indicative of what refs would do if not subjected to social pressure, then we should conclude from the data that refs are fundamentally biased against the home team and only referee in a balanced way when their bias is balanced by crowd pressure. This would indeed be evidence that social pressure matters, but it seems unlikely that refs would be so disposed against the home team. A perhaps more plausible interpretation of the findings is that Italian refs are generally pretty balanced and not affected by crowds, but in the "empty stadium" games they punished the home team for not following the rules on stadium security. This interpretation of course makes the finding less generally applicable. In the end the example highlights the difficulty of finding "natural experiments" that really do what you want them to do -- in this case, illustrate what would happen if, quite randomly, no fans showed up for the game.

Posted by Andy Eggers at 8:25 AM

16 February 2009

### Garip presenting "How Do Network Externalities Lead to Intergroup Inequality?"

Please join us this Wednesday when Filiz Garip, Harvard Department of Sociology, will present here joint-work with Paul Dimaggio, "How Do Network Externalities Lead to Intergroup Inequality?". Filiz provided the following abstract for her talk:

In this paper, we identify a mechanism, which we contend chronically reproduces and, under some conditions, may generate or even efface intergroup inequality. That mechanism is (a) the diffusion of goods, services, and practices that (b) are characterized by strong network externalities under conditions of (c) social homophily. When the value of a good or practice to an agent is a function of the number of persons in that agent's network who also possess the good or engage in the practice, and when networks are homophilic with respect to certain social characteristics, this mechanism will exacerbate initial individual-level differences in access to the good or practice and, under some conditions, induce persistent intergroup inequality. We illustrate this claim in two empirical contexts. For the first, the diffusion of access to and use of the Internet, we start with observed data on the relationship between cost and adoption and between adoption levels and price, and produce a computational model that permits us to predict variation in intergroup inequality over time as a function of variation in the strength of network externalities and the extent of social homophily. For the second, the practice of rural-to-urban migration by young people in rural Thailand, we use village-level data on family resources and migration patterns to explore the relationship between information sharing, homophily, and intergroup differences in migration.

The Applied Statistics Workshop meets each Wednesday at 12 noon in K-354 CGIS-Knafel (1737 Cambridge St). The workshop begins with a light lunch and presentations usually start around 1215 and last until about 130 pm.

Posted by Justin Grimmer at 8:55 PM

15 February 2009

### How many people could improved COBRA cover?

One feature of the recently-passed stimulus legislation is a temporary federal subsidy program for individuals to purchase transitional health insurance coverage in the event they lose their job. This coverage -- called COBRA after the legislation that created the program in the 1980's -- is generally available to displaced workers in firms with more than 20 employees, although some states have also adopted policies that allow employees of smaller firms to buy into transitional coverage. The catch, however, is that workers must pay the full premium amount plus 2 percent for administrative expenses.

The new federal program allows for subsidies of up to 65 percent of the cost of health insurance, which is aimed to provide a boost to individuals trying to make ends meet and remain insured while they're unemployed. But the question is, how many people could this program potentially cover in a given year?

Below I've created a population flow diagram using R's diagram package. The underlying data are drawn from the 2006 Medical Expenditure Panel Survey (MEPS) -- a nationally-representative survey of American households conducted each year by the Agency for Health Care Research and Quality. To construct the diagram I took the health insurance coverage status of non-elderly adults and children in January 2006, and compared this to their coverage status in December of that year.

Clearly, the potential for reducing the number of uninsured is large -- according to these estimates, 6.4 million adults and 1.4 million children lost employer-based group insurance between January and December 2006.* If just one-half of these individuals took up a subsidy and were able to continue that group coverage, the number of uninsured in 2006 could have been reduced by about 4 million. Moreover, this is almost certainly an underestimate, since there are also individuals who lost employer-based coverage prior to January, as well as individuals who were intermittently uninsured during the year, who may not show up in the diagrammed flows but could also have benefited from access to subsidized transitional coverage. Finally, I would also note the potential for spillover and cost-offsetting effects within Medicaid, as the estimated 1.2 million who went from group coverage to public coverage could have also retained their employer-based insurance, saving states and the federal government the costs of these extra Medicaid/SCHIP enrollees.

* Note, however, that I have made no attempt to produce confidence intervals around these figures, though in principle this would be straightforward to do with R's survey package. If anyone knows how to easily input these into the figures (I couldn't figure out a way), please let me know!

Posted by John Graves at 4:51 PM

### Bayesian Propensity Score Matching

Many people have realized that conventional propensity score matching (PSM) method does not take into account the uncertainties of estimating propensity scores. In other words, for each observation, PSM assumes that there is only one fixed propensity score. In contrast, Bayesian methods can generate a sample of propensity scores for any observation, by either monitoring the posterior distributions of the estimated propensity scores directly or predicting propensity scores from the posterior samples of the parameters of the propensity score model.

Then matching on thus obtained propensity scores, we should expect to get a distribution of estimated treatment effects. This will also provide us with an estimation of the standard error of the treatment effect. The Bayesian S.E. will be larger than the S.E. based on PSM estimate, as it takes into account more uncertainties. This conjecture is indeed confirmed by a recent paper written by Lawrence C. McCandless, Paul Gustafson and Peter C. Austin, "Bayesian propensity score analysis for observational data", which appears in Statistics in Medicine (2009; 28:94-112). The authors show that, the Bayesian 95% credible interval for the treatment effect is 10% wider than conventional propensity score C.I.

It seems that we should expect Bayesian propensity score matching (BPSM) perform better than PSM in cases where there are a lot of uncertainties in estimating the propensity scores. Before running into any simulations, however, the question is: what are the sources of the uncertainties in estimating propensity scores? From my point of view, there is at least one source of uncertainties, the uncertainties due to omitted variables. I do not think BPSM can do any better than PSM in solving this issue. But maybe, BPSM can model the error terms and so provide better estimations of the propensity scores? The above authors argue that when the association between treatment and covariates is weak (i.e., when the betas are smaller), the uncertainties in estimating propensity scores are higher. Weak association means smaller R-square or larger AIC, etc. Is this equivalent to larger bias due to omitted variables?

Another type of uncertainty related to BPSM, but not to propensity scores, is the uncertainty due to matching procedure. This is avoidable or negligible. Radically, we can just abandon the matching method and resort to linear regression model to predict the outcomes. Or we can neglect the bias from matching procedure, because when we only care about ATT and there is sufficient number of control cases, the bias is negligible, according to Abadie and Imbens 2006. ("Large Sample Properties of Matching Estimators for Average Treatment Effects." Econometrica 74 (1): 235 - 267.)

Of course, the logit model for the propensity scores could be wrong as well. But this can be manipulated in the simulations. Now my question is: how should we do the simulations to evaluate the performance of BPSM vs. that of conventional PSM?

Posted by Weihua An at 12:06 AM

9 February 2009

### Western on "Analyzing Inequality with Variance Function Regressions"

Please join us this Wednesday, February 11th when Bruce Western, Professor of Sociology, will present "Analyzing Inequality with Variance Function Regressions". Bruce provided the following abstract:

Regression-based studies of inequality model only between-group differences, yet often these differences are far exceeded by residual inequality. Residual inequality is usually attributed to measurement error or the influence of unobserved characteristics. We present a regression that includes covariates for both the mean and variance of a dependent variable. In this model, the residual variance is treated as a target for analysis. We apply this model to study the effects of union membership decline on the growth in men's earnings inequality from 1970 to 2006. The union membership data offer additional challenge for data analysis, because survey respondents may misreport their union membership status.

The Applied Statistics Workshop meets each Wednesday at 12 noon in K-354 CGIS-Knafel (1737 Cambridge St). The workshop begins with a light lunch and presentations usually start around 1215 and last until about 130 pm.

Posted by Justin Grimmer at 4:47 PM

5 February 2009

### Deaton on use of randomized trials in development economics

A new NBER paper by Angus Deaton takes on the trendiness of randomized trials, instrumental variables and natural experiments in development economics. One of the main points: well-designed experiments are most useful when they help uncover general mechanisms (i.e. inform theory) and can support real-life policy-making outside their narrow context. A good if lengthy read.

> Deaton, A (2009) Instruments of development: Randomization in the tropics, and the search for the elusive keys to economic development, NBER Working Paper 14690. http://papers.nber.org/papers/w14690

> Harvard users click here.

> There is currently much debate about the effectiveness of foreign aid and about what kind of projects can engender economic development. There is skepticism about the ability of econometric analysis to resolve these issues, or of development agencies to learn from their own experience. In response, there is movement in development economics towards the use of randomized controlled trials (RCTs) to accumulate credible knowledge of what works, without over-reliance on questionable theory or statistical methods. When RCTs are not possible, this movement advocates quasi-randomization through instrumental variable (IV) techniques or natural experiments. I argue that many of these applications are unlikely to recover quantities that are useful for policy or understanding: two key issues are the misunderstanding of exogeneity, and the handling of heterogeneity. I illustrate from the literature on aid and growth. Actual randomization faces similar problems as quasi-randomization, notwithstanding rhetoric to the contrary. I argue that experiments have no special ability to produce more credible knowledge than other methods, and that actual experiments are frequently subject to practical problems that undermine any claims to statistical or epistemic superiority. I illustrate using prominent experiments in development. As with IV methods, RCT-based evaluation of projects is unlikely to lead to scientific progress in the understanding of economic development. I welcome recent trends in development experimentation away from the evaluation of projects and towards the evaluation of theoretical mechanisms.

Posted by Sebastian Bauhoff at 8:12 AM

3 February 2009

### What is Japan doing at 2:04pm?

You can now answer that question and so many more. The Japanese Statistics Bureau conducts a survey every five years called the "Survey on Time Use and Leisure Activities" where they give people journals to record their activities throughout the day. Thus, they have a survey of what people are in Japan at any given time of the day. This is fun data in of itself, but it was made downright addictive by Jonathan Soma who created a slick Stream Graph based on the data. (via kottke)

There are actually three Stream Graphs: one for the various activities, another for how the current activity differs between sexes and a final for how the current activity breaks down by economic status. Thus, the view contains not only information about daily routines, but also how those routines vary across sex and activity. For instance, gardening tends to happen in the afternoon and evening at around equal intensity and is fairly evenly distributed between men and women. Household upkeep, on the other hand, is done mostly by women and mostly in the morning. This visualization is so compelling, I think, because it allows for deep exploration of rich and interesting data (to be honest, though, I find the economic status categories a little strange and not incredibly useful).

I think there are two points that come to mind when seeing this. First is that it would fascinating to see how these would look across countries, even if it was just one other country. The category of this survey on the website for the Japanese Bureau of Statistics is "culture." Seeing the charts actually makes me wonder how different this culture is from other countries. Soma does point out, though, that Japanese men are rather interested in "productive sports" which is perhaps unique to the island.

Second, I think that Stream Graphs might be useful for other time-based data types. Long term survey projects, such as the General Social Survey, track respondent spending priorities. It seems straightforward to use a Stream Graph to capture how priorities shift over time. Other implemented Stream Graphs are the NYT box-office returns data and Lee Byron's last.fm playlist data. This graph type seems best suited for showing how different categories change over time and how rapidly they grow and how quickly they shrink. They also seem to require some knowledge of Processing. There are still some open questions here: What other types of social science data might these charts be useful for? How or should we incorporate uncertainty? (Soma warns that the Japan data is rather slim on the number of respondents)

Also: October 18th is Statistics Day in Japan. There are posters. And a slogan: "Statistical Surveys Owe You and You Owe Statistical Data"!

Posted by Matt Blackwell at 5:37 PM

2 February 2009

### Lok on "Bayesian Combination of State Polls and Election Forecasts"

The first meeting of the applied statistics workshop will be this Wednesday, February 4th, when Kari Lock, Graduate Student in the Department of Statistics, will present "Bayesian Combination of State Polls and Election Forecasts". Kari provided the following abstract:

A wide range of potentially useful data are available for election forecasting: the results of previous elections, a multitude of pre-election polls, and predictors such as measures of national and statewide economic performance. How accurate are different forecasts? We estimate predictive uncertainty via analysis of data collected from past elections (actual outcomes, pre-election polls, and model estimates). With these estimated uncertainties, we use Bayesian inference to integrate the various sources of data to form posterior distributions for the state and national two-party Democratic vote shares for the 2008 election. Our key idea is to separately forecast the national popular vote shares and the relative positions of the states.

The Applied Statistics Workshop meets each Wednesday at 12 noon in K-354 CGIS-Knafel (1737 Cambridge St). The workshop begins with a light lunch and presentations usually start around 1215 and last until about 130 pm.

Hope to see you there--

Posted by Justin Grimmer at 2:14 PM

1 February 2009

### Visualizing partisan discourse

Burt Monroe, Michael Colaresi, and our own Kevin Quinn have written an interesting paper (forthcoming in Political Analysis) assessing methods for selecting partisan features in language, e.g. which words are particularly likely to be used by Republicans or Democrats on a given topic. They have also provided a dynamic visualization of partisan language in the Senate on defense issues between 1997 and 2004 (screenshot below).

The most striking feature coming out of the visualization is that language on defense went through an unpolarized period leading up to 9/11 and even for several months afterward, but that polarized language blossomed in the leadup to the Iraq War and through the end of the period they examine, with Republicans talking about what they thought was at stake ("Saddam", "Hussein". "oil", "freedom", "regime") and the Democrats emphasizing the process ("unilateral", "war", "reconstruction", "billions"). (Link to visualization, a QuickTime movie.)

Posted by Andy Eggers at 8:36 AM