More Discussion on Matching Estimators

At first, I thought my interpretation of matching was a little oversimplified. In fact, Andrew Gelman stated the following concern about this (in the STATA Journal), particularly the interpretation from Angrist and Pischke:

"A casual reader of the book might be left with the unfortunate impression that matching is a competitor to regression rather than a tool for making regression more effective."

I'm a big fan of Dr. Gelman's blog 'Statistical Modeling, Causal Inference and Social Science'  and link to it from this site. I've actually referenced discussions he has had in the comments section with Hal Varian related to Instrumental Variables. For further clarification I contacted Angrist and Pischke via thier blog and  this is what they stated:

Question: You state:

“Our view is that regression can be motivated as a particular sort of
weighted matching estimator, and therefore the differences between
regression and matching estimates are unlikely to be of major
empirical importance” (Chapter 3 p. 70)

I take this to mean that in a ‘mostly harmless way’ regular OLS
regression is in fact a method of matching, or is a matching
estimator.  Is that an appropriate interpretation?  In ‘The Stata
Journal and his blog, Andrew Gelman takes issue with my understanding,
he states:

“A casual reader of the book might be left with the unfortunate
impression that matching is a competitor to regression rather than a
tool for making regression more effective.”

Any guidance?

-Matt Bogard, Western Kentucky University

Reply:

Well Matt, Andrew Gelman’s intentions are undoubtedly good but I’m afraid he risks doing some harm here.  Suppose you’re interested in the effects of treatment, D, and you have a discrete control variable, X, for a selection-on-observables story.  Regress on D an a full set of dummies (i.e., saturated) model for X.  The resulting estimate of  the effect of D is equal to matching on X, and weighting across cells by the variance of X, as explained in Chapter 3.  While you might not always want to saturate, any other regression model for X gives the best linear approx to this version subject to whatever parameterization you’re using.
This means that i can’t imagine a situation where matching makes sense but regression does not (though some my say that I’m known for my lack of imagination when it comes to econometric methods)

I also inquired with Dr. Gelman, who to my surprise also followed up on his blog:

I don't know what Angrist and Pischke actually do in their applied analysis. I'm sorry to report that many users of matching do seem to think of it as a pure substitute for regression: once they decide to use matching, they try to do it perfectly and they often don't realize they can use regression on the matched data to do even better. In my book with Jennifer, we try to clarify that the primary role of matching is to correct for lack of complete overlap between control and treatment groups.
But I think in their comment you quoted above, Angrist and Pischke are just giving a conceptual perspective rather than detailed methodological advice. They're saying that regression, like matching, is a way of comparing-like-with-like in estimating a comparison. This point seems commonplace from a statistical standpoint but may be news to some economists who might think that regression relies on the linear model being true.

Gary King and I discuss this general idea in our 1990 paper on estimating incumbency advantage. Basically, a regression model works if either of two assumptions is satisfied: if the linear model is true, or if the two groups are balanced so that you're getting an average treatment effect. More recently this idea (of their being two bases for an inference) has been given the name "double robustness"; in any case, it's a fundamental aspect of regression modeling, and I think that, by equating regression with matching, Angrist and Pischke are just trying to emphasize that these are just tow different ways of ensuring balance in a comparison.

In many examples, neither regression nor matching works perfectly, which is why it can be better to do both (as Don Rubin discussed in his Ph.D. thesis in 1970 and subsequently in some published articles with his advisor, William Cochran).

References:

Gelman, Andrew. The Stata Journal (2009) 9, Number 2, pp. 315–320
A statistician’s perspective on “Mostly Harmless Econometrics: An Empiricist’s Companion”, by Joshua D. Angrist and J¨orn-Steffen Pischke.

Angrist and Pischke, Mostly Harmless Econometrics, 2009