The Oregon Medicaid Experiment, Applied Econometrics, & Causal Inference

Recently, the findings of a paper published in Science that finds an increase in ER visits among patients benefiting from expanded medicaid in Oregon has been in the news. I like this work because it represents a great example of applied econometrics and causal inference in the field of healthcare econometrics:

http://www.npr.org/blogs/health/2014/01/02/259128081/medicaid-expansion-boosted-emergency-room-visits-in-oregon 

"The result, said Finkelstein, was that the groups of people with or without insurance were identical, "except for the fact that some have insurance and some don't. You've literally randomized the allocation of insurance coverage."

If you are not familiar with the context, the state of Oregon expanded medicaid coverage (pre PPACA) but only to randomly selected winners of a lottery. About half the winners did not apply for and utilize the expanded coverage, so the only TRUE RANDOM comparisons involve lottery winners to losers. So, as Finkelstein is quoted, it is literally a randomization of the allocation of insurance. This is a valid analysis under an 'intent-to-treat' framework, but bear in mind this is NOT necessarily a comparison of groups of identical people with or without insurance. Further, you cannot compare those 50% or so lottery winners that took the new coverage to losers without coverage and appeal to randomization or claim that the groups are identical and comparable. (there could be huge issues related to selection bias) However, if the authors used  instrumental variables to get an estimate of 'local average treatment effects' comparing those winners that took the new coverage to statistically similar losers, who likely would have taken coverage if they would have been winners:

"We compare outcomes between the “treatment group” (those randomly selected in the lottery) and the “control group” (those not randomly selected)......Our intent-to-treat analysis, comparing the outcomes in the treatment and control groups, provides an estimate of the causal effect of winning the lottery (and being permitted to apply for OHP Standard)."

"Of greater interest may be the effect of Medicaid coverage itself. Not everyone selected by the lottery enrolled in Medicaid; some did not apply and some who applied were not eligible for coverage. To estimate the causal effect of Medicaid coverage, we use a standard instrumental-variable approach with lottery selection as an instrument for Medicaid coverage. This analysis uses the lottery’s random assignment to isolate the causal effect of Medicaid coverage."

So, what is the practical difference between intent-to-treat and the local average treatment effect (via instrumental variables) in the context of this research? The authors explain that very well:

"The intent-to-treat estimate may be a relevant parameter for gauging the effect of the ability to apply for Medicaid; the local-average-treatment-effect estimate is the relevant parameter for evaluating the causal effect of Medicaid for those actually covered."

Also as discussed in the supplementary appendix, both the ITT and IV estimates are based on linear probability models and comparison to marginal effects derived from logistic regression. (see Oregon Medicaid Experiment and Linear Probability Models)

You can find a good discussion about this experiment intent to treat, etc in the context of the NEJM paper in an EconTalk podcast with Jim Manzi and Russ Roberts this past year: http://www.econtalk.org/archives/2013/05/jim_manzi_on_th.html 

Also, a nice profile of MIT economist Amy Finkelstein in a related story from Bloomberg: "MIT Economist Seeks Facts in Health-Care Policy Debate."  http://www.bloomberg.com/news/2014-01-03/mit-economist-seeks-facts-in-health-care-policy-debate.html 


References: 

"The Oregon Experiment--Effects of Medicaid on Clinical Outcomes," by Katherine Baicker, et al. New England Journal of Medicine, 2013; 368:1713-1722. http://www.nejm.org/doi/full/10.1056/NEJMsa1212321

Medicaid Increases Emergency-Department Use: Evidence from Oregon's Health Insurance Experiment. Sarah L. Taubman,Heidi L. Allen, Bill J. Wright, Katherine Baicker, and Amy N. Finkelstein. Science 1246183Published online 2 January 2014 [DOI:10.1126/science.1246183]