The Oregon Medicaid Experiment and Linear Probability Models

I just recently discussed the methodology used in some recent papers analyzing the Oregon Medicaid expansion (see: http://econometricsense.blogspot.com/2014/01/the-oregon-medicaid-experiment-applied.html ). This was one of the papers:

"The Oregon Experiment--Effects of Medicaid on Clinical Outcomes," by Katherine Baicker, et al. New England Journal of Medicine, 2013; 368:1713-1722. http://www.nejm.org/doi/full/10.1056/NEJMsa1212321 

If you read the supplementary appendix you will find the following:

In all of our ITT estimates and in our subsequent instrumental variable estimates (see below), we fit linear models even though a number of our outcomes are binary. Because we are interested in the difference in conditional means for the treatments and controls, linear probability models would pose no concerns in the absence of covariates or in fully saturated models (Angrist 2001, Angrist and Pischke 2009). Our models are not fully saturated, however, so it is possible that results could be affected by this functional form choice, especially for outcomes with very low or very high mean probability. We therefore explore the sensitivity of our results to an alternate specification using logistic regression and calculating average marginal effects for all binary outcomes, and are reassured that the results look very similar (see Table S15a-d below). 

http://www.nejm.org/doi/suppl/10.1056/NEJMsa1212321/suppl_file/nejmsa1212321_appendix.pdf

You will find a similar methodology in the more recent article in Science previously discussed. This weaves well with some of my past posts:

Linear Regression and Analysis of Variance with Binary Dependent Variables

Regression as an Empirical Tool (matching and linear probability models)

The Oregon Medicaid Experiment, Applied Econometrics, & Causal Inference

Recently, the findings of a paper published in Science that finds an increase in ER visits among patients benefiting from expanded medicaid in Oregon has been in the news. I like this work because it represents a great example of applied econometrics and causal inference in the field of healthcare econometrics:

http://www.npr.org/blogs/health/2014/01/02/259128081/medicaid-expansion-boosted-emergency-room-visits-in-oregon 

"The result, said Finkelstein, was that the groups of people with or without insurance were identical, "except for the fact that some have insurance and some don't. You've literally randomized the allocation of insurance coverage."

If you are not familiar with the context, the state of Oregon expanded medicaid coverage (pre PPACA) but only to randomly selected winners of a lottery. About half the winners did not apply for and utilize the expanded coverage, so the only TRUE RANDOM comparisons involve lottery winners to losers. So, as Finkelstein is quoted, it is literally a randomization of the allocation of insurance. This is a valid analysis under an 'intent-to-treat' framework - comparing winners to losers overall. However, you cannot compare those 50% or so lottery winners that took the new coverage to losers without coverage and appeal to randomization or claim that the groups are identical and comparable. (there could be huge issues related to selection bias) However, the authors used  instrumental variables which made it possible to get an estimate of 'local average treatment effects' comparing those winners that took the new coverage to statistically similar losers, who likely would have taken coverage if they would have been winners:

"We compare outcomes between the “treatment group” (those randomly selected in the lottery) and the “control group” (those not randomly selected)......Our intent-to-treat analysis, comparing the outcomes in the treatment and control groups, provides an estimate of the causal effect of winning the lottery (and being permitted to apply for OHP Standard)."

"Of greater interest may be the effect of Medicaid coverage itself. Not everyone selected by the lottery enrolled in Medicaid; some did not apply and some who applied were not eligible for coverage. To estimate the causal effect of Medicaid coverage, we use a standard instrumental-variable approach with lottery selection as an instrument for Medicaid coverage. This analysis uses the lottery’s random assignment to isolate the causal effect of Medicaid coverage."

So, what is the practical difference between intent-to-treat and the local average treatment effect (via instrumental variables) in the context of this research? The authors explain that very well:

"The intent-to-treat estimate may be a relevant parameter for gauging the effect of the ability to apply for Medicaid; the local-average-treatment-effect estimate is the relevant parameter for evaluating the causal effect of Medicaid for those actually covered."

Also as discussed in the supplementary appendix, both the ITT and IV estimates are based on linear probability models and comparison to marginal effects derived from logistic regression. (see Oregon Medicaid Experiment and Linear Probability Models)

You can find a good discussion about this experiment intent to treat, etc in the context of the NEJM paper in an EconTalk podcast with Jim Manzi and Russ Roberts this past year: http://www.econtalk.org/archives/2013/05/jim_manzi_on_th.html 

Also, a nice profile of MIT economist Amy Finkelstein in a related story from Bloomberg: "MIT Economist Seeks Facts in Health-Care Policy Debate."  http://www.bloomberg.com/news/2014-01-03/mit-economist-seeks-facts-in-health-care-policy-debate.html 


References: 

"The Oregon Experiment--Effects of Medicaid on Clinical Outcomes," by Katherine Baicker, et al. New England Journal of Medicine, 2013; 368:1713-1722. http://www.nejm.org/doi/full/10.1056/NEJMsa1212321

Medicaid Increases Emergency-Department Use: Evidence from Oregon's Health Insurance Experiment. Sarah L. Taubman,Heidi L. Allen, Bill J. Wright, Katherine Baicker, and Amy N. Finkelstein. Science 1246183Published online 2 January 2014 [DOI:10.1126/science.1246183]

An Actuarial Application of Social Network Analysis

While the article from The Future Actuary linked to below doesn't really mention social network analysis (SNA), it speaks to insurance products designed to protect you from loss, personal liability, or related litigation issues involving social media. I'm sure there has already been plenty of work done in this area and if I get the chance I'll provide some updates. I would think knowledge of SNA would be critical i.e what is the relationship between centrality measures and network structure and risk? How can this help price products designed to protect individuals and businesses?

"As a social media entrepreneur, Matt Mullenweg put it: “we’re only one click away from a billion people.” While these technological advancements have made ‘getting connected’ one of the top priorities for employers and individuals, it doesn’t come without its perils. The increasing risks of legality/operation, security, and defamation are also just one click away. Maybe it’s time for social media risk management plans to take off. "

http://www.soa.org/News-and-Publications/Newsletters/The-Future-Actuary/2013/fall/The-Bane-of-Hyperconnectivity--Social-Media-Risk.aspx

Econometrics and Big Data

Thanks to Dave Giles at Econometrics Beat for pointing out Hal Varian's paper. Big Data: New Tricks for Econometrics

"I believe that these methods have a lot to offer and should be more widely known and used by economists. In fact, my standard advice to graduate students these days is “go to the computer science department and take a class in machine learning.” There have been very fruitful collaborations between computer scientists and statisticians in the last decade or so, and I expect collaborations between computer scientists and econometricians will also be productive in the future."

See also Economists as Data Scientists.

The Convergence of Big Data, Genomics, Biotechnology and Seed Choice

Some recent discussion including big data found on my applied economics blog EconomicSense:

-Big Data, Genomics, and Seed Choice
-AgriTalk Discussion

Also, Blake Hurst's American article:

"Tyler Cowen argues that we’re about to see an even wider disparity in incomes between the 10 to 15 percent of the population that can relate well to computers and the vast majority of us who will deliver services to the computer-savvy class. Farming may be one of the first industries to explore the validity of Cowen’s thesis. All of us involved in agriculture will soon have to decide whether we want to occupy the nostalgic niche providing artisanal beets and heritage pork to Cowen’s 10 percent, or whether we’ll roll the dice on surviving the transition to a data-driven agriculture."

Kentucky Association of Institutional Research (KAIR) Presentations

I'll be giving the following two talks at KAIR this year.
  
A Guide to Quasi-Experimental Designs

Quasi-experimental designs including propensity score methods, instrumental variables, regression discontinuity, and difference-in-difference estimators offer an inferentially rigorous alternative for program evaluation. In this guide, I begin with an introduction to the potential outcomes framework for rigorously characterizing selection bias and follow with discussions of quasi-experimental methods that may be useful to practitioners involved in program evaluation.

Paper: http://works.bepress.com/matt_bogard/24/
Slides: http://works.bepress.com/matt_bogard/27/

A Data Driven Analytic Strategy for Increasing Yield and Retention

As many Universities face the constraints of declining enrollment demographics, pressure from state governments for increased student success, as well as declining revenues, the costs of utilizing anecdotal evidence and intuition based on ‘gut’ feelings to make time and resource allocation decisions become significant. This presentation describes several data mining algorithms that could be used to develop a model to score university students based on their probability of enrollment and retention early in the enrollment funnel so that staff and administrators can work to recruit students that not only have an average or better chance of enrolling but also succeeding once they enroll. The use of SAS® Enterprise Miner and SAS® EBI server are discussed as an example of a business intelligence platform that can be used for implementing the models and delivering flexible web based reports to end users.

Paper: http://works.bepress.com/matt_bogard/26/
Slides:http://works.bepress.com/matt_bogard/28/

Causal Inference and Quasi-experimental Design Roundup

I’ve dedicated several posts recently to the subject of quasi-experimental designs and causal inference. I’ve tried to organize the following related links for a bigger picture.

First off, regression is often mischaracterized by a clinical view of  assumptions related to linearity. However, as Angrist and Pischke state:

"In fact, the validity of linear regression as an empirical tool does not turn on linearity either...The statement that regression approximates the CEF lines up with our view of empirical work as an effort to describe the essential features of statistical relationships, without necessarily trying to pin them down exactly."  - Mostly Harmless Econometrics, p. 26 & 29

For more discussion see:

Linear Regression as an Empirical Tool

In Mostly Harmless Econometrics, not only is linear regression rigorously developed and discussed, but quasi-experimental designs are given very heavy emphasis. As discussed in Cellini (2008):

… proxy variable, fixed effects, and difference in- differences approaches are becoming quite common. Indeed, these approaches have replaced basic multivariate regression as the new standard for education research in the economics literature”

The links below attempt to highlight at least in a heuristic sense, these methods and the issues they attempt to address:

Selection Bias and the Rubin Causal Model and Potential Outcomes Framework

Unobserved Heterogeneity and Endogeneity

Fixed Effects

A Toy Instrumental Variables Application

Propensity Score Matching

IPTW Regression

Regression Discontinuity Designs

Difference-in-Difference Estimators

Time Series Methods:

Interrupted Time Series (Intervention Analysis)

Intervention Analysis with SAS and R

Granger Causality