When Wicked Problems Meet Biased Data

In "Dissecting racial bias in an algorithm used to manage the health of populations" (Science, Vol 366 25 Oct. 2019) the authors discuss inherent racial bias in widely adopted algorithms in healthcare. In a nutshell these algorithms use predicted cost as a proxy for health status. Unfortunately, in healthcare, costs can proxy for other things as well:

"Black patients generate lesser medical expenses, conditional on health, even when we account for specific comorbidities. As a result, accurate prediction of costs necessarily means being racially biased on health."

So what happened? How can it be mitigated? What can be done going forward?

 In data science, there are some popular frameworks for solving problems. One widely known approach is the CRISP-DM framework. Alternatively, in The Analytics Lifecycle Toolkit a similar process is proposed:

(1) - Problem Framing
(2) - Data Sense Making
(3) - Analytics Product Development
(4) - Results Activation

The wrong turn in Albuquerque here may have been at the corner of problem framing and data understanding or data sense making.

The authors state:

"Identifying patients who will derive the greatest benefit from these programs is a challenging causal inference problem that requires estimation of individual treatment effects. To solve this problem health systems make a key assumption: Those with the greatest care needs will benefit the most from the program. Under this assumption, the targeting problem becomes a pure prediction public policy problem."

The distinctions between 'predicting' and 'explaining' have been made in the literature by multiple authors in the last two decades. The problem with this substitution has important implications. To quote Galit Shmueli:

"My thesis is that statistical modeling, from the early stages of study design and data collection to data usage and reporting, takes a different path and leads to different results, depending on whether the goal is predictive or explanatory."

Almost a decade before, Leo Brieman encouraged us to think outside the box when solving problems by considering multiple approaches:

"Approaching problems by looking for a data model imposes an a priori straight jacket that restricts the ability of statisticians to deal with a wide range of statistical problems. The best available solution to a data problem might be a data model; then again it might be an algorithmic model. The data and the problem guide the solution. To solve a wider range of data problems, a larger set of tools is needed."

A number of data analysts today may not be cognizant of the differences in predictive vs explanatory modeling and statistical inference. It may not be clear to them how that impacts their work. This could be related to background, training, or the kinds of problems they have worked on given their experience.  It is also important that we don't compartmentalize so much that we miss opportunities to approach our problem from a number of different angles (Leo Breiman's 'straight jacket') This is perhaps what happened in the Science article, once the problem was framed as a predictive modeling problem other modes of thinking may have shut down even if developers were aware of all of these distinctions.

The take away is that we think differently when doing statistical inference/explaining vs. predicting or doing machine learning. Making the substitution of one for the other impacts the way we approach the problem (things we care about, things we consider vs. discount etc.) and this impacts the data preparation, modeling, and interpretation.

For instance, in the Science article, after framing the problem as a predictive modeling problem, a pivotal focus became the 'labels' or target for prediction.

"The dilemma of which label to choose relates to a growing literature on 'problem formulation' in data science: the task of turning an often amorphous concept we wish to predict into a concrete variable that can be predicted in a given dataset."

As noted in the paper 'labels are often measured with errors that reflect structural inequalities.'

Addressing the issue with label choice can come with a number of challenges briefly alluded to in the article:

1) deep understanding of the domain - i.e subject matter expertise
2) identification and extraction of relevant data - i.e. data engineering and data governance
3) capacity to iterate and experiment - i.e. understanding causality, testing and measurement strategy

Data science problems in healthcare are wicked problems defined by interacting complexities with social, economic, and biological dimensions that transcend simply fitting a model to data. Expertise in a number of disciplines is required.

Bias in Risk Adjustment

In the Science article, the specific example was in relation to predictive models targeting patients for disease management programs. However, there are a number of other predictive modeling applications where these same issues can be prevalent in the healthcare space.

In Fair Regression for Health Care Spending, Sherri Rose and Anna Zink discuss these challenges in relation to popular regression based risk adjustment applications. Aligning with the analytics lifecycle discussed above, they point out there are several places where issues of bias can be addressed including pre-processing, model fitting, and post processing stages of analysis. In this article they focus largely on the modeling stage leveraging a number of constrained and penalized regression algorithms designed to optimize fairness. This work looks really promising, but the authors point out a number of challenges related to scalability and optimizing fairness across a number of metrics or groups.

Toward Causal AI and ML

Previously I referenced Galit Shmueli's work that discussed how differently we approach and think about predictive vs explanatory modeling. In the Book of Why, Judea Pearl discusses causal inferential thinking:

"Causal Analysis is emphatically not just about data; in causal analysis we must incorporate some understanding of the process that produces the data and then we get something that was not in the data to begin with." 

There is currently a lot of work fusing machine learning and causal inference that could create more robust learning algorithms. For example, Susan Athey's work with causal forests, Leon Bottou's work related to causal invariance, and Elias Barenboim's work on the data fusion problem.  This work, including the kind of work mentioned before related to fair regression will help inform the next generation of predictive modeling, machine learning, and causal inference models in the healthcare space that hopefully will represent a marked improvement over what is possible today.

However, we can't wait half a decade or more while the theory is developed and adopted by practitioners. In the Science article, the authors found alternative metrics for targeting disease management programs besides total costs that calibrate much more fairly across groups. Bridging the gap in other areas will require a combination of awareness of these issues and creativity throughout the analytics product lifecycle. As the authors conclude:

"careful choice can allow us to enjoy the benefits of algorithmic predictions while minimizing the risks."

References and Additional Reading:

This paper was recently discussed on the Casual Inference podcast.

Annual Review of Public Health 2020 41:1

Rudin, C. Stop explaining black box machine learning models for high stakes decisions and use interpretable models instead. Nat Mach Intell 1, 206–215 (2019) doi:10.1038/s42256-019-0048-x

Breiman, Leo. Statistical Modeling: The Two Cultures (with comments and a rejoinder by the author). Statist. Sci. 16 (2001), no. 3, 199--231. doi:10.1214/ss/1009213726. https://projecteuclid.org/euclid.ss/1009213726

Shmueli, G., "To Explain or To Predict?", Statistical Science, vol. 25, issue 3, pp. 289-310, 2010.

Fair Regression for Health Care Spending. Anna Zink, Sherri Rose. arXiv:1901.10566v2 [stat.AP]

Wicked Problems and The Role of Expertise and AI in Data Science

In 2018, an article in Science characterized the challenge of pesticide resistance as a wicked problem:

“If we are to address this recalcitrant issue of pesticide resistance, we must treat it as a “wicked problem,” in the sense that there are social, economic, and biological uncertainties and complexities interacting in ways that decrease incentives for actions aimed at mitigation.”

In graduate school, I worked on this same problem, attempting to model the social and economic systems with game theory and behavioral economics and capturing biological complexities leveraging population genetics. 

Wicked vs. Kind Environments

In data science, we also have 'wicked' learning environments in which we try to train our models. In the EconTalk podcast with Russ Roberts, Mastery, Specialization, and Range, David Epstein discusses wicked and kind learning environments:

"The way that chess works makes it what's called a kind learning environment. So, these are terms used by psychologist Robin Hogarth. And what a kind learning environment is, is one where patterns recur; ideally a situation is constrained--so, a chessboard with very rigid rules and a literal board is very constrained; and, importantly, every time you do something you get feedback that is totally obvious...you see the consequences. The consequences are completely immediate and accurate. And you adjust accordingly. And in these kinds of kind learning environments, if you are cognitively engaged you get better just by doing the activity."

"On the opposite end of the spectrum are wicked learning environments. And this is a spectrum, from kind to wicked. Wicked learning environments: often some information is hidden. Even when it isn't, feedback may be delayed. It may be infrequent. It may be nonexistent. And it maybe be partly accurate, or inaccurate in many of the cases. So, the most wicked learning environments will reinforce the wrong types of behavior."

As discussed in the podcast, many problems fall within some spectrum ranging between very kind environments like Chess to more complex environments like self driving cars or medical diagnosis. What do experts have to offer where AI/ML falls short? The type of environment determines to a great extent the scope of disruption we might be able to expect from AI applications.

The Role of Human Expertise

In Thinking Fast and Slow, Kahneman discusses two conditions for acquiring skill:

1) an environment that is sufficiently regular to be predictable

2) an opportunity to learn these regularities through prolonged practice

This sounds a lot like the 'kind' environments discussed above. Based on research by Robin Hogarth, Kahneman also makes these distinctions describing 'wicked' environments as those environments in which those with expertise are likely to learn the wrong lessons from experience. The problem is that with wicked environments, experts often default to heuristics which can lead to wrong conclusions. Even if aware of these biases, social norms often nudge experts into the wrong direction. Kahneman gives an example involving physicians:

"Generally it is considered a weakness and a sign of vulnerability for clinicians to appear unsure. Confidence is valued over uncertainty and there is a prevailing censure against disclosing uncertainty to patients...acting on pretended knowledge is often the preferred solution."

This likely explains many of the mistakes and low value care that are problematic with healthcare delivery as well as dissatisfaction with both the quality and costs of healthcare. How many of us want our physicians to pretend to know what they are talking about? On the other hand, how many people are willing to accept an answer from their physician that rhymes with "let me look this up and get back to you later." 

One advantage AI may have over experts in kind environments is as Kahneman puts it, the opportunity to learn through prolonged practice. Machine learning can handle many more training examples than a human so to speak.

Even in kind environments, an expert may swing and miss when dealing with cases where the correct decision is like a pitch straight over the plate. One reason Kahneman discusses in Thinking Fast and Slow is the idea of 'ego' depletion. This is related to the idea that mental energy can become exhausted after significant exertion. As self-control breaks down, its easy to default to heuristics and biases that can lead to decisions that look like careless mistakes. This would certainly apply to physicians given the number of stories we hear about burnout in the profession. 

The solution seems to be what polymath economist Tyler Cowen suggested several years ago in the econtalk podcast discussion he had about his book Average is Over with Russ Roberts:

"I would stress much more that humans can always complement robots. I'm not saying every human will be good at this. That's a big part of the problem. But a large number of humans will work very effectively with robots and become far more productive, and this will be one of the driving forces behind that inequality."

Imagine the clinical situation where a physician's 'ego' is substantially depleted from a difficult case. They could then lean on AI to prevent mistakes treating more routine decisions that follow. Or perhaps leveraging AI tools, a clinician could conserve additional mental energy throughout the day so that they are less likely to default to heuristics when they encounter more complex issues. The way this synergy materializes is uncertain, but it will certainly continue to involve substantial expertise on the part of many professionals going forward. Together human expertise and AI might have the greatest chance tackling the most wicked problems.

References:

Wicked evolution: Can we address the sociobiological dilemma of pesticide resistance? | Science  https://science.sciencemag.org/content/360/6390/728.full

Thinking Fast and Slow. Daniel Kahneman. 2011

EconTalk:David Epstein on Mastery, Specialization, and Range
https://www.econtalk.org/david-epstein-on-mastery-specialization-and-range/

EconTalk: Tyler Cowen on Inequality, the Future, and Average is Over
https://www.econtalk.org/tyler-cowen-on-inequality-the-future-and-average-is-over/

Modeling Claims with Linear vs. Non-Linear Difference-in-Difference Models

Previously I have discussed the issues with modeling claims costs. Typically medical claims exhibit non-negative highly skewed values with high zero mass and heterskedasticity. The most commonly suggested approach to addressing these distributional concerns in the literature call for the use of non-linear GLM models.  However, as previously discussed (see here and here) there are challenges with using difference-in-difference models in the context of GLM models. So once again, the gap between theory and application presents challenges, tradeoffs, and compromises that need to be made by the applied econometrician.

In the past I have written about the accepted (although controversial in some circles) practice of leveraging linear probability models to estimate marginal effects in applied work when outcomes are dichotomous. But what about doing this in the context of claims analysis? In my original post regarding the challenges of using difference-in-differences with claims I speculated:

"So as Angrist and Pischke might ask, what is an applied guy to do? One approach even in the context of skewed distributions with high mass points (as is common in the healthcare econometrics space) is to specify a linear model. For count outcomes (utilization like ER visits or hospital admissions are often dichotomized and modeled by logit or probit models) you can just use a linear probability model. For skewed distributions with heavy mass points, dichotomization with a LPM may also be an attractive alternative."

 I have found that this advice is pretty consistent with the social norms and practices in the field.

In their analysis of the ACA Cantor, et al (2012) leverage linear probability models for difference-in-differences for healthcare utilization stating:

"Linear probability models are fit to produce coefficients that are direct estimates of the relevant policy impacts and are easily interpreted as percentage point changes in coverage outcomes. This approach has been applied in earlier evaluations of insurance market reforms (Buchmueller and DiNardo 2002; Monheit and Steinberg Schone 2004;  Levine, McKnight, and Heep 2011;  Monheit et al. 2011). It also avoids complications associated with estimation and interpretation of multiple interaction terms and their standard errors in logit or probit models (Ai and Norton 2003)."

Jhamb et al (2015) use LPMs for dichotomous outcomes as well as OLS models for counts in a DID framework.

Interestingly, Deb and Norton (2018) discuss an approach to address the challenges of DID in a GLM framework head on:

"Puhani argued, using the potential outcomes framework, that the treatment effect on the treated in the difference-in-difference regression equals the expected value of the dependent variable for the treatment group in the post period with treatment compared with the hypothetical expected value of the dependent variable for the treatment group in the post period if they had not received treatment. In nonlinear models, the treatment effect on the treated equals the difference in two predicted values. It always has the same sign as the coefficient on the interaction term. Because we estimate many nonlinear models using a difference-in-differences study design, we report the treatment effect on the treated in all tables of results."

In presenting their results they compare their GLM based approach to results from linear models of healthcare expenditures. While they argue the differences are substantial in supporting their approach, I did not find the OLS estimate (-$323.4) to be practically different from the second part (conditional on positive) of the two part GLM model (-$321.4), although the combined results from the two part model had large practical differences from OLS. It does not appear they compared a two-part GLM to a two-part linear model (which could be problematic if the first part OLS model gave probabilities greater than 1 or less than zero). In their paper they cited a number of authors using linear difference-in-differences to model claims you will find below.

See the references below for a number of examples (including those cited above).

Related: Linear Literalism and Fundamentalist Econometrics

References:

Cantor JC, Monheit AC, DeLia D, Lloyd K. Early impact of the Affordable Care Act on health insurance coverage of young adults. Health Serv Res. 2012;47(5):1773-90.

Modeling Health Care Expenditures and Use
Partha Deb and Edward C. Norton
Annual Review of Public Health 2018 39:1, 489-505

Buchmueller T, DiNardo J. “Did Community Rating Induce an Adverse Selection Death Spiral? Evidence from New York, Pennsylvania and Connecticut” American Economic Review. 2002;92(1):280–94.

Monheit AC, Cantor JC, DeLia D, Belloff D. “How Have State Policies to Expand Dependent Coverage Affected the Health Insurance Status of Young Adults?” Health Services Research. 2011;46(1 Pt 2):251–67

Amuedo-Dorantes C, Yaya ME. 2016. The impact of the ACA’s extension of coverage to dependents on young adults’ access to care and prescription drugs. South. Econ. J. 83:25–44

Barbaresco S, Courtemanche CJ, Qi Y. 2015. Impacts of the Affordable Care Act dependent coverage provision on health-related outcomes of young adults. J. Health Econ. 40:54–68

Jhamb J, Dave D, Colman G. 2015. The Patient Protection and Affordable Care Act and the utilization of health care services among young adults. Int. J. Health Econ. Dev. 1:8–25

Sommers BD, Buchmueller T, Decker SL, Carey C, Kronick R. 2013. The Affordable Care Act has led
to significant gains in health insurance and access to care for young adults. Health Aff. 32:165–74

Modeling Healthcare Claims as a Dependent Variable

Healthcare claims present challenges to the applied econometrician. Claims costs typically exhibit a large number of zero values (high zero mass), extreme skewness, and heteroskedasticity. Below is a histogram depicting the distributional properties typical of claims data.

The literature (see references below) addresses a number of approaches (i.e. log models, GLM, and two part models) often used for modeling claims data. However, without proper context the literature can leave one with a lot of unanswered questions, or several seemingly plausible answers to the same question.

The department of Veteran's Affairs runs a series of healthcare econometrics cyberseminars covering these topics. Particularly, they have two video lectures devoted to modeling healthcare costs as a dependent variable.

https://www.hsrd.research.va.gov/cyberseminars/series.cfm#hec3

Principles discussed include:

1) Despite what is taught in a lot of statistics classes about skewed data, in claims analysis we usually DO want to look at MEANS not MEDIANS.

2) Why logging claims and then running analysis on the logged data to deal with skewness is probably not the best practice in this context.

3) How adding a small constant number to zero values prior to logging can lead to estimates that are very sensitive to the choice of constant value.

4) Why in many cases it could be a bad idea to exclude ‘high cost claimants’ from an analysis without good reasons. This probably should not be an arbitrary routine practice.

5)When and why you may or may not prefer ‘2-part models’

Note: Utilization data like ER visits, primary care visits and hospital admissions are also typically non-negative and skewed with high mass points.  Utilization can be modeled as counts using poisson, negative binomial, or zero-inflated poisson and zero inflated negative binomial models in a GLM framework although not discussed here.

References:

Mullahy, John. "Much Ado Abut Two: Reconsidering Retransformation And The Two-Part Model In Health Econometrics," Journal of Health Economics, 1998, v17(3,Jun), 247-281.

Liu L, Cowen ME, Strawderman RL, Shih Y-CT. A Flexible Two-Part Random Effects Model for Correlated Medical Costs. Journal of health economics. 2010;29(1):110-123. doi:10.1016/j.jhealeco.2009.11.010.

Too much ado about two-part models
and transformation? Comparing methods of modeling Medicare expenditures
Melinda Beeuwkes Buntin a,∗, Alan M. Zaslavsky
Journal of Health Economics 23 (2004) 525–542

REVIEW OF STATISTICAL METHODS FOR ANALYSING HEALTHCARE RESOURCES AND COSTS
BORISLAVA MIHAYLOVAa,, ANDREW BRIGGSb, ANTHONY O’HAGANcand SIMON G. THOMPSON
Health Econ. 20: 897–916 (2011)

Generalized modeling approaches to risk adjustment of skewed outcomes data.
J Health Econ. 2005 May;24(3):465-88.
Manning WG1, Basu A, Mullahy J.

Econometric Modeling of Health Care Costs and Expenditures: A Survey of Analytical Issues and Related Policy Considerations . John Mullahy. Medical Care. Vol. 47, No. 7, Supplement 1: Health Care Costing: Data, Methods, Future Directions (Jul., 2009), pp. S104-S108

Analyzing Health Care Costs: A Comparison of
Statistical Methods Motivated by Medicare Colorectal Cancer Charges. MICHAEL GRISWOLD, GIOVANNI PARMIGIANI,ARNIE POTOSKY,JOSEPH LIPSCOMB. Biostatistics (2004), 1, 1, pp. 1–23

Estimating log models: to transform or not to transform? Willard G. Manning and John Mullahy. Journal of Health Economics 20 (2001) 461–494

Angrist, J.D. Estimation of Limited Dependent Variable Models With Dummy Endogenous Regressors: Simple Strategies for Empirical Practice. Journal of Business & Economic Statistics January 2001, Vol. 19, No. 1.

P Dier, D Yanez, A Ash, M Hornbrook, DY Lin. Methods for analyzing health care utilization and costs Ann Rev Public Health (1999) 20:125-144
Lachenbruch P. A. 2001. “Comparisons of two-part models with competitors” Statistics in Medicine, 20:1215–1234.

Lachenbruch P.A. 2001. “Power and sample size requirements for two-part models” Statistics in Medicine, 20:1235–1238.

 Diehr,P. ,Yanez,D. Ash, A. Hornbrook, M. & Lin, D. Y. 1999 “Methods for analyzing health care utilization and costs.” Annu. Rev. Public Health, 20:125–44.

Instrumental Variables vs. Intent to Treat

 "ITT analysis includes every subject who is randomized according to randomized treatment assignment. It ignores noncompliance, protocol deviations, withdrawal, and anything that happens after randomization. ITT analysis is usually described as “once randomized, always analyzed”.

"ITT analysis avoids overoptimistic estimates of the efficacy of an intervention resulting from the removal of non-compliers by accepting that noncompliance and protocol deviations are likely to occur in actual clinical practice"  - Gupta, 2011

 In Mastering Metrics, Angrist and Pischke describe intent-to-treat analysis:

"In randomized trials with imperfect compliance, when treatment assignment differs from treatment delivered, effects of random assignment...are called intention-to-treat (ITT) effects. An ITT analysis captures the causal effect of being assigned to treatment."

While treatment assignment is random, non-compliance is not! Therefore if instead of using intent to treat comparisons we compared those actually treated to those untreated we would get biased results, because this is essentially making uncontrolled comparisons between treated and untreated subjects.

Angrist and Pishke describe how instrumental variables can be used in this context:

 “The instrumental variables (IV) method harnesses partial or incomplete random assignment, whether naturally occurring or generated by researchers"

 "Instrumental variable methods allow us to capture the causal effect of treatment on the treated in spite of the nonrandom compliance decisions made by participants in experiments....Use of randomly assigned intent to treat as an instrumental variable for treatment delivered eliminates this source of selection bias."

In  Intent-to-Treat vs. Non-Intent-to-Treat Analyses under Treatment Non-Adherence in Mental Health Randomized Trials there is a nice discussion of ITT and IV methods with applications related to clinical research.  Below is a nice treatment of IV in this context:

“Instrumental variables are assumed to emulate randomization variables, unrelated to unmeasured confounders influencing the outcome. In the case of randomized trials, the same randomized treatment assignment variable used in defining treatment groups in the ITT analysis is instead used as the instrumental variable in IV analyses. In particular, the instrumental variable is used to obtain for each patient a predicted probability of receiving the experimental treatment. Under the assumptions of the IV approach, these predicted probabilities of receipt of treatment are unrelated to unmeasured confounders in contrast to the vulnerability of the actually observed receipt of treatment to hidden bias. Therefore, these predicted treatment probabilities replace the observed receipt of treatment or treatment adherence in the AT model to yield an estimate of the as-received treatment effect protected against hidden bias when all of the IV assumptions hold.”

A great example of IV and ITT applied to health care can be found in Finkelstein et. al. (2013 & 2014) - See the Oregon Medicaid Experiment, Applied Econometics, and Causal Inference.

Over at the Incidental Economist, there was a nice discussion of ITT in the context of medical research that does a good job of explaining the rationale as well as when departures from ITT make more sense (such as safety and non-inferiority trials).

See also:  
Instrumental Explanations of Instrumental Variables

A Toy IV Application

Other IV Related Posts

References: 

Mastering ’Metrics:
The Path from Cause to Effect
Joshua D. Angrist & Jörn-Steffen Pischke
2015

Gupta, S. K. (2011). Intention-to-treat concept: A review. Perspectives in Clinical Research, 2(3), 109–112. http://doi.org/10.4103/2229-3485.83221

Ten Have, T. R., Normand, S.-L. T., Marcus, S. M., Brown, C. H., Lavori, P., & Duan, N. (2008). Intent-to-Treat vs. Non-Intent-to-Treat Analyses under Treatment Non-Adherence in Mental Health Randomized Trials. Psychiatric Annals, 38(12), 772–783. http://doi.org/10.3928/00485713-20081201-10

"The Oregon Experiment--Effects of Medicaid on Clinical Outcomes," by Katherine Baicker, et al. New England Journal of Medicine, 2013; 368:1713-1722. http://www.nejm.org/doi/full/10.1056/NEJMsa1212321

Medicaid Increases Emergency-Department Use: Evidence from Oregon's Health Insurance Experiment. Sarah L. Taubman,Heidi L. Allen, Bill J. Wright, Katherine Baicker, and Amy N. Finkelstein. Science 1246183Published online 2 January 2014 [DOI:10.1126/science.1246183] 

Detry MA, Lewis RJ. The Intention-to-Treat PrincipleHow to Assess the True Effect of Choosing a Medical Treatment. JAMA. 2014;312(1):85-86. doi:10.1001/jama.2014.7523

Molecular Genetics and Economics

A really interesting article in JEP:

A slice:

"In fact, the costs of comprehensively genotyping human subjects have fallen to the point where major funding bodies, even in the social sciences, are beginning to incorporate genetic and biological markers into major social surveys. The National Longitudinal Study of Adolescent Health, the Wisconsin Longitudinal Study, and the Health and Retirement Survey have launched, or are in the process of launching, datasets with comprehensively genotyped subjects…These samples contain, or will soon contain, data on hundreds of thousands of genetic markers for each individual in the sample as well as, in most cases, basic economic variables. How, if at all, should economists use and combine molecular genetic and economic data? What challenges arise when analyzing genetically informative data?"


Link:

https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3306008/


Reference:
Beauchamp JP, Cesarini D, Johannesson M, et al. Molecular Genetics and Economics. The journal of economic perspectives : a journal of the American Economic Association. 2011;25(4):57-82.

What's the difference between difference-in-difference models in a linear vs nonlinear context?

A while back I discussed a powerful methodology for identification of causal effects from both a selection on observables and unobservables context, namely combining propensity score matching and difference-in-differences. 

But recently I ran across a tweet from Felix Bethke (https://twitter.com/F_Bethke) sharing a blog post by Tom Pepinsky related to plug and play models. At the risk of oversimplifying, the take away was that we can't just take a methodology like DID used in a standard linear regression context and necessarily 'plug it into'  a non-linear context and get the same results. (often we see arguments going the other way around, we can't use linear models in a non-linear context but that is a different battle for another day).  I highly recommend Tom's post for more details and he links to a number of papers that clarify the issues in a very technical sense.

In a linear difference-in-difference (DID) analysis, identification of causal effects hinge on a common trend assumption and interpretation of the estimated regression coefficient on the time x treatment interaction term.

y = b₀ + b₁ x + b₂ t+b₃ x*t + e

In Tom's post, and some of the papers, specific attention is given to how the interpretation of the interaction term (and our estimated treatment effect or b3 in a specification like above) changes in a logit or probit context and its something quite different from the causal effect of interest.

I was specifically interested in knowing, is this an issue just for probit and logit models or other nonlinear models, like GLM models in general. For instance, in the healthcare economics literature, its very common to use probit or logit models in a two part modeling context where the second part of a two part model is a GLM model with a log link and gamma distribution. And I have seen some papers using a difference-in-differences across the board with these models.

I took a look at a couple of papers and it appears that these issues are a concern for any GLM model.

In a Health Services Research paper, Karaca-Mandic et al discuss these issues and in the abstract imply that this would apply to log transformed models often used in healthcare economics:

"We discuss the motivation for including interaction terms in multivariate analyses. We then explain how the straightforward interpretation of interaction terms in linear models changes in nonlinear models, using graphs and equations. We extend the basic results from logit and probit to difference‐in‐differences models, models with higher powers of explanatory variables, other nonlinear models (including log transformation and ordered models), and panel data models."

After pointing out several issues, they state:

"It is important to understand that the issues about interaction terms discussed here apply to all nonlinear models, including log transformation models"

More specifically, what are these issues, at least at a high level? Recall, difference-in-difference models are a special case of fixed effects panel data models, where unobserved differences and individual specific effects essentially cancel out providing clean identification of causal effects.  For this to work in the DID framework, a common trends assumption is required.  In the referenced paper below, Lechner points out (quite rigorously in the context of the potential outcomes framework):

"We start with a “natural” nonlinear model with a linear index structure which is transformed by a link function, G(·), to yield the conditional expectation of the potential outcome.....The common trend assumption relies on differencing out specific terms of the unobservable potential outcome, which does not happen in this nonlinear specification... Whereas the linear specification requires the group specific differences to be time constant, the nonlinear specification requires them to be absent. Of course, this property of this nonlinear specification removes the attractive feature that DiD allows for some selection on unobservable group and individual specific differences. Thus, we conclude that estimating a DiD model with the standard specification of a nonlinear model would usually lead to an inconsistent estimator if the standard common trend assumption is upheld. In other words, if the standard DiD assumptions hold, this nonlinear model does not exploit them (it will usually violate them). Therefore, estimation based on this model does not identify the causal effect "

Because they demonstrate that this applies to any GLM specification/link function, this seems to strike a blow to using DID in the context of a lot of the modeling approaches used in healthcare economics or any other field relying on similar GLM specifications.

So as Angrist and Pischke might ask, what is an applied guy to do? One approach even in the context of skewed distributions with high mass points (as is common in the healthcare econometrics space) is to specify a linear model. For count outcomes (utilization like ER visits or hospital admissions are often dichotomized and modeled by logit or probit models) you can just use a linear probability model. For skewed distributions with heavy mass points, dichotomization with a LPM may also be an attractive alternative.

References:

Special thanks to tweets and additional input from Tom Pepinsky and Marc Bellemare.

Interaction Terms in Nonlinear Models
Pinar Karaca-Mandic, Edward C. Norton, and Bryan Dowd
HSR: Health Services Research 47:1, Part I (February 2012)

The Estimation of Causal Effects by Difference-in-Difference Methods
By Michael Lechner Foundations and Trends in Econometrics
Vol. 4, No. 3 (2010) 165–224

Do We Really Need Zero-Inflated Models?-Paul Allison

Paul Allison discusses zero inflated vs negative binomial models in a post I stumbled across recently. Also William Greene and Paul go back and forth on some technical distinctions and nuances (which may be quite important) in the comments.

http://m.statisticalhorizons.com/?url=http%3A%2F%2Fstatisticalhorizons.com%2Fzero-inflated-models

"In all data sets that I've examined, the negative binomial model fits much better than a ZIP model, as evaluated by AIC or BIC statistics. And it's a much simpler model to estimate and interpret. So if the choice is between ZIP and negative binomial, I'd almost always choose the latter."

"But what about the zero-inflated negative binomial (ZINB) model? It's certainly possible that a ZINB model could fit better than a conventional negative binomial model regression model. But the latter is a special case of the former, so it's easy to do a likelihood ratio test to compare them (by taking twice the positive difference in the log-likelihoods). In my experience, the difference in fit is usually trivial..."

"So next time you're thinking about fitting a zero-inflated regression model, first consider whether a conventional negative binomial model might be good enough. Having a lot of zeros doesn't necessarily mean that you need a zero-inflated model."

Mendelian Instruments (Applied Econometrics meets Bioinformatics)

Recently I defended the use of quasi-experimental methods in wellness studies, and a while back I sort of speculated that genomic data might be useful in a quasi-experimental setting-but wasn’t sure how: 

“If causality is the goal, then merge 'big data' from the gym app with biometrics and the SNP profiles and employ some quasi-expermental methodology to investigate causality.”

Then this morning at marginal revolution I ran across a link to a blog post that mentioned exploiting mendelian variation as instruments for a particular study related to alcohol consumption.

This piece gives a nice intro I think:

Stat Med. 2008 Apr 15;27(8):1133-63. Mendelian randomization: using genes as instruments for making causal inferences in epidemiology.

Lawlor DA1, Harbord RM, Sterne JA, Timpson N, Davey Smith G

Link: http://www.ncbi.nlm.nih.gov/pubmed/17886233

“Observational epidemiological studies suffer from many potential biases, from confounding and from reverse causation, and this limits their ability to robustly identify causal associations. Several high-profile situations exist in which randomized controlled trials of precisely the same intervention that has been examined in observational studies have produced markedly different findings. In other observational sciences, the use of instrumental variable (IV) approaches has been one approach to strengthening causal inferences in non-experimental situations. The use of germline genetic variants that proxy for environmentally modifiable exposures as instruments for these exposures is one form of IV analysis that can be implemented within observational epidemiological studies. The method has been referred to as 'Mendelian randomization', and can be considered as analogous to randomized controlled trials. This paper outlines Mendelian randomization, draws parallels with IV methods, provides examples of implementation of the approach and discusses limitations of the approach and some methods for dealing with these.”

See also: QTL analysis in R and A Toy Instrumental Variable Application

Healthcare Analytics at SAS Global Forum 2015

I was not able to attend this year's SAS Global Forum, but have had a chance to browse the numerous session papers as well as enjoy some live content. The conference page has a searchable catalog and for each paper you will find links to similar sessions in the sidebar. There were around 5000 attendees at this year's conference and over 600 papers. For a 2 1/2 day conference that's more than 200 papers to cover per day. Below is a selection of some papers related to healthcare analytics. If we widen the search to include papers related to other fields, but applications in healthcare analtyics the selection would probably double. I'm sure I missed something, and would be glad to know if you had a favorite paper or presentation you'd like to share in the comments.


1329 - Causal Analytics: Testing, Targeting, and Tweaking to Improve Outcomes This session is an introduction to predictive analytics and causal analytics in the context of improving outcomes. The session covers the following topics: 1) Basic... View More 20 minutes Breakout Jason Pieratt

1340 - Using SAS® Macros to Flag Claims Based on Medical Codes Many epidemiological studies use medical claims to identify and describe a population. But finding out who was diagnosed, and who received treatment, isn't always simple.... View More 50 minutes Breakout Andy Karnopp

2382 - Reducing the Bias: Practical Application of Propensity Score Matching in Health-Care Program Evaluation To stay competitive in the marketplace, health-care programs must be capable of reporting the true savings to clients. This is a tall order, because most health-care programs... View More 20 minutes Breakout Amber Schmitz

2920 - Text Mining Kaiser Permanente Member Complaints with SAS® Enterprise Miner™ This presentation details the steps involved in using SAS® Enterprise Miner™ to text mine a sample of member complaints. Specifically, it describes how the Text Parsing, Text... View More 30 minutes E-Poster Amanda Pasch

3214 - How is Your Health? Using SAS® Macros, ODS Graphics, and GIS Mapping to Monitor Neighborhood and Small-Area Health Outcomes With the constant need to inform researchers about neighborhood health data, the Santa Clara County Health Department created socio-demographic and health profiles for 109... View More 20 minutes Breakout Roshni Shah

3254 - Predicting Readmission of Diabetic Patients Using the High-Performance Support Vector Machine Algorithm of SAS® Enterprise Miner™ 13.1 Diabetes is a chronic condition affecting people of all ages and is prevalent in around 25.8 million people in the U.S. The objective of this research is to predict the... View More 20 minutes Breakout Hephzibah Munnangi

3281 - Using SAS® to Create Episodes-of-Hospitalization for Health Services Research An essential part of health services research is describing the use and sequencing of a variety of health services. One of the most frequently examined health services is... View More 20 minutes Breakout Meriç Osman

3282 - A Case Study: Improve Classification of Rare Events with SAS® Enterprise Miner™ Imbalanced data are frequently seen in fraud detection, direct marketing, disease prediction, and many other areas. Rare events are sometimes of primary interest. Classifying... View More 20 minutes Breakout Ruizhe Wang

3411 - Identifying Factors Associated with High-Cost Patients Research has shown that the top five percent of patients can account for nearly fifty percent of the total healthcare expenditure in the United States. Using SAS® Enterprise... View More 30 minutes E-Poster Jialuo Cheng

3488 - Text Analytics on Electronic Medical Record Data This session describes our journey from data acquisition to text analytics on clinical, textual data. 50 minutes Breakout Mark Pitts

3560 - A SAS Macro to Calculate the PDC Adjustment of Inpatient Stays The Centers for Medicare & Medicaid Services (CMS) uses the Proportion of Days Covered (PDC) to measure medication adherence. There is also some PDC-related research based on... View More 20 minutes Breakout anping chang


3600 - When Two Are Better Than One: Fitting Two-Part Models Using SAS In many situations, an outcome of interest has a large number of zero outcomes and a group of nonzero outcomes that are discrete or highly skewed.  For example, in modeling... View More 20 minutes Breakout Laura Kapitula

3740 - Risk-Adjusting Provider Performance Utilization Metrics Pay-for-performance programs are putting increasing pressure on providers to better manage patient utilization through care coordination, with the philosophy that good... View More 50 minutes Breakout Tracy Lewis

3741 - The Spatio-Temporal Impact of Urgent Care Centers on Physician and ER Use The unsustainable trend in healthcare costs has led to efforts to shift some healthcare services to less expensive sites of care. In North Carolina, the expansion of urgent... View More 50 minutes Breakout Laurel Trantham

3760 - Methodological and Statistical Issues in Provider Performance Assessment With the move to value-based benefit and reimbursement models, it is essential toquantify the relative cost, quality, and outcome of a service. Accuratelymeasuring the cost... View More 50 minutes Breakout Daryl Wansink

SAS1855 - Using the PHREG Procedure to Analyze Competing-Risks Data Competing risks arise in studies in which individuals are subject to a number of potential failure events and the occurrence of one event might impede the occurrence of other... View More 20 minutes Breakout Ying So


SAS1900 - Establishing a Health Analytics Framework Medicaid programs are the second largest line item in each state’s budget. In 2012, they contributed $421.2 billion, or 15 percent of total national healthcare expenditures.... View More 20 minutes Breakout Krisa Tailor


SAS1951 - Using SAS® Text Analytics to Examine Labor and Delivery Sentiments on the Internet In today’s society, where seemingly unlimited information is just a mouse click away, many turn to social media, forums, and medical websites to research and understand how mothers feel about the birthing process. Mining the data in these resources helps provide an understanding of what mothers value and how they feel. This paper shows the use of SAS® Text Analytics to gather, explore, and analyze reports from mothers to determine their sentiment about labor and delivery topics. Results of this analysis could aid in the design and development of a labor and delivery survey and be used to understand what characteristics of the birthing process yield the highest levels of importance. These resources can then be used by labor and delivery professionals to engage with mothers regarding their labor and delivery preferences. View Less 20 minutes Breakout Michael Wallis

A Credibility Revolution in Wellness Program Analysis?

Last november there was a post on the Health Affairs blog related to the evaluation of wellness programs. Here are some tidbits:

"This blog post will consider the results of two compelling study designs — population-based wellness-sensitive medical event analysis, and randomized controlled trials (RCTs). Then it will look at the popular, although weaker, participant vs. non-participant study design."

"More often than not wellness studies simply compare participants to “matched” non-participants or compare a subset of participants (typically high-risk individuals) to themselves over time."

“Looking at how participants improve versus non-participants…ignores self-selection bias. Self-improvers are likely to be drawn to self-improvement programs, and self-improvers are more likely to improve.” Further, passive non-participants can be tracked all the way through the study since they cannot “drop out” from not participating, but dropouts from the participant group—whose results would presumably be unfavorable—are not counted and are considered lost to follow-up. So the study design is undermined by two major limitations, both of which would tend to overstate savings."

Does Wellness need a credibility revolution?

These criticisms are certainly valid, however, my thoughts are that panel methods, difference-in-difference and propensity score matching fit firmly in the Rubin Causal Model or potential outcomes framework for addressing issues related to selection bias. And what about examples of more robust quasi experimental approaches (like instrumental variables)?   These are all methods that are meant to deal specifically with the drawbacks of self comparisons and the issues mentioned above by the authors, and are at the heart of techniques related to the credibility revolution in econometrics.

A RCT is by far the most reliable way to identify treatment effects, but I know when it comes to applied work, RCT just isn't happening for a lot of obvious reasons. As Marc Bellemare might say, let the credibility revolution flow through you!

***this post was revised July 22, 2016, originally titled "Are Quasi-Experimental Designs Off the Table in Wellness Program Analysis"

Linear Probability Models for Skewed Distributions with High Mass Points

There are a lot of methods discussed in the literature related to modeling skewed distributions with high mass points including log transformations, two part models,  GLM etc. In some previous posts I have discussed linear probability models in the context of causal inference.  I've also discussed the use of quantile regression as a strategy to model highly skewed continuous and count data. Mullahy (2009) alludes to the use of quantile regression as well:

"Such concerns should translate into empirical strategies that target the high-end parameters of particular interest, e.g. models for Prob(y ≥ k | x) or quantile regression models"

The focus on high end parameters  using linear probability models is mentioned in Angrist and Pischke (2009) :

"COP [conditional-on-positive] effects are sometimes motivated by a researcher's sense that when the outcome distribution has a mass point-that is, when it piles up on a particular value, such as zero-or has a heavily skewed distribution, or both, then an analysis of effects on averages misses something. Analysis of effects on averages indeed miss some things, such as changes in the probability of specific values or a shift in quantiles away from the median. But why not look at these distribution effects directly? Distribution outcomes include the likelihood that annual medical expenditures exceed zero, 100 dollars, 200 dollars, and so on. In other words, put 1[Yi > c] for different choices of c on the left hand side of the regression of interest...the idea of looking directly at distribution effects with linear probability models is illustrated by Angrist (2001),...Alternatively, if quantiles provide a focal point, we can use quantile regressions to model them."

References:

Mostly Harmless Econometrics. Angrist and Pischke. 2009

Angrist, J.D. Estimation of Limited Dependent Variable Models With Dummy Endogenous Regressors: Simple Strategies for Empirical Practice. Journal of Business & Economic Statistics January 2001, Vol. 19, No. 1.

ECONOMETRIC MODELING OF HEALTH CARE COSTS AND EXPENDITURES: A SURVEY OF ANALTICAL ISSUES AND RELATED POLICY CONSIDERATIONS
John Mullahy Univ. of Wisconsin-Madison
January 2009

Is distance a proxy for pesticide exposure and is it related to ASD? Some thoughts...

Recently a paper has made some headlines, and the message getting out seems to be that living near a farm field where there has been pesticide applications has been found to increase the risk of Autism spectrum disorder. A few things about the paper. First, one of the things I admire about econometric work is the attempt to make use of some data set, some variable, or some measurement to estimate the effect of some intervention or policy, in a world where we can’t always get our hands on the thing we are really trying to measure. The book Freakonomics comes to mind, or quasi-experimental designs and the use of instrumental variables.

Second, I’m not an epidemiologist, entomologist, or have a background in toxicology,  but my expertise is more focused on statistical methods so I will comment on the article from that perspective. While the authors could not  (or simply did not) actually measure pesticide exposure in any medical or biological sense, they attempted to infer that distance from an agricultural field might correlate well enough to proxy for exposure. That is a large assumption and perhaps one of the greatest challenges of the study. It is not a study on actual exposure. So I’ll  try to only refer to exposure from this point in quotes.  But the authors did make clever use of some interesting data sources. They matched up required reported pesticide applications and report dates with zipcodes of the study respondents and reported pregnancy stages to determine distance from application and at what point of their pregnancy they were exposed.  They reported distance in three bands  or buffer zones of 1.25, 1.5, & 1.75 km   This was actually nice work, if distance could be equated to some known level of exposure. Unfortunately, while they cited some other work attempting to tie exposure to ASD, I did not see a citation in the body of the text where any work had been done justifying the use of distance as a proxy, or those particular bands. More on this later. They also attempted to control for a number of confounders, applied survey weighting to ‘weight up’ the effects to reflect the parent population, and in addition, at least based on my reading, may have even tried to control for some level of selection bias by using IPTW regression with SAS.

Discussion of Results

There were at least four major findings in the paper:

(1) Proximity to organophosphates at some point during gestation was associated with a 60% increased risk for ASD

(2) higher for 3rd trimester exposures [OR = 2.0, 95% confidence interval (CI) = (1.1, 3.6)],

(3) and 2nd trimester chlorpyrifos applications: OR = 3.3 [95% CI = (1.5, 7.4)].

(4)Children of mothers residing near pyrethroid insecticide applications just prior to conception or during 3rd trimester were at greater risk for both ASD and DD, with OR's ranging from 1.7 to 2.3.

So where do we go with these results? First off all of these findings are based on odds ratios. The reported odds ratio in the first finding above was 1.60 which implies a [1.6-1.0]*100 = 60% increase in odds of ASD for ‘exposed’ vs ‘non-exposed’ children. This is an increase in odds, and does not have the exact same interpretation as an increase in probability. (see more about logistic regression and odds ratios here). Some might read the headline and walk away with the wrong idea that living within proximity of farm fields with organophosphate applications constitutes  ‘exposure’ to organophosphates  and is associated with a 60% increased probability of ASD, but that is stacking one large assumption on top of another misinterpretation.

However, these findings are but a slice of the full results reported in the paper. Table 3 reports a number of findings across the distance bands, types of pesticide, and pregnancy stage. One thing about odds ratios, an odds ratio of ‘1’ implies no effect. The vast majority of these findings were associated with odds ratios with 95% confidence intervals containing 1, or very very close to 1. For those that like to interpret p-values, a 95% CI for an odds ratio that contains 1 implies that the estimated regression coefficient in the model has a p-value > .05, i.e. non-significant results.

Another interesting thing about the table, is that there doesn’t seem to be any pattern of distance/pregnancy stage/chemistry associated with the estimated effects or odds ratios. A point made well in a recent blog post regarding this study at scienceblogs.com here.

Sensitivity

From the paper: “In additional analyses, we evaluated the sensitivity of the estimates to the choice of buffer size, using 4 additional sizes between 1 and 2km: results and interpretation remained stable (data not shown).”

That’s unfortunate too. Given the previous discussion of odds ratios, lack of empirical support or literature related to using distance as a proxy for exposure, you would think more sensitivity analysis would be merited to show robustness to all of these assumptions even if and especially if there is no previous precedent in the literature related to distance.  This in combination with the previous discussion regarding the large number of insignificant odds ratios and select reporting of the marginally significant results is probably what fueled accusations of data drudging.

Omitted Controls

From the Paper: “Primarily, our exposure estimation approach does not encompass all potential sources of exposure to each of these compounds: among them external non-agricultural sources (e.g. institutional use, such as around schools); residential indoor use; professional pesticide application in or around the home for gardening, landscaping or other pest control; as well as dietary sources (Morgan 2012).”

So, there are a number of important routes of exposure that were not controlled for, or perhaps a good deal of omitted variable bias and unobserved heterogeneity.  The point of my post is not to pick apart a study linking pesticides to ASD. There are no perfect data sets and no perfect experimental designs. All studies have weaknesses, and my interpretation of this study certainly has flaws. The point is, while this study has made some headlines with some media outlets, and seems scary; it is not one that should be used to draw sharp conclusions or to run to your legislator for new regulations.

This reminds me of a quote I have shared here recently:

"Social scientists and policymakers alike seem driven to draw sharp conclusions, even when these can be generated only by imposing much stronger assumptions than can be defended. We need to develop a greater tolerance for ambiguity. We must face up to the fact that we cannot answer all of the questions that we ask." (Manski, 1995)

References:

Manski, C.F. 1995. Identification Problems in the Social Sciences. Cambridge: Harvard University Press.

Neurodevelopmental Disorders and Prenatal Residential Proximity to Agricultural Pesticides: The CHARGE Study

Janie F. Shelton, Estella M. Geraghty, Daniel J. Tancredi, Lora D. Delwiche, Rebecca J. Schmidt, Beate Ritz, Robin L. Hansen, and Irva Hertz-Picciotto

Environmental Health Perspectives.   June 23, 2014

AllAnalytics - Michael Steinhart - Doctors: Time to Unleash Medical Big Data

Examples:  "Correlating grocery shopping patterns with incidence of obesity and diabetes

Measuring response rates to cholesterol-lowering drugs by correlating pharmacy refills with exercise data from wearable sensors

Correlating physical distance to hospitals and pharmacies with utilization of healthcare services

Analyzing the influence of social network connections on lifestyle choices and treatment compliance."

http://www.allanalytics.com/author.asp?section_id=3314&doc_id=273502&f_src=allanalytics_sitedefault 

Quantile Regression and Healthcare Costs

I thought this was a nice statement that speaks to the utility of quantile regression (which holds to any distribution with these issues not just cost data):

The quantile regression framework allows us to obtain a more complete picture of the effects of the covariates on the health care cost, and is naturally adapted to the skewness and heterogeneity of the cost data.

More:

Health care cost data are characterized by a high level of skewness and heteroscedastic variances…Most of the existing literature on health care cost data analysis have been focused on modeling the conditional mean (or average) of the health care cost given the covariates such as age, gender, race, marital status and disease status. The conditional mean framework has two important limitations. First, the application of the conditional mean regression model to health care cost data analysis is usually not straightforward. Due to the presence of skewness and nonconstant variances, transformation of the response variable is often required when constructing the mean regression model and retransformation is needed in order to obtain direct inference on the mean cost. Second, the conditional mean model focuses primarily on the marginal effects of the risk factors on the central tendency of the conditional distribution. When the marginal effects vary across the conditional distribution, focusing on the marginal effects at the central tendency may substantially distort the information of interest at the tails. For example, a weak relationship between a risk factor and the mean health care cost does not preclude a stronger relationship at the upper or lower quantiles of the conditional distribution….By considering different quantiles, we are able to obtain a more complete picture of the effects of the covariates on health care cost.

From:

Weighted Quantile Regression for Analyzing Health Care Cost Data with Missing Covariates. Ben Sherwood, Lan Wang and Xiao-Hua Zhou Statistics in Medicine. 2012

 “heavy upper tails may influence the "robustness" with which some parameters are estimated. Indeed, in worlds described by heavy-tailed Pareto or Burr- Singh-Maddala distributions (Mandelbrot, 1963; Singh and Maddala, 1976) some traditionally interesting parameters (means, variances) may not even be finite, a situation never encountered in, e.g., a normal or log-normal world. Such concerns should translate into empirical strategies that target the high-end parameters of particular interest, e.g. models for Prob(y ≥ k | x) or quantile regression models.."

 

From:

ECONOMETRIC MODELING OF HEALTH CARE COSTS AND EXPENDITURES:

A SURVEY OF ANALTICAL ISSUES AND RELATED POLICY CONSIDERATIONS

John Mullahy

Univ. of Wisconsin-Madison

January 2009

See also: Quantile Regression with Count Data

Quantile Regression with Count Data

I stumbled upon this paper recently:

Reforming health care: Evidence from quantile regressions for counts
Rainer Winkelmann
Journal of Health Economics 25 (2006) 131–145

"Basically, the approach transforms the discrete data problem into a continuous data problem by adding a random uniform variable to each count. The quantile regression functions of the transformed variable can then be estimated using standard quantile regression software. To interpret the results, one can compare the freely estimated quantile functions to those implied by the respective Poisson or negative binomial estimates in order to detect excess sensitivity in specific parts of the distribution, such as the lower or upper tails."

See also: 
Machado, J.A.F. and Santos Silva, J.M.C. (2005), Quantiles for Counts, Journal of the American Statistical Association, vol. 100, no. 472, pp. 1226-1237.

R:
http://www.inside-r.org/packages/cran/lqmm/docs/lqm.counts

STATA:
http://ideas.repec.org/c/boc/bocode/s456714.html

Propensity Score Matching Meets Survival Analysis

In one of my earlier posts regarding propensity score applications in higher ed research, a reader asked in the comment section about using propensity score methods in the context of survival analysis. Ironically, just a few days prior, I was having a similar discussion with another higher education researcher. Unfortunately, I have not been able to answer their questions adequately, but I think this is an interesting topic. I've recently located a few articles that deal with this. Unfortunately I have not had a chance to read through them but thought they may be of interest. In the least I now have them bookmarked for future reference. Hopefully they address some of these issues.

Effect of radiation therapy on survival in surgically resected retroperitoneal sarcoma: a propensity score-adjusted SEER analysis

 Ann Oncol (2012)

A. H. Choi1,

J. S. Barnholtz-Sloan2 and

J. A. Kim3*

Propensity score methods were used to perform survival analysis in patients who received radiation matched with patients who underwent surgery alone...Propensity scoring (309 matched pairs) and survival analysis using Kaplan–Meier methods demonstrated no difference between propensity score-matched patients receiving radiation therapy and those who did not (P = 0.35).

 Propensity score applied to survival data analysis through proportional hazards models: a Monte Carlo study.

Pharm Stat. 2012 Mar 12. doi: 10.1002/pst.537.

Gayat E, Resche-Rigon M, Mary JY, Porcher R.

A Monte Carlo simulation study was used to compare the performance of several survival models to estimate both marginal and conditional treatment effects. The impact of accounting or not for pairing when analysing propensity-score-matched survival data was assessed. In addition, the influence of unmeasured confounders was investigated....Our study showed that propensity scores applied to survival data can lead to unbiased estimation of both marginal and conditional treatment effect, when marginal and adjusted Cox models are used. In all cases, it is necessary to account for pairing when analysing propensity-score-matched data, using a robust estimator of the variance.

The performance of different propensity score methods for estimating marginal hazard ratios. Stat Med. 2013 Jul 20;32(16):2837-49. doi: 10.1002/sim.5705. Epub 2012 Dec 12. Austin PC.

...in biomedical research, time-to-event outcomes occur frequently. There is a paucity of research into the performance of different propensity score methods for estimating the effect of treatment on time-to-event outcomes....We conducted an extensive series of Monte Carlo simulations to examine the performance of propensity score matching (1:1 greedy nearest-neighbor matching within propensity score calipers), stratification on the propensity score, inverse probability of treatment weighting (IPTW) using the propensity score, and covariate adjustment using the propensity score to estimate marginal hazard ratios. We found that both propensity score matching and IPTW using the propensity score allow for the estimation of marginal hazard ratios with minimal bias. Of these two approaches, IPTW using the propensity score resulted in estimates with lower mean squared error when estimating the effect of treatment in the treated. Stratification on the propensity score and covariate adjustment using the propensity score result in biased estimation of both marginal and conditional hazard ratios. Applied researchers are encouraged to use propensity score matching and IPTW using the propensity score when estimating the relative effect of treatment on time-to-event outcomes.