Professional Science Master's Degree Programs in Biotechnology and Management

As an undergraduate I always had an interest in biotechnology and molecular genetics. However, lab work did not particularly appeal to me. I also recognized early on that science does not occur in a vacuum- its subject to social, political, economic, and financial forces. This drew me to the field of economics, specifically public choice theory.

When it came time for graduate school I was still torn. I really wasn't interested in an MBA and didn't really have the background to work in a lab or do field work in genetic research. I really liked economics. The combination of mathematically precise theories (microeconomics/game theory) and empirically sound methods (econometrics) provided a powerful framework for applied problem solving.

I had two advisers make recommendations that got me thinking outside the box. One suggested ultimately I would find a niche that combined both economics and genetics. The other suggested I look at programs like the Bioscience Management program that was being offered at the time at George Mason University (now Bioinformatics Management). While there were not a lot of programs like that being offered at the time, the Agriculture Department at Western Kentucky University provided enough flexibility in their masters program to include courses in biostatistics, genetics, and applied economics. I was able to work on research projects analyzing consumer perceptions of biotechnology and biotech trait resistance management using tools from econometrics, game theory, and population genetics.  Additionally I took courses in applied economics and finance from both the Department of Agriculture and College of Business where I was exposed to tools related to investment analysis, options pricing, and analysis and valuation of biotech companies as well as the impacts of technological change and biotechnology on food and economic development.

With this combination of quantitative training and applied work I have been able to leverage SAS, R, and Python to solve a number of challenging problems throughout a number of professional analytics and consulting roles. 

Today there are a larger number of professional science masters programs with curriculums similar to the programs I contemplated over 10 years ago. 

According to National Professional Science Master’s Association:

"Professional Science Master's (PSMs) are designed for students who are seeking a graduate degree in science or mathematics and understand the need for developing workplace skills valued by top employers. A perfect fit for professionals because it allows you to pursue advanced training and excel in science or math without a Ph.D., while simultaneously developing highly-valued business skills....PSM programs consist of two years of coursework along with a professional component that includes business, communications and/or regulatory affairs."

In 2012 there was an article in Science detailing these degrees and some data related to salaries which seemed attractive. According to the article the first program was officially offered in 1997, reaching 140 programs by 2009 with over 247 at the time of printing.

This commentary from the article corroborates how I feel about my experience:

“There is a tendency for students to buy into the line that if you don't get a Ph.D., you're not a serious professional, that you're wasting your mind,” she says. After spending a decade talking with PSM students and graduates, she is certain that’s not true. “There is so much potential for growth and satisfaction with a PSM degree. You can become a person you didn’t even know you wanted to be.”

Below are some programs that would look interesting to me that students interested in this option should check out.  (there is a program locator you can find here) . Many of these programs are a mash up of biology/biotech and applied economics and business degrees.

George Mason University- PSM Bioinformatics Management

University of Illinois - Agricultural Production

Cornell- MPS Agriculture and Life Sciences

Washington State University - PSM Molecular Biosciences

Middle Tennesee State University - PSM Biotechnology

California State - MS Biotechnology/MBA 

Johns Hopkins - MBA/MS Biotechnology

Rice - PSM Bioscience and Health Policy

North Carolina State University - MBA (Biosciences Mgt Concentration)

Purdue/Kelley - MS-MBA  (not a heavy science emphasis but a very cool degree regardles from great schools)

See also:
Analytical Translators
Why Study Agricultural/Applied Economics

Estimating the Causal Effect of Advising Contacts on Fall to Spring Retention Using Propensity Score Matching and Inverse Probability of Treatment Weighted Regression

Matt Bogard, Western Kentucky University

Abstract

In the fall of 2011 academic advising and residence life staff working for a southeastern university utilized a newly implemented advising software system to identify students based on attrition risk. Advising contacts, appointments, and support services were prioritized based on this new system and information regarding the characteristics of these interactions was captured in an automated format. It was the goal of this study to investigate the impact of this advising initiative on fall to spring retention rates. It is a challenge on college campuses to evaluate interventions that are often independent and decentralized across many university offices and organizations. In this study propensity score methods were utilized to address issues related to selection bias. The findings indicate that advising contacts associated with the utilization of the new software had statistically significant impacts on fall to spring retention for first year students on the order of a 3.26 point improvement over comparable students that were not contacted.

Suggested Citation

Matt Bogard. 2013. "Estimating the Causal Effect of Advising Contacts on Fall to Spring Retention Using Propensity Score Matching and Inverse Probability of Treatment Weighted Regression" The SelectedWorks of Matt Bogard
Available at: http://works.bepress.com/matt_bogard/25

Institutional Research Presentations at SAS Global Forum

I'm not attending #SASGF14, but some of my colleagues in higher ed are. Here is what they are doing. If you are not attending global forum, or can't make their talks, I encourage you to check out their papers via the online proceedings once they are posted.

Tuesday March 25

Paper 1448 - From Providing Support to Driving Decisions: Improving the Value of Institutional Research For almost two decades, Western Kentucky University's Office of Institutional Research (WKU-IR) has used SAS® to help shape the future of the institution by providing faculty and administrators with information they can use to make a difference in the lives of their students. This presentation provides specific examples of how WKU-IR has shaped the policies and practices of our institution and discusses how WKU-IR moved from a support unit to a key strategic partner. In addition, the presentation covers the following topics: How the WKU Office of Institutional Research developed over time; Why WKU abandoned reactive reporting for a more accurate, convenient system using SAS® Enterprise Intelligence Suite for Education; How WKU shifted from investigating what happened to predicting outcomes using SAS® Enterprise Miner™ and SAS® Text Miner; How the office keeps the system relevant and utilized by key decision makers; What the office has accomplished and key plans for the future.


Paper 1638 - Institutional Research: Serving University Deans and Department Heads Administrators at Western Kentucky University rely on the Institutional Research department to perform detailed statistical analyses to deepen the understanding of issues associated with enrollment management, student and faculty performance, and overall program operations. This paper presents several instances of analyses performed for the university to help it identify and recruit suitable candidates, uncover root causes in grade and enrollment trends, evaluate faculty effectiveness, and assess the impact of student characteristics, programs, or student activities on retention and graduation rates. The paper briefly discusses the data infrastructure created and used by Institutional Research. For each analysis performed, it reviews the SAS® program and key components of the SAS code involved. The studies presented include the use of SAS® Enterprise Miner™ to create a retention model incorporating dozens of student background variables. It shows an examination of grade trends in the same courses taught by different faculty and subsequent student behavior and success, providing insights into the nuances and subtleties of evaluating faculty performance. Another analysis uncovers the possible influence of fraternities and sororities in freshmen algebra courses. Two investigations explore the impact of programs on student retention and graduation rates. Each example and its findings illustrate how Institutional Research can support the administration of university operations. The target audience is any SAS professional interested in learning more about Institutional Research in higher education and how SAS software is used by an Institutional Research department to serve its organization.


Monday March 24

Paper 1689 - Simple ODS Tips to Get RWI (Really Wonderful Information) SAS® continues to expand and improve its reporting capability. With new SAS® 9.4 enhancements in ODS (Output Delivery System), the opportunity to create stunning reports has expanded even further. If you are charged with creating relevant, informative, easy-to-read reports for clients or administrators, then the ODS Report Writing Interface, ODS LAYOUT enhancements, and the new ODSTEXT procedure are important tools to use. These tools allow you to create reports in a smart, eye-catching format that can be turned around quite quickly and programmed to provide optimum flexibility. How many times have you worked hours to tweak and fine-tune a report directly in Microsoft Excel, Microsoft Word, Microsoft Power Point or some other similar software only to be asked for a “quick update”, which would then take hours to recreate because you are manually transferring data? Do you ever dread receiving the compliment, “This is really wonderful information!!!!” because you know it will be followed by “Can you run this for EVERY region?” Well, dread no more, because when you harness the power of SAS® ODS, you can create first-rate, flexible, fabulous reports! Join me as I share with you two real-world examples of ODS capabilities using (1) a marketing piece I designed to help the president of our university spotlight county- and region-specific data as he recruited across the state and (2) our academic program review form, a multi-page report that outputs to Word so that program coordinators can add personalized commentary to support their program’s effectiveness.

Analytics vs. Causal Inference

When I think of analytics, I primarily think about what Leo Breiman referred to as an 'algorithmic' approach to analysis. Breiman states:"There are two cultures in the use of statistical modeling to reach conclusions from data.  One "assumes that the data are generated by a given stochastic data model" the other"uses algorithmic models and treats the data mechanism as unknown."

To take an example from higher education, lets look at a hypothetical I have proposed before. Assume we want to evaluate the causal effects of a summer camp designed to prepare high school graduates for their first year of college. If we are interested in making inferences about the causal impact of the camp on retention (this fits under the stochastic data modeling culture) we realize that the impact of the camp program itself (which is the causal effect of interest) as well as academic potential and motivation are confounded. Students that attend the camp may also be very motivated and academically strong, and likely to have high retention rates regardless of their attendance. We proceed with our analysis using some form of experimental or quasi-experimental design to attempt to statistically estimate  or 'identify' the causal effects related to camp. We are concerned with things like standard errors, confidence intervals, statistical significance etc. We may use the results to evaluate the effectiveness of the camp program and improve resource allocation.

On the other hand, we may simply be interested in building a model that gives a measure of the probability of retention for first year students. We may want to take those results and segment the student population into strata based on their risk profile, and tailor programs to help with their academic success an improve resource allocation. The variable indicating 'camp' attendance among others might be a good predictor and aid in producing the probability estimates and well calibrated stratifications. We might accomplish this with logistic regression, decision trees, neural networks, random forests or gradient boosting or some other machine learning algorithm. We are concerned with things like predictive accuracy, discrimination, sensitivity, specificity, true positives, false positives,  ranking, and calibration.  We are not so concerned with p-values, statistical significance, standard errors etc.

Both approaches are data driven, and one is not more meritorious than the other. People often adopt one culture  or paradigm and impugn the other.  As Brieman states:

 "Approaching problems by looking for a data model imposes an apriori straight jacket that restricts the ability of statisticians to deal with a wide range of statistical problems."

In a past Campus Technology article "The Predictive Analytics Framework Moves Forward" the following comments depict this schism that sometimes occurs between cultures:

"For some in education what we’re doing might seem a bit heretical at first--we’ve all been warned in research methods classes that data snooping is bad! But the newer technologies and the sophisticated analyses that those technologies have enabled have helped us to move away from looking askance at pattern recognition. That may take a while in some research circles, but in decision-making circles it’s clear that pattern recognition techniques are making a real difference in terms of the ways numerous enterprises in many industries are approaching their work"

 In fact, both paradigms can often be complimentary in application. We might first develop an algorithm as indicated in the second approach and based on those results design a program or intervention targeted at students with a certain risk profile. We might then step into the world of inferential statistics and attempt to evaluate the effectiveness of this program.

In summary, both are going to utilize some sort of data driven model development, and as the popular parahrase of statistican George E.P. Box goes, all models are wrong, but some are useful. Regardless of the paradigm you are working in, the key is to to produce something useful for solving the problem at hand.

References:

The Predictive Analytics Reporting Framework Moves Forward
A Q & A with WCET Executive Director Ellen Wagner on the PAR Framework.  Mary Grush 01/18/12 Campus Technology. http://campustechnology.com/articles/2012/01/18/the-predictive-analytics-reporting-framework-moves-forward.aspx

 'Statistical Modeling: The Two Cultures' by L. Breiman (Statistical Science
2001, Vol. 16, No. 3, 199–231)

See also: 

Predictive Analytics in Higher Ed http://econometricsense.blogspot.com/2012/01/predictive-analytics-in-higher-ed.html

Culture War: Classical Statistics vs. Machine Learning: http://econometricsense.blogspot.com/2011/01/classical-statistics-vs-machine.html 

Economists as Data Scientists http://econometricsense.blogspot.com/2012/10/economists-as-data-scientists.html 

Kentucky Association of Institutional Research (KAIR) Presentations

I'll be giving the following two talks at KAIR this year.
  
A Guide to Quasi-Experimental Designs

Quasi-experimental designs including propensity score methods, instrumental variables, regression discontinuity, and difference-in-difference estimators offer an inferentially rigorous alternative for program evaluation. In this guide, I begin with an introduction to the potential outcomes framework for rigorously characterizing selection bias and follow with discussions of quasi-experimental methods that may be useful to practitioners involved in program evaluation.

Paper: http://works.bepress.com/matt_bogard/24/
Slides: http://works.bepress.com/matt_bogard/27/

A Data Driven Analytic Strategy for Increasing Yield and Retention

As many Universities face the constraints of declining enrollment demographics, pressure from state governments for increased student success, as well as declining revenues, the costs of utilizing anecdotal evidence and intuition based on ‘gut’ feelings to make time and resource allocation decisions become significant. This presentation describes several data mining algorithms that could be used to develop a model to score university students based on their probability of enrollment and retention early in the enrollment funnel so that staff and administrators can work to recruit students that not only have an average or better chance of enrolling but also succeeding once they enroll. The use of SAS® Enterprise Miner and SAS® EBI server are discussed as an example of a business intelligence platform that can be used for implementing the models and delivering flexible web based reports to end users.

Paper: http://works.bepress.com/matt_bogard/26/
Slides:http://works.bepress.com/matt_bogard/28/

Causal Inference and Quasi-experimental Design Roundup

I’ve dedicated several posts recently to the subject of quasi-experimental designs and causal inference. I’ve tried to organize the following related links for a bigger picture.

First off, regression is often mischaracterized by a clinical view of  assumptions related to linearity. However, as Angrist and Pischke state:

"In fact, the validity of linear regression as an empirical tool does not turn on linearity either...The statement that regression approximates the CEF lines up with our view of empirical work as an effort to describe the essential features of statistical relationships, without necessarily trying to pin them down exactly."  - Mostly Harmless Econometrics, p. 26 & 29

For more discussion see:

Linear Regression as an Empirical Tool

In Mostly Harmless Econometrics, not only is linear regression rigorously developed and discussed, but quasi-experimental designs are given very heavy emphasis. As discussed in Cellini (2008):

… proxy variable, fixed effects, and difference in- differences approaches are becoming quite common. Indeed, these approaches have replaced basic multivariate regression as the new standard for education research in the economics literature”

The links below attempt to highlight at least in a heuristic sense, these methods and the issues they attempt to address:

Selection Bias and the Rubin Causal Model and Potential Outcomes Framework

Unobserved Heterogeneity and Endogeneity

Fixed Effects

A Toy Instrumental Variables Application

Propensity Score Matching

IPTW Regression

Regression Discontinuity Designs

Difference-in-Difference Estimators

Time Series Methods:

Interrupted Time Series (Intervention Analysis)

Intervention Analysis with SAS and R

Granger Causality

Quasi-experimental Designs in Financial Aid Research

 The following are some very good articles related to regression discontinuity, difference-in- difference, and instrumental variable applications in assessing the impact of financial aid on student college enrollment decisions:

van der Klaauw (2002). Estimating the effect of financial aid offers on college enrollment: A regression-discontinuity approach. International Economic Review.  43(4), 1249–1287.

 “An important problem faced by colleges and universities, that of evaluating the effect of their financial aid offers on student enrollment decisions, is complicated by the likely endogeneity of the aid offer variable in a student enrollment equation. This article shows how discontinuities in an East Coast college’s aid assignment rule can be exploited to obtain credible estimates of the aid effect without having to rely on arbitrary exclusion restrictions and functional form assumptions. Semiparametric estimates based on a regression–discontinuity (RD) approach affirm the importance of financial aid as an effective instrument in competing with other colleges for students.”

 Stephanie Riegg Cellini. Causal Inference and Omitted Variable Bias in Financial Aid Research: Assessing Solutions The Review of Higher Education Spring 2008, Volume 31, No. 3, pp. 329–354

 Discusses the shortfall of multivariable regression estimates when used to estimate treatment effects in the face of omitted variable and selection bias in the case of analyzing treatment effects of financial aid offers on enrollment. Introduces quasi-experimental methods such as instrumental variables, difference in difference, and regression discontinuity approaches. This paper acknowledges that these methods are the new standard (over multivariable regression) in program evaluation in higher ed research and advocates the use of multiple methods to assess the robustness of results in the face of these challenges.

  Godman, J. Who merits financial aid?: Massachusetts' Adams Scholarship. Journal of Public Economics. Volume 92, Issues 10–11, October 2008, Pages 2121–2131

  This paper discusses the potential biases involved in simply comparing scholarship recipients to losers. Consistent with Cellini (2008) it uses multiple methods to addresses these issues including difference-in-difference and regression discontinuity estimates.  The DD estimates identify treatment effects by comparing variation across time between groups that would have received the scholarship in the past had it been implemented to winners in the present. The RD methods (similar to van der Klaauw (2002)) by contrast utilize a single contemporary cohort based on the identifying assumption that treatment assignment near the cutoff is as good as random, providing an estimate of an average treatment effect of scholarships.

References: Social Network Analysis and Student Integration and Persistence

See also: An Introduction to SNA using R and NetDraw, SNA & Predictive Modeling, Using Twitter to Demonstrate Basic Concepts from SNA,  An Introduction to SNA with Applications

Social Psychology of Education

June 2012, Volume 15, Issue 2, pp 165-180

A social network analysis of student retention using archival data

James E. Eckles,

Eric G. Stradley 

This study attempts to determine if a relationship exists between first-to-second-year retention and social network variables for a cohort of first-year students at a small liberal arts college. The social network is reconstructed using not survey data as is most common, but rather using archival data from a student information system. Each student is given a retention score and an attrition score based on the behavior of their immediate relationships in the network. Those scores are then entered into a logistic regression that includes tradition background and performance variables that are traditionally significantly related to retention. Students' friends' retention and attrition behaviors are found to have a greater impact on retention that any background or performance variable.

JOURNAL OF COLLEGE STUDENT RETENTION, Vol. 4(1) 39-52, 2002-2003

THE ROLE OFSOCIAL SUPPORT NETWORK

IN COLLEGE PERSISTENCE AMONG

FRESHMAN STUDENTS

MICHAEL P. SKAHILL

University of San Francisco, California

 Used social network analysis to examine the role of social support networks in student persistence among residential and commuter students. Found that commuter students are less likely to persist, while residential students who reported making greater numbers of new friends with connections to the school also reported attaining personal and academic goals at a significantly greater rate.

The Journal of Higher Education   

Vol. 71, No. 5, Sep. - Oct., 2.

Ties That Bind: A Social Network Approach to Understanding Student Integration and Persistence

Scott L. Thomas

This study examined the social networks of college students and how such networks affect student commitment and persistence. The study's theoretical framework was based on application of the social network paradigm to Tinto's Student Integration Model, in which a student's initial commitment is modified over time as a result of the student's integration into the campus community. Freshmen enrolled for the spring 1993 semester responded (322 of 379) to the First-Year Experiences Survey, which involved identifying students with whom they frequently spoke and the dimensions on which they related to these students. Results were compared with enrollment data for the fall 1993 semester to identify students returning for their sophomore year. The largest effect on persistence was associated with the number of nominations received from other students, and this factor operated indirectly through enhanced social integration, institutional commitment, and intention. Overall, students with broader, well-connected networks were more likely to persist, whereas students with a higher proportion of ties falling within their social peer group were less likely to persist.

Physics Education Research Conference 2009

Part of the PER Conference series

Ann Arbor, Michigan: July 29-30, 2009

Volume 1179, Pages 105-108

Investigating Student Communities with Network Analysis of Interactions in a Physics Learning Center written by Eric Brewe, Laird H. Kramer, and George O'Brien 

 Developing a sense of community among students is one of the three pillars of an overall reform effort to increase participation in physics, and the sciences more broadly, at Florida International University. The emergence of a research and learning community, embedded within a course reform effort, has contributed to increased recruitment and retention of physics majors. Finn and Rock [1] link the academic and social integration of students to increased rates of retention. We utilize social network analysis to quantify interactions in Florida International University's Physics Learning Center (PLC) that support the development of academic and social integration,. The tools of social network analysis allow us to visualize and quantify student interactions, and characterize the roles of students within a social network. After providing a brief introduction to social network analysis, we use sequential multiple regression modeling to evaluate factors which contribute to participation in the learning community. Results of the sequential multiple regression indicate that the PLC learning community is an equitable environment as we find that gender and ethnicity are not significant predictors of participation in the PLC. We find that providing students space for collaboration provides a vital element in the formation of supportive learning community.

Regression Discontinuity Designs in Higher Ed Research

 Previously I posted a brief introduction to RD designs. Here are some applications in Higher Ed research in the areas of developmental education and financial aid:

van der Klaauw (2002). Estimating the effect of financial aid offers on college enrollment: A regression-discontinuity approach. International Economic Review.  43(4), 1249–1287.

An important problem faced by colleges and universities, that of evaluating the effect of their financial aid offers on student enrollment decisions, is complicated by the likely endogeneity of the aid offer variable in a student enrollment equation. This article shows how discontinuities in an East Coast college’s aid assignment rule can be exploited to obtain credible estimates of the aid effect without having to rely on arbitrary exclusion restrictions and functional form assumptions. Semiparametric estimates based on a regression–discontinuity (RD) approach affirm the importance of financial aid as an effective instrument in competing with other colleges for students.

Brian G. Moss  and William H. Yeaton 
Shaping Policies Related to Developmental Education: An Evaluation Using the Regression-Discontinuity Design. EDUCATIONAL EVALUATION AND POLICY ANALYSIS September 21, 2006 vol. 28 no. 3 215-229

Utilizing the regression-discontinuity research design, this article explores the effectiveness of a developmental English program in a large, multicampus community college. Routinely collected data
were extracted from existing records of a cohort of first-time college students followed for approximately 6 years (N = 1,473). Results are consistent with a conclusion that students’ participation in the program increases English academic achievement to levels similar to those of students not needing developmental coursework. The findings are also consistent with a conclusion that those students in greatest need of developmental English benefit the most from the program. This study provides an inexpensive, inferentially rigorous, program evaluation strategy that can be applied with few additional efforts to assess existing programs and to guide policy decisions.

Lasik, S.A. (2008). Evaluating developmental education programs in higher education. ASHE/Lumina Policy Brief Issue 4

A key benefi t of the regression-discontinuity design is that it effectively assesses the extent that developmental programs result in improving student retention and academic success.

Colcagno, J. C. and Long, B.T. (2008).  The Impact of Postsecondary Remediation Using a Regression Discontinuity Approach: Addressing Endogenous Sorting and Noncompliance. NCPR Working Paper.

Remedial or developmental courses are the most common policy instruments used to assist underprepared postsecondary students who are not ready for college-level coursework. However, despite its important role in higher education and its substantial costs, there is little rigorous evidence on the effectiveness of college remediation on the outcomes of students. This study uses a detailed dataset to identify the causal effect of remediation on the educational outcomes of nearly 100,000 college students in Florida, an important state that reflects broader national trends in remediation policy and student diversity. Moreover, using a Regression Discontinuity design, we discuss concerns about endogenous sorting around the policy cutoff, which poses a threat to the assumptions of the model in multiple research contexts. To address this concern, we implement methods proposed by McCrary (2008) and discuss the strengths of this approach. The results suggest math and reading remedial courses have mixed benefits. Being assigned to remediation appears to increase persistence to the second year and the total number of credits completed for students on the margin of passing out of the requirement, but it does not increase the completion of college-level credits or eventual degree completion. Taken together, the results suggest that remediation might promote early persistence in college, but it does not necessarily help students on the margin of passing the placement cutoff make long-term progress toward earning a degree.

Actually there is a nice bibliography of research related to RD designs provided by Keith Smolkowski here: 

http://homes.ori.org/~keiths/bibliography/statistics-regdisc.html

I’ve pulled out most of the references related to higher education research and education in general:

Jacob, B. A., & Lefgren, L. (2004). Remedial education and student achievement: A regression-discontinuity analysis. Review of Economics and Statistics, 86(1), 226 -244.

Seaver, W. B., & Quarton, R. J. (1976). Regression-discontinuity analysis of Dean's List effects. Journal of Educational Psychology, 68(4), 459-465.

Wong, V.C., Cook, T. D., Barnett, W.S., & Jung, K. (2008). An effectiveness-based evaluation of five state pre-k programs. Journal of Policy Analysis and Management, 27(1), 122-154

Bloom, H. S. (2012). Modern regression discontinuity analysis. Journal of Research on Educational Effectiveness, 5(1), 43-82.

Ludwig, J., & Miller, D. L. (in press). Does Head Start improve children's life chances: Evidence from a regression discontinuity approach. Quarterly Journal of Economics, 122(1), 159-208

Gorard, S., & Cook, T. D. (2007). Where does good evidence come from? International Journal of Research and Method in Education, 30(3), 307-323.

Lesik, S. (2006). Applying the regression-discontinuity design to infer causality with non-random assignment. Review Of Higher Education, 30(1), 1-19.

Marsh, H. W. (1998). Simulation study of nonequivalent group-matching and regression-discontinuity designs: Evaluations of gifted and talented programs. Journal of Experimental Education, 66(2), 163-192.

Reardon, S. F., & Robinson, J. P. (2012). Regression discontinuity designs with multiple rating-score variables. Journal of Research on Educational Effectiveness, 5(1), 83-104

 

Schochet, P. Z. (2008). Technical methods report: Statistical power for regression discontinuity designs in education evaluations (NCEE 2008-4026). Washington, DC: National Center for Education Evaluation and Regional Assistance, Institute of Education Sciences, U.S. Department of Education. Retrieved from the National Center

for Education Evaluation: http://ncee.ed.gov 

Schochet, P. Z. (2009). Statistical power for regression discontinuity designs in education evaluations. Journal Of Educational And Behavioral Statistics, 34(2), 238-266

Schochet, P., Cook, T., Deke, J., Imbens, G., Lockwood, J.R., Porter, J., Smith, J. (2010). Standards for regression discontinuity designs. Washington DC: U.S. Department of Education, Institute of Education Sciences, What Works Clearinghouse. Retrieved from http://ies.ed.gov/ncee/wwc/pdf/wwc_rd.pdf

Schumacker, R. E., & Mount, R. E. (2007). Regression discontinuity: Examining model misspecification. Paper presented at the 2007 Annual Meeting of the American Educational Research Association, April, 2007 Chicago,Illinois

Thistlethwaite, D. L., & Campbell, D. T. (1960). Regression discontinuity analysis: An alternative to the ex post facto experiment. Journal of Educational Psychology, 51(6), 309-317

Angrist, J. D., & Lavy, V. (1999). Using Maimonides' Rule to estimate the effect of class size on scholastic achievement. Quarterly Journal of Economics, 144(2), 533-576

Gamse, B. C., Jacob, R. T., Horst, M., Boulay, B., & Unlu, F. (2008). Reading First Impact Study Final Report (NCEE 2009-4038). Washington, DC: National Center for Education Evaluation and Regional Assistance Institute of Education Sciences, U.S. Department of Education. Retrieved from http://ies.ed.gov/ncee/pubs/20094038.asp 

Regression Discontinuity Designs

Suppose a policy or intervention is implemented or a treatment is applied based on arbitrary values of some observed covariate value or values X0. If there is some positive relationship between ‘X’ and the outcome ‘Y’ then how do we know if a treatment applied to subjects where X > X0 isn’t biased since subjects with higher values of X are more likely to exhibit higher levels of the outcome variable Y anyway?  Is it valid to make comparisons of observed outcomes (Y) between groups with differing values of (X)?  One solution would be to implement matched comparisons between groups with similar values of covariates.  Regression discontinuity designs allow us to compare differences between groups in the neighborhood of the cutoff value X0 giving us unbiased estimates of treatment effects.

• Treatment effects can be characterized by a change in intercept or main effect at the discontinuity.

•  Treatment assignment is equivalent to random assignment within the neighborhood of the cutoff   (Lee & Lemieux,2010).

• More complicated functional forms may be estimated:

Y = f(x) +ρ D + e where f(x) may be a pth order polynomial

• Comparisons of outcomes in the neighborhood of X0 provide estimates of the treatment effect  ρ that does not depend on an exactly correct specification of the functional form of E[Y|X] (Angrist &Pischke, 2009)

•  Even more complicated methods including local linear regression may be implemented

The above illustrates only one potential visualization of RD designs.  As illustrated below, treatment effects  can be visualized as discontinuities  or changes in either the intercept or slope or both at the cutoff X₀. 

Application:

In Shaping Policies Related to Developmental Education: An Evaluation Using the Regression-Discontinuity Design,  the authors use RD design to assess the impact of developmental education on student success in subsequent level English courses :

They find that ‘students’ participation in the program increases English academic achievement to levels similar to those of students not needing developmental coursework.’ Note in this case, the treatment (developmental course work) is applied where X < X0 = 85, vs. where X > X0 in the cases I presented above. The discontinuity/treatment effect in this case is represented by a change in slope/interaction at the cutoff.

References:

Brian G. Moss  and William H. Yeaton 

Shaping Policies Related to Developmental Education: An Evaluation Using the Regression-Discontinuity Design. EDUCATIONAL EVALUATION AND POLICY ANALYSIS September 21, 2006 vol. 28 no. 3 215-229

Imbens, Guido W. & Lemieux, Thomas, 2008. "Regression discontinuity designs: A guide to practice," Journal of Econometrics, Elsevier, vol. 142(2), pages 615-635, February.

Regression Discontinuity Designs in Economics

David S. Lee and Thomas Lemieux.

 Journal of Economic Literature 48 (June 2010)281-355

REGRESSION DISCONTINUITY

PATRICIA BAUMER

SOUTHERN METHODIST UNIVERS ITY.  http://faculty.smu.edu/kyler/courses/7312/presentations/baumer/Baumer_RD.pdf

Mostly Harmless Econometrics. Angrist & Pischke. 2009.