Learning About and From Variation in Program Impacts Using Multisite Trials

70 Pages Posted: 23 May 2015

See all articles by Stephen Raudenbush

Stephen Raudenbush

University of Chicago - Department of Sociology

Howard S. Bloom

MDRC

Date Written: April 15, 2015

Abstract

The present paper, which is intended for a diverse audience of evaluation researchers, applied social scientists, and research funders, provides a broad overview of the conceptual and statistical issues involved in using multisite randomized trials to learn about and from variation in program effects across individuals, across policy-relevant and theoretically relevant subgroups of individuals, and across program sites. Learning about variation in program effects involves detecting and quantifying this variation. Learning from variation in program effects involves studying the factors which predict or explain it. The paper is divided into four sections, plus a brief final discussion. The first section introduces the concepts and issues involved. Section 2 focuses on detecting and quantifying variation in effects of program assignment, which are often referred to as effects of intent to treat (ITT). Section 3 extends the discussion to variation in effects of program participation, which are often referred to as a complier average causal effect (CACE) or a local average treatment effect (LATE). Section 4 considers moderators of program effects (individual-level or site-level factors that influence the sign and magnitude of these effects) and mediators of program effects (individual-level or site-level “mechanisms” by which these effects are produced).

Keywords: random assignment, multisite trials, variation in effects

JEL Classification: C40, I21

Suggested Citation

Raudenbush, Stephen and Bloom, Howard S., Learning About and From Variation in Program Impacts Using Multisite Trials (April 15, 2015). Available at SSRN: https://ssrn.com/abstract=2609419 or http://dx.doi.org/10.2139/ssrn.2609419

Stephen Raudenbush

University of Chicago - Department of Sociology ( email )

1126 East 59th Street
Chicago, IL 60637
United States

Howard S. Bloom (Contact Author)

MDRC ( email )

200 Vesey Street
New York, NY 10281
United States

Do you have negative results from your research you’d like to share?

Paper statistics

Downloads
76
Abstract Views
680
Rank
567,594
PlumX Metrics