NBER

Hunt Allcott

Bibliographic Information

NBER Working Paper No. 18373
Issued in September 2012, Revised in September 2014
NBER Program(s):EEE, LS

Published: Hunt Allcott, 2015. "Site Selection Bias in Program Evaluation," The Quarterly Journal of Economics, vol 130(3), pages 1117-1165.

This paper was revised on September 15, 2014

Available Formats

Abstract

"Site selection bias" can occur when the probability that a program is adopted or evaluated is correlated with its impacts. I test for site selection bias in the context of the Opower energy conservation programs, using 111 randomized control trials involving 8.6 million households across the U.S. Predictions based on rich microdata from the first ten replications substantially overstate efficacy in the next 101 sites. Several mechanisms caused this positive selection. For example, utilities in more environmentalist areas are more likely to adopt the program, and their customers are more responsive to the treatment. Also, because utilities initially target treatment at higher-usage consumer subpopulations, efficacy drops as the program is later expanded. The results illustrate how program evaluations can still give systematically biased out-of-sample predictions, even after many replications.

National Bureau of Economic Research
1050 Massachusetts Ave.
Cambridge, MA 02138
617-868-3900
info@nber.org

Twitter RSS

View Full Site: One timeAlways