Use of design effects and sample weights in complex health survey data: a review of published articles using data from 3 commonly used adolescent health surveys.
OBJECTIVE
We assessed how frequently researchers reported the use of statistical techniques that take into account the complex sampling structure of survey data and sample weights in published peer-reviewed articles using data from 3 commonly used adolescent health surveys.


METHODS
We performed a systematic review of 1003 published empirical research articles from 1995 to 2010 that used data from the National Longitudinal Study of Adolescent Health (n=765), Monitoring the Future (n=146), or Youth Risk Behavior Surveillance System (n=92) indexed in ERIC, PsycINFO, PubMed, and Web of Science.


RESULTS
Across the data sources, 60% of articles reported accounting for design effects and 61% reported using sample weights. However, the frequency and clarity of reporting varied across databases, publication year, author affiliation with the data, and journal.


CONCLUSIONS
Given the statistical bias that occurs when design effects of complex data are not incorporated or sample weights are omitted, this study calls for improvement in the dissemination of research findings based on complex sample data. Authors, editors, and reviewers need to work together to improve the transparency of published findings using complex sample data.
Educational Psychology, Research, and Foundations Program, University of South Carolina, Columbia SC 29208, USA. babell@sc.edu
Bethany A
BA
Bell
Anthony J
AJ
Onwuegbuzie
John M
JM
Ferron
Qun G
QG
Jiao
Susan T
ST
Hibbard
Jeffrey D
JD
Kromrey
D000293:Adolescent; D015982:Bias; D003627:Data Interpretation, Statistical; D006306:Health Surveys / Q000379:methods*; D006801:Humans; D010506:Periodicals as Topic; D015983:Selection Bias