title: Quantifying bias in a health survey: modeling total survey error in the national immunization survey.
abstract: Random-digit-dial telephone surveys are experiencing both declining response rates and increasing under-coverage due to the prevalence of households that substitute a wireless telephone for their residential landline telephone. These changes increase the potential for bias in survey estimates and heighten the need for survey researchers to evaluate the sources and magnitudes of potential bias. We apply a Monte Carlo simulation-based approach to assess bias in the NIS, a land-line telephone survey of 19-35 month-old children used to obtain national vaccination coverage estimates. We develop a model describing the survey stages at which component nonsampling error may be introduced due to nonresponse and under-coverage. We use that model and components of error estimated in special studies to quantify the extent to which noncoverage and nonresponse may bias the vaccination coverage estimates obtained from the NIS and present a distribution of the total survey error. Results indicated that the total error followed a normal distribution with mean of 1.72 per cent(95 per cent CI: 1.71, 1.74 per cent) and final adjusted survey weights corrected for this error. Although small, the largest contributor to error in terms of magnitude was nonresponse of immunization providers. The total error was most sensitive to declines in coverage due to cell phone only households. These results indicate that, while response rates and coverage may be declining, total survey error is quite small. Since response rates have historically been used to proxy for total survey error, the finding that these rates do not accurately reflect bias is important for evaluation of survey data. Published in 2011 by John Wiley & Sons, Ltd.
authors.affiliation: National Center for Immunization and Respiratory Diseases, Centers for Disease Control and Prevention, Atlanta, GA 30333, USA. nmolinari@cdc.gov
authors.forename: NoelleAngelique M
authors.initials: NM
authors.lastname: Molinari
authors.forename: Kirk M
authors.initials: KM
authors.lastname: Wolter
authors.forename: Benjamin
authors.initials: B
authors.lastname: Skalland
authors.forename: Robert
authors.initials: R
authors.lastname: Montgomery
authors.forename: Meena
authors.initials: M
authors.lastname: Khare
authors.forename: Philip J
authors.initials: PJ
authors.lastname: Smith
authors.forename: Martin L
authors.initials: ML
authors.lastname: Barron
authors.forename: Kennon
authors.initials: K
authors.lastname: Copeland
authors.forename: Kathleen
authors.initials: K
authors.lastname: Santos
authors.forename: James A
authors.initials: JA
authors.lastname: Singleton
mesh_terms: D000465:Algorithms; D015982:Bias; D040421:Cell Phone / Q000706:statistics & numerical data; D002675:Child, Preschool; D003198:Computer Simulation; D003625:Data Collection / Q000706:statistics & numerical data; D006282:Health Personnel / Q000706:statistics & numerical data; D006306:Health Surveys / Q000379:methods* / Q000706:statistics & numerical data*; D006801:Humans; D017589:Immunization Programs; D007223:Infant; D007407:Interviews as Topic; D015233:Models, Statistical; D009010:Monte Carlo Method; D016011:Normal Distribution; D033282:Parental Consent / Q000706:statistics & numerical data; D014481:United States; D014611:Vaccination / Q000706:statistics & numerical data*