title: Nonprobability Web surveys to measure sexual behaviors and attitudes in the general population: a comparison with a probability sample interview survey.
abstract: BACKGROUND
  Nonprobability Web surveys using volunteer panels can provide a relatively cheap and quick alternative to traditional health and epidemiological surveys. However, concerns have been raised about their representativeness.
  
  
  OBJECTIVE
  The aim was to compare results from different Web panels with a population-based probability sample survey (n=8969 aged 18-44 years) that used computer-assisted self-interview (CASI) for sensitive behaviors, the third British National Survey of Sexual Attitudes and Lifestyles (Natsal-3).
  
  
  METHODS
  Natsal-3 questions were included on 4 nonprobability Web panel surveys (n=2000 to 2099), 2 using basic quotas based on age and sex, and 2 using modified quotas based on additional variables related to key estimates. Results for sociodemographic characteristics were compared with external benchmarks and for sexual behaviors and opinions with Natsal-3. Odds ratios (ORs) were used to express differences between the benchmark data and each survey for each variable of interest. A summary measure of survey performance was the average absolute OR across variables. Another summary measure was the number of key estimates for which the survey differed significantly (at the 5% level) from the benchmarks.
  
  
  RESULTS
  For sociodemographic variables, the Web surveys were less representative of the general population than Natsal-3. For example, for men, the average absolute OR for Natsal-3 was 1.14, whereas for the Web surveys the average absolute ORs ranged from 1.86 to 2.30. For all Web surveys, approximately two-thirds of the key estimates of sexual behaviors were different from Natsal-3 and the average absolute ORs ranged from 1.32 to 1.98. Differences were appreciable even for questions asked by CASI in Natsal-3. No single Web survey performed consistently better than any other did. Modified quotas slightly improved results for men, but not for women.
  
  
  CONCLUSIONS
  Consistent with studies from other countries on less sensitive topics, volunteer Web panels provided appreciably biased estimates. The differences seen with Natsal-3 CASI questions, where mode effects may be similar, suggest a selection bias in the Web surveys. The use of more complex quotas may lead to some improvement, but many estimates are still likely to differ. Volunteer Web panels are not recommended if accurate prevalence estimates for the general population are a key objective.
authors.affiliation: London School of Hygiene & Tropical Medicine, London, United Kingdom. bob.erens@lshtm.ac.uk.
authors.forename: Bob
authors.identifier: http://orcid.org/0000-0002-4430-954X
authors.initials: B
authors.lastname: Erens
authors.forename: Sarah
authors.identifier: http://orcid.org/0000-0002-8988-4724
authors.initials: S
authors.lastname: Burkill
authors.forename: Mick P
authors.identifier: http://orcid.org/0000-0001-5006-9346
authors.initials: MP
authors.lastname: Couper
authors.forename: Frederick
authors.identifier: http://orcid.org/0000-0002-1933-3273
authors.initials: F
authors.lastname: Conrad
authors.forename: Soazig
authors.identifier: http://orcid.org/0000-0002-4171-0805
authors.initials: S
authors.lastname: Clifton
authors.forename: Clare
authors.identifier: http://orcid.org/0000-0002-4612-1858
authors.initials: C
authors.lastname: Tanton
authors.forename: Andrew
authors.identifier: http://orcid.org/0000-0002-4686-2293
authors.initials: A
authors.lastname: Phelps
authors.forename: Jessica
authors.identifier: http://orcid.org/0000-0003-4300-0724
authors.initials: J
authors.lastname: Datta
authors.forename: Catherine H
authors.identifier: http://orcid.org/0000-0002-4220-5034
authors.initials: CH
authors.lastname: Mercer
authors.forename: Pam
authors.identifier: http://orcid.org/0000-0002-1067-1583
authors.initials: P
authors.lastname: Sonnenberg
authors.forename: Philip
authors.identifier: http://orcid.org/0000-0002-4742-9686
authors.initials: P
authors.lastname: Prah
authors.forename: Kirstin R
authors.identifier: http://orcid.org/0000-0002-4409-6601
authors.initials: KR
authors.lastname: Mitchell
authors.forename: Kaye
authors.identifier: http://orcid.org/0000-0003-1053-8640
authors.initials: K
authors.lastname: Wellings
authors.forename: Anne M
authors.identifier: http://orcid.org/0000-0003-1330-7100
authors.initials: AM
authors.lastname: Johnson
authors.forename: Andrew J
authors.identifier: http://orcid.org/0000-0001-8968-5963
authors.initials: AJ
authors.lastname: Copas
mesh_terms: D000293:Adolescent; D000328:Adult; D001290:Attitude; D005260:Female; D006306:Health Surveys / Q000379:methods*; D006801:Humans; D020407:Internet; D008019:Life Style; D008297:Male; D008875:Middle Aged; D015995:Prevalence; D012494:Sampling Studies; D012725:Sexual Behavior; D055815:Young Adult
keywords: Internet survey; Web survey; sampling bias; selection bias; sexual behavior; survey methods