title: Sources of variation and bias in studies of diagnostic accuracy: a systematic review.
abstract: BACKGROUND
  Studies of diagnostic accuracy are subject to different sources of bias and variation than studies that evaluate the effectiveness of an intervention. Little is known about the effects of these sources of bias and variation.
  
  
  OBJECTIVE
  To summarize the evidence on factors that can lead to bias or variation in the results of diagnostic accuracy studies.
  
  
  METHODS
  MEDLINE, EMBASE, and BIOSIS, and the methodologic databases of the Centre for Reviews and Dissemination and the Cochrane Collaboration. Methodologic experts in diagnostic tests were contacted.
  
  
  METHODS
  Studies that investigated the effects of bias and variation on measures of test performance were eligible for inclusion, which was assessed by one reviewer and checked by a second reviewer. Discrepancies were resolved through discussion.
  
  
  METHODS
  Data extraction was conducted by one reviewer and checked by a second reviewer.
  
  
  RESULTS
  The best-documented effects of bias and variation were found for demographic features, disease prevalence and severity, partial verification bias, clinical review bias, and observer and instrument variation. For other sources, such as distorted selection of participants, absent or inappropriate reference standard, differential verification bias, and review bias, the amount of evidence was limited. Evidence was lacking for other features, including incorporation bias, treatment paradox, arbitrary choice of threshold value, and dropouts.
  
  
  CONCLUSIONS
  Many issues in the design and conduct of diagnostic accuracy studies can lead to bias or variation; however, the empirical evidence about the size and effect of these issues is limited.
authors.affiliation: University of York, York, United Kingdom. pfw2@york.ac.uk
authors.forename: Penny
authors.initials: P
authors.lastname: Whiting
authors.forename: Anne W S
authors.initials: AW
authors.lastname: Rutjes
authors.forename: Johannes B
authors.initials: JB
authors.lastname: Reitsma
authors.forename: Afina S
authors.initials: AS
authors.lastname: Glas
authors.forename: Patrick M M
authors.initials: PM
authors.lastname: Bossuyt
authors.forename: Jos
authors.initials: J
authors.lastname: Kleijnen
mesh_terms: D015982:Bias; D035843:Biomedical Research / Q000379:methods / Q000592:standards*; D019937:Diagnostic Techniques and Procedures / Q000592:standards*; D005260:Female; D006801:Humans; D008297:Male; D012107:Research Design / Q000592:standards