title: A review of solutions for diagnostic accuracy studies with an imperfect or missing reference standard.
abstract: OBJECTIVE
  In diagnostic accuracy studies, the reference standard may be imperfect or not available in all patients. We systematically reviewed the proposed solutions for these situations and generated methodological guidance.
  
  
  METHODS
  Review of methodological articles.
  
  
  RESULTS
  We categorized the solutions into four main groups. The first group includes methods that impute or adjust for missing data on the reference standard. The second group consists of methods that correct estimates of accuracy obtained with an imperfect reference standard. In the third group a reference standard is constructed by combining multiple test results through a predefined rule, based on a consensus procedure, or through statistical modeling. In the fourth group, the diagnostic accuracy paradigm is abandoned in favor of validation studies that relate index test results to relevant clinical data, such as history, future clinical events, and response to therapy.
  
  
  CONCLUSIONS
  Most of the methods try to impute, adjust, or construct a reference standard. In situations that deviate only marginally from the classical diagnostic accuracy paradigm, these are valuable methods. In cases where an acceptable reference standard does not exist, the concept of clinical test validation may provide an alternative paradigm to evaluate a diagnostic test.
authors.affiliation: Department of Clinical Epidemiology, Biostatistics and Bioinformatics, Academic Medical Center, University of Amsterdam, Amsterdam, The Netherlands. j.reitsma@amc.uva.nl
authors.forename: Johannes B
authors.initials: JB
authors.lastname: Reitsma
authors.forename: Anne W S
authors.initials: AW
authors.lastname: Rutjes
authors.forename: Khalid S
authors.initials: KS
authors.lastname: Khan
authors.forename: Arri
authors.initials: A
authors.lastname: Coomarasamy
authors.forename: Patrick M
authors.initials: PM
authors.lastname: Bossuyt
mesh_terms: D003627:Data Interpretation, Statistical; D003955:Diagnostic Tests, Routine / Q000592:standards*; D006801:Humans; D012015:Reference Standards; D015203:Reproducibility of Results; D012985:Software Design