A review of solutions for diagnostic accuracy studies with an imperfect or missing reference standard.
OBJECTIVE
In diagnostic accuracy studies, the reference standard may be imperfect or not available in all patients. We systematically reviewed the proposed solutions for these situations and generated methodological guidance.


METHODS
Review of methodological articles.


RESULTS
We categorized the solutions into four main groups. The first group includes methods that impute or adjust for missing data on the reference standard. The second group consists of methods that correct estimates of accuracy obtained with an imperfect reference standard. In the third group a reference standard is constructed by combining multiple test results through a predefined rule, based on a consensus procedure, or through statistical modeling. In the fourth group, the diagnostic accuracy paradigm is abandoned in favor of validation studies that relate index test results to relevant clinical data, such as history, future clinical events, and response to therapy.


CONCLUSIONS
Most of the methods try to impute, adjust, or construct a reference standard. In situations that deviate only marginally from the classical diagnostic accuracy paradigm, these are valuable methods. In cases where an acceptable reference standard does not exist, the concept of clinical test validation may provide an alternative paradigm to evaluate a diagnostic test.
Reitsma
Johannes B
JB
Department of Clinical Epidemiology, Biostatistics and Bioinformatics, Academic Medical Center, University of Amsterdam, Amsterdam, The Netherlands. j.reitsma@amc.uva.nl
Rutjes
Anne W S
AW
Khan
Khalid S
KS
Coomarasamy
Arri
A
Bossuyt
Patrick M
PM
D003627:Data Interpretation, Statistical; D003955:Diagnostic Tests, Routine / Q000592:standards*; D006801:Humans; D012015:Reference Standards; D015203:Reproducibility of Results; D012985:Software Design