Hui and Walter's latent-class reference-free approach may be more useful in assessing agreement than diagnostic performance.
OBJECTIVE
Hui and Walter developed a latent class approach to assess the accuracy of a diagnostic procedure when no reference test is available. Our objective was to compare sensitivity and specificity estimates obtained with this reference-free approach and standard approaches, and to examine how and why they differed on a computerized tomography (CT) scan case study.


METHODS
We compared two sets of sensitivity and specificity estimates from four radiologists independently assessing tumoral and lymph node extension of 85 lung cancer patients with preoperative thoracic CT scan, those obtained relative to pathology findings from surgical specimens (reference set), and those derived from Hui and Walter's approach.


RESULTS
The two sets of estimates significantly and markedly differed from each other. From simulations, we found that small-sample bias in Hui and Walter's estimates could be a major factor in explaining this difference. Furthermore, errors in pathology findings could account for part of this difference. Finally, our analyses revealed that the latent classes may differ intrinsically from the reference classes as defined from pathology findings and may have a different interpretation.


CONCLUSIONS
Diagnostic parameters estimated with respect to latent classes may be more useful in providing a complete assessment of interobserver agreement than in assessing diagnostic performance.
Bertrand
Philippe
P
Laboratoire de Biostatistiques, Epidémiologie et Informatique Médicale, Faculté de Médecine de Tours, BP 3223, 37032 Tours Cedex 1, France. bertrand@med.univ-tours.fr
Bénichou
Jacques
J
Grenier
Philippe
P
Chastang
Claude
C
D002289:Carcinoma, Non-Small-Cell Lung / Q000175:diagnosis* / Q000000981:diagnostic imaging / Q000473:pathology; D003951:Diagnostic Errors; D006801:Humans; D008175:Lung Neoplasms / Q000175:diagnosis* / Q000000981:diagnostic imaging / Q000473:pathology; D008207:Lymphatic Metastasis; D015233:Models, Statistical; D009367:Neoplasm Staging; D015588:Observer Variation; D012015:Reference Standards; D015203:Reproducibility of Results; D012680:Sensitivity and Specificity; D014057:Tomography, X-Ray Computed / Q000379:methods