Interexpert agreement of plus disease diagnosis in retinopathy of prematurity.
OBJECTIVE
To measure agreement of plus disease diagnosis among retinopathy of prematurity (ROP) experts.


METHODS
A set of 34 wide-angle retinal photographs from infants with ROP was compiled on a secure Web site and was interpreted independently by 22 recognized ROP experts. Diagnostic agreement was analyzed using 3-level (plus, pre-plus, or neither) and 2-level (plus or not plus) categorizations.


RESULTS
In the 3-level categorization, all experts agreed on the same diagnosis in 4 of 34 images (12%), and the mean weighted kappa statistic for each expert compared with all others was between 0.21 and 0.40 (fair agreement) for 7 experts (32%) and between 0.41 and 0.60 (moderate agreement) for 15 experts (68%). In the 2-level categorization, all experts who provided a diagnosis agreed in 7 of 34 images (21%), and the mean kappa statistic for each expert compared with all others was between 0 and 0.20 (slight agreement) for 1 expert (5%), between 0.21 and 0.40 (fair agreement) for 3 experts (14%), between 0.41 and 0.60 (moderate agreement) for 12 experts (55%), and between 0.61 and 0.80 (substantial agreement) for 6 experts (27%).


CONCLUSIONS
Interexpert agreement of plus disease diagnosis is imperfect. This may have important implications for clinical ROP management, continued refinement of the international ROP classification system, development of computer-based diagnostic algorithms, and implementation of ROP telemedicine systems.
Chiang
Michael F
MF
Department of Ophthalmology, Columbia University College of Physicians and Surgeons, New York, NY 10032, USA. chiang@dbmi.columbia.edu
Jiang
Lei
L
Gelman
Rony
R
Du
Yunling E
YE
Flynn
John T
JT
D003941:Diagnostic Techniques, Ophthalmological; D004108:Dilatation, Pathologic / Q000175:diagnosis; D006801:Humans; D007091:Image Processing, Computer-Assisted; D007231:Infant, Newborn; D007234:Infant, Premature; D015588:Observer Variation; D010781:Photography; D011361:Professional Competence; D015203:Reproducibility of Results; D012171:Retinal Vessels / Q000473:pathology*; D012178:Retinopathy of Prematurity / Q000175:diagnosis*