title: Interexpert agreement of plus disease diagnosis in retinopathy of prematurity.
abstract: OBJECTIVE
  To measure agreement of plus disease diagnosis among retinopathy of prematurity (ROP) experts.
  
  
  METHODS
  A set of 34 wide-angle retinal photographs from infants with ROP was compiled on a secure Web site and was interpreted independently by 22 recognized ROP experts. Diagnostic agreement was analyzed using 3-level (plus, pre-plus, or neither) and 2-level (plus or not plus) categorizations.
  
  
  RESULTS
  In the 3-level categorization, all experts agreed on the same diagnosis in 4 of 34 images (12%), and the mean weighted kappa statistic for each expert compared with all others was between 0.21 and 0.40 (fair agreement) for 7 experts (32%) and between 0.41 and 0.60 (moderate agreement) for 15 experts (68%). In the 2-level categorization, all experts who provided a diagnosis agreed in 7 of 34 images (21%), and the mean kappa statistic for each expert compared with all others was between 0 and 0.20 (slight agreement) for 1 expert (5%), between 0.21 and 0.40 (fair agreement) for 3 experts (14%), between 0.41 and 0.60 (moderate agreement) for 12 experts (55%), and between 0.61 and 0.80 (substantial agreement) for 6 experts (27%).
  
  
  CONCLUSIONS
  Interexpert agreement of plus disease diagnosis is imperfect. This may have important implications for clinical ROP management, continued refinement of the international ROP classification system, development of computer-based diagnostic algorithms, and implementation of ROP telemedicine systems.
authors.affiliation: Department of Ophthalmology, Columbia University College of Physicians and Surgeons, New York, NY 10032, USA. chiang@dbmi.columbia.edu
authors.forename: Michael F
authors.initials: MF
authors.lastname: Chiang
authors.forename: Lei
authors.initials: L
authors.lastname: Jiang
authors.forename: Rony
authors.initials: R
authors.lastname: Gelman
authors.forename: Yunling E
authors.initials: YE
authors.lastname: Du
authors.forename: John T
authors.initials: JT
authors.lastname: Flynn
mesh_terms: D003941:Diagnostic Techniques, Ophthalmological; D004108:Dilatation, Pathologic / Q000175:diagnosis; D006801:Humans; D007091:Image Processing, Computer-Assisted; D007231:Infant, Newborn; D007234:Infant, Premature; D015588:Observer Variation; D010781:Photography; D011361:Professional Competence; D015203:Reproducibility of Results; D012171:Retinal Vessels / Q000473:pathology*; D012178:Retinopathy of Prematurity / Q000175:diagnosis*