title: Inter-expert and intra-expert agreement on the diagnosis and treatment of retinopathy of prematurity.
abstract: OBJECTIVE
  To evaluate inter-expert and intra-expert agreement on the diagnosis and treatment of retinopathy of prematurity (ROP).
  
  
  METHODS
  Prospective intra- and inter-rater reliability analysis.
  
  
  METHODS
  In this multicenter study, 260 wide-field digital photographs of 52 patients were presented to 7 recognized ROP experts on 2 consecutive assessment days 8 weeks apart. Experts were asked to assess the patients for ROP stage, presence of plus disease, presence of aggressive posterior ROP, necessity for treatment, and suggested treatment. Agreement levels were measured with Fleiss' kappa and Cohen's kappa.
  
  
  RESULTS
  Inter-expert agreement was fair for the ROP stage (κ = 0.24), plus disease (κ = 0.32), and aggressive posterior ROP (κ = 0.35); moderate for the necessity for treatment (κ = 0.41); and fair for the kind of treatment (κ = 0.38). Perfect inter-expert agreement was found in 9.6% of all patients for ROP stage 0-5, 45.1% for ≥ stage 2 ROP, 17.3% for plus disease, 57.7% for aggressive posterior ROP, and 25% for the necessity for treatment. Intra-expert agreement was higher than inter-expert agreement and was moderate for the ROP stage (κ = 0.56) and plus disease (κ = 0.51), moderate to substantial for aggressive posterior ROP (κ = 0.60), moderate for the necessity for treatment (κ = 0.47), and substantial for the kind of treatment (κ = 0.63).
  
  
  CONCLUSIONS
  ROP diagnosis and treatment decisions differ between experts and by 1 expert made on different days, indicating that the grading process is subjective and there is an observer bias when diagnosing ROP. These results could influence current practice in ROP assessment and training, and prompt further refinement of international ROP guidelines.
authors.affiliation: Department of Ophthalmology, Medical University of Vienna, Vienna, Austria.
authors.forename: Andreas
authors.initials: A
authors.lastname: Gschließer
authors.affiliation: Department of Ophthalmology, Medical University of Vienna, Vienna, Austria. Electronic address: eva.stifter@meduniwien.ac.at.
authors.forename: Eva
authors.initials: E
authors.lastname: Stifter
authors.affiliation: Department of Ophthalmology, Medical University of Vienna, Vienna, Austria.
authors.forename: Thomas
authors.initials: T
authors.lastname: Neumayer
authors.affiliation: Department of Ophthalmology, Medical University of Vienna, Vienna, Austria.
authors.forename: Elisabeth
authors.initials: E
authors.lastname: Moser
authors.affiliation: Department of Ophthalmology, Medical University of Vienna, Vienna, Austria.
authors.forename: Andrea
authors.initials: A
authors.lastname: Papp
authors.affiliation: Department of Ophthalmology, Medical University of Vienna, Vienna, Austria.
authors.forename: Niklas
authors.initials: N
authors.lastname: Pircher
authors.affiliation: Department of Ophthalmology, Medical University of Vienna, Vienna, Austria.
authors.forename: Guido
authors.initials: G
authors.lastname: Dorner
authors.affiliation: Department of Ophthalmology, Paracelsus Medical University, Salzburg, Austria.
authors.forename: Stefan
authors.initials: S
authors.lastname: Egger
authors.affiliation: Department of Ophthalmology, University Hospital Center, Zagreb, Croatia.
authors.forename: Nenad
authors.initials: N
authors.lastname: Vukojevic
authors.affiliation: Department of Ophthalmology, University Medical Center Regensburg, Regensburg, Germany.
authors.forename: Isabel
authors.initials: I
authors.lastname: Oberacher-Velten
authors.affiliation: Department of Ophthalmology, Medical University of Vienna, Vienna, Austria.
authors.forename: Ursula
authors.initials: U
authors.lastname: Schmidt-Erfurth
mesh_terms: D005260:Female; D005865:Gestational Age; D006801:Humans; D007231:Infant, Newborn; D019102:Infant, Very Low Birth Weight; D008297:Male; D015588:Observer Variation; D009885:Ophthalmology / Q000592:standards*; D010781:Photography; D011446:Prospective Studies; D015203:Reproducibility of Results; D012178:Retinopathy of Prematurity / Q000175:diagnosis* / Q000628:therapy*