title: Improving inter-observer variability in the evaluation of ultrasonographic features of polycystic ovaries.
abstract: BACKGROUND
  We recently reported poor inter-observer agreement in identifying and quantifying individual ultrasonographic features of polycystic ovaries. Our objective was to determine the effect of a training workshop on reducing inter-observer variation in the ultrasonographic evaluation of polycystic ovaries.
  
  
  METHODS
  Transvaginal ultrasound recordings from thirty women with polycystic ovary syndrome (PCOS) were evaluated by three radiologists and three reproductive endocrinologists both before and after an ultrasound workshop. The following endpoints were assessed: 1) follicle number per ovary (FNPO), 2) follicle number per single cross-section (FNPS), 3) largest follicle diameter, 4) ovarian volume, 5) follicle distribution pattern and 6) presence of a corpus luteum (CL). Lin's concordance correlation coefficients (rho) and kappa statistics for multiple raters (kappa) were used to assess level of inter-observer agreement (>0.80 good, 0.60 - 0.80 moderate/fair, <0.60 poor).
  
  
  RESULTS
  Following the workshop, inter-observer agreement improved for the evaluation of FNPS (rho = 0.70, delta rho = +0.11), largest follicle diameter (rho = 0.77, delta rho = +0.10), ovarian volume (rho = 0.84, delta rho = +0.12), follicle distribution pattern (kappa = 0.80, delta kappa = +0.21) and presence of a CL (kappa = 0.87, delta kappa = +0.05). No improvement was evident for FNPO (rho = 0.54, delta rho = -0.01). Both radiologists and reproductive endocrinologists demonstrated improvement in scores (p < 0.001).
  
  
  CONCLUSIONS
  Reliability in evaluating ultrasonographic features of polycystic ovaries can be significantly improved following participation in a training workshop. If ultrasonographic evidence of polycystic ovaries is to be used as an objective measure in the diagnosis of PCOS, then standardized training modules should be implemented to unify the approach to evaluating polycystic ovarian morphology.
authors.affiliation: Obstetrics, Gynecology & Reproductive Sciences, University of Saskatchewan, Saskatoon, Canada. lujan@erato.usask.ca
authors.forename: Marla E
authors.initials: ME
authors.lastname: Lujan
authors.forename: Donna R
authors.initials: DR
authors.lastname: Chizen
authors.forename: Andrew K
authors.initials: AK
authors.lastname: Peppin
authors.forename: Stefan
authors.initials: S
authors.lastname: Kriegler
authors.forename: David A
authors.initials: DA
authors.lastname: Leswick
authors.forename: Terri G
authors.initials: TG
authors.lastname: Bloski
authors.forename: Roger A
authors.initials: RA
authors.lastname: Pierson
mesh_terms: D000328:Adult; D004493:Education; D004502:Education, Medical, Continuing / Q000379:methods*; D004704:Endocrinology / Q000193:education*; D005260:Female; D006801:Humans; D015588:Observer Variation; D011085:Polycystic Ovary Syndrome / Q000000981:diagnostic imaging*; D015203:Reproducibility of Results; D014463:Ultrasonography / Q000592:standards* / Q000706:statistics & numerical data*