title: JOURNAL CLUB: Assessment of Interobserver Variability in the Peer Review Process: Should We Agree to Disagree?
abstract: OBJECTIVE
  Peer review is an important and necessary part of radiology. There are several options to perform the peer review process. This study examines the reproducibility of peer review by comparing two scoring systems.
  
  
  METHODS
  American Board of Radiology-certified radiologists from various practice environments and subspecialties were recruited to score deidentified examinations on a web-based PACS with two scoring systems, RADPEER and Cleareview. Quantitative analysis of the scores was performed for interrater agreement.
  
  
  RESULTS
  Interobserver variability was high for both the RADPEER and Cleareview scoring systems. The interobserver correlations (kappa values) were 0.17-0.23 for RADPEER and 0.10-0.16 for Cleareview. Interrater correlation was not statistically significantly different when comparing the RADPEER and Cleareview systems (p = 0.07-0.27). The kappa values were low for the Cleareview subscores when we evaluated for missed findings (0.26), satisfaction of search (0.17), and inadequate interpretation of findings (0.12).
  
  
  CONCLUSIONS
  Our study confirms the previous report of low interobserver correlation when using the peer review process. There was low interobserver agreement seen when using both the RADPEER and the Cleareview scoring systems.
authors.affiliation: 1 Department of Radiology, University of Florida, Gainesville, FL.
authors.forename: Nupur
authors.initials: N
authors.lastname: Verma
authors.affiliation: 2 Department of Radiology, Harborview Medical Center, University of Washington, 325 9th Ave, Box 359728, Seattle, WA 98104.
authors.forename: Daniel S
authors.initials: DS
authors.lastname: Hippe
authors.affiliation: 2 Department of Radiology, Harborview Medical Center, University of Washington, 325 9th Ave, Box 359728, Seattle, WA 98104.
authors.forename: Jeffrey D
authors.initials: JD
authors.lastname: Robinson
mesh_terms: D007090:Image Interpretation, Computer-Assisted / Q000379:methods / Q000592:standards*; D015588:Observer Variation; D010380:Peer Review / Q000379:methods / Q000592:standards*; D011871:Radiology / Q000592:standards*; D011873:Radiology Information Systems / Q000145:classification* / Q000592:standards*; D015203:Reproducibility of Results; D012680:Sensitivity and Specificity; D014481:United States
keywords: RADPEER; interobserver variability peer review; quality improvement