title: A test of performance of breast MRI interpretation in a multicentre screening study.
abstract: OBJECTIVE
  The aim of this study was to assess the consistency and performance of radiologists interpreting breast magnetic resonance imaging (MRI) examinations.
  
  
  METHODS
  Two test sets of eight cases comprising cancers, benign disease, technical problems and parenchymal enhancement were prepared from two manufacturers' equipment (X and Y) and reported by 15 radiologists using the recording form and scoring system of the UK MRI breast screening study [(MAgnetic Resonance Imaging in Breast Screening (MARIBS)]. Variations in assessments of morphology, kinetic scores and diagnosis were measured by assessing intraobserver and interobserver variability and agreement. The sensitivity and specificity of reporting performances was determined using receiver operating characteristic (ROC) curve analysis.
  
  
  RESULTS
  Intraobserver variation was seen in 13 (27.7%) of 47 of the radiologists' conclusions (four technical and seven pathological differences). Substantial interobserver variation was observed in the scores recorded for morphology, pattern of enhancement, quantification of enhancement and washout pattern. The overall sensitivity of breast MRI was high [88.6%, 95% confidence interval (CI) 77.4-94.7%], combined with a specificity of 69.2% (95% CI 60.5-76.7%). The sensitivities were similar for the two test sets (P=.3), but the specificity was significantly higher for the Manufacturer X dataset (P<.001). ROC curve analysis gave an area under the curve of 0.85 (95% CI 0.79-0.92)
  
  
  CONCLUSIONS
  Substantial variation in all elements of the scoring system and in the overall diagnostic conclusions was observed between radiologists participating in MARIBS. High overall sensitivity was achieved with moderate specificity. Manufacturer-related differences in specificities possibly occurred because the numerical thresholds set for the scoring system were not optimised for both equipment manufacturers. Scoring systems developed on one equipment software may not be transferable to other manufacturers.
authors.affiliation: Department of Radiology, Addenbrooke's Hospital, Cambridge CB2 2QQ, UK.
authors.forename: Ruth
authors.initials: R
authors.lastname: Warren
authors.forename: Carmel
authors.initials: C
authors.lastname: Hayes
authors.forename: Linda
authors.initials: L
authors.lastname: Pointon
authors.forename: Rebecca
authors.initials: R
authors.lastname: Hoff
authors.forename: Fiona J
authors.initials: FJ
authors.lastname: Gilbert
authors.forename: Anwar R
authors.initials: AR
authors.lastname: Padhani
authors.forename: Caroline
authors.initials: C
authors.lastname: Rubin
authors.forename: Glenda
authors.initials: G
authors.lastname: Kaplan
authors.forename: Kauza
authors.initials: K
authors.lastname: Raza
authors.forename: Laura
authors.initials: L
authors.lastname: Wilkinson
authors.forename: Margaret
authors.initials: M
authors.lastname: Hall-Craggs
authors.forename: Preminda
authors.initials: P
authors.lastname: Kessar
authors.forename: Sheila
authors.initials: S
authors.lastname: Rankin
authors.forename: Adrian K
authors.initials: AK
authors.lastname: Dixon
authors.forename: James
authors.initials: J
authors.lastname: Walsh
authors.forename: Lindsay
authors.initials: L
authors.lastname: Turnbull
authors.forename: Peter
authors.initials: P
authors.lastname: Britton
authors.forename: Ruchi
authors.initials: R
authors.lastname: Sinnatamby
authors.forename: Doug
authors.initials: D
authors.lastname: Easton
authors.forename: Deborah
authors.initials: D
authors.lastname: Thompson
authors.forename: Sunil R
authors.initials: SR
authors.lastname: Lakhani
authors.forename: Martin O
authors.initials: MO
authors.lastname: Leach
mesh_terms: D001943:Breast Neoplasms / Q000175:diagnosis*; D002983:Clinical Competence; D005260:Female; D006801:Humans; D007090:Image Interpretation, Computer-Assisted; D008279:Magnetic Resonance Imaging; D008403:Mass Screening; D015588:Observer Variation; D012372:ROC Curve; D015203:Reproducibility of Results; D012680:Sensitivity and Specificity