title: On the choice of acceptance radius in free-response observer performance studies.
abstract: OBJECTIVE
  Choosing an acceptance radius or proximity criterion is necessary to analyse free-response receiver operating characteristic (FROC) observer performance data. This is currently subjective, with little guidance in the literature about what is an appropriate acceptance radius. We evaluated varying acceptance radii in a nodule detection task in chest radiography and suggest guidelines for determining an acceptance radius.
  
  
  METHODS
  80 chest radiographs were chosen, half of which contained nodules. We determined each nodule's centre. 21 radiologists read the images. We created acceptance radii bins of <5 pixels, <10 pixels, <20 pixels and onwards up to <200 and 200+ pixels. We counted lesion localisations in each bin and visually compared marks with the borders of nodules.
  
  
  RESULTS
  Most reader marks were tightly clustered around nodule centres, with tighter clustering for smaller than for larger nodules. At least 70% of readers' marks were placed within <10 pixels for small nodules, <20 pixels for medium nodules and <30 pixels for large nodules. Of 72 inspected marks that were less than 50 pixels from the centre of a nodule, only 1 fell outside the border of a nodule.
  
  
  CONCLUSIONS
  The acceptance radius should be based on the larger nodule sizes. For our data, an acceptance radius of 50 pixels would have captured all but 2 reader marks within the borders of a nodule, while excluding only 1 true-positive mark. The choice of an acceptance radius for FROC analysis of observer performance studies should be based on the size of larger abnormalities.
authors.affiliation: The University of Texas MD Anderson Cancer Center, Houston, TX, USA. tamara.haygood@mdanderson.org
authors.forename: T M
authors.initials: TM
authors.lastname: Haygood
authors.forename: J
authors.initials: J
authors.lastname: Ryan
authors.forename: P C
authors.initials: PC
authors.lastname: Brennan
authors.forename: S
authors.initials: S
authors.lastname: Li
authors.forename: E M
authors.initials: EM
authors.lastname: Marom
authors.forename: M F
authors.initials: MF
authors.lastname: McEntee
authors.forename: M
authors.initials: M
authors.lastname: Itani
authors.forename: M
authors.initials: M
authors.lastname: Evanoff
authors.forename: D
authors.initials: D
authors.lastname: Chakraborty
mesh_terms: D006801:Humans; D015588:Observer Variation; D012372:ROC Curve; D011856:Radiographic Image Enhancement / Q000379:methods; D011857:Radiographic Image Interpretation, Computer-Assisted / Q000379:methods*; D013902:Radiography, Thoracic / Q000379:methods*; D015203:Reproducibility of Results; D012680:Sensitivity and Specificity; D003074:Solitary Pulmonary Nodule / Q000000981:diagnostic imaging*; D014057:Tomography, X-Ray Computed / Q000009:adverse effects / Q000379:methods*