title: Observer sensitivity to small differences: a multipoint rank-order experiment.
abstract: OBJECTIVE
  To evaluate observer sensitivity to small differences in image presentation, a multipoint rank-order experiment was used to identify small differences or trends in observations.
  
  
  METHODS
  Ten observers were presented with 50 sets of breast images that had been compressed at five different levels. Each set contained six images ranging from noncompressed to approximately 101:1 compression. Observers were asked to review all images of a case side by side and rank order the quality of each to enable determination of the presence or absence of masses and clustered microcalcifications.
  
  
  RESULTS
  As a group, observers were able to detect small differences among the images, even at the lower compression levels (p < .001). As compression levels and image degradation increased, the ability to identify differences between different modes also increased. Large observer variability in discrimination ability was observed.
  
  
  CONCLUSIONS
  Multipoint rank ordering of images viewed side by side can be an efficient method to identify small differences in image presentation. This approach to image ranking could be used to rule out or confirm the need for objective observer performance-type studies.
authors.affiliation: Department of Radiology, University of Pittsburgh, PA 15261, USA.
authors.forename: W F
authors.initials: WF
authors.lastname: Good
authors.forename: J H
authors.initials: JH
authors.lastname: Sumkin
authors.forename: N
authors.initials: N
authors.lastname: Dash
authors.forename: C M
authors.initials: CM
authors.lastname: Johns
authors.forename: M L
authors.initials: ML
authors.lastname: Zuley
authors.forename: H E
authors.initials: HE
authors.lastname: Rockette
authors.forename: D
authors.initials: D
authors.lastname: Gur
mesh_terms: D001941:Breast Diseases / Q000000981:diagnostic imaging; D001943:Breast Neoplasms / Q000000981:diagnostic imaging; D002114:Calcinosis / Q000000981:diagnostic imaging; D003951:Diagnostic Errors / Q000379:methods / Q000592:standards / Q000706:statistics & numerical data; D005260:Female; D006801:Humans; D008327:Mammography / Q000379:methods / Q000592:standards / Q000706:statistics & numerical data*; D015588:Observer Variation; D011856:Radiographic Image Enhancement / Q000379:methods / Q000592:standards; D011897:Random Allocation; D012680:Sensitivity and Specificity