Observer sensitivity to small differences: a multipoint rank-order experiment.
OBJECTIVE
To evaluate observer sensitivity to small differences in image presentation, a multipoint rank-order experiment was used to identify small differences or trends in observations.


METHODS
Ten observers were presented with 50 sets of breast images that had been compressed at five different levels. Each set contained six images ranging from noncompressed to approximately 101:1 compression. Observers were asked to review all images of a case side by side and rank order the quality of each to enable determination of the presence or absence of masses and clustered microcalcifications.


RESULTS
As a group, observers were able to detect small differences among the images, even at the lower compression levels (p < .001). As compression levels and image degradation increased, the ability to identify differences between different modes also increased. Large observer variability in discrimination ability was observed.


CONCLUSIONS
Multipoint rank ordering of images viewed side by side can be an efficient method to identify small differences in image presentation. This approach to image ranking could be used to rule out or confirm the need for objective observer performance-type studies.
Good
W F
WF
Department of Radiology, University of Pittsburgh, PA 15261, USA.
Sumkin
J H
JH
Dash
N
N
Johns
C M
CM
Zuley
M L
ML
Rockette
H E
HE
Gur
D
D
D001941:Breast Diseases / Q000000981:diagnostic imaging; D001943:Breast Neoplasms / Q000000981:diagnostic imaging; D002114:Calcinosis / Q000000981:diagnostic imaging; D003951:Diagnostic Errors / Q000379:methods / Q000592:standards / Q000706:statistics & numerical data; D005260:Female; D006801:Humans; D008327:Mammography / Q000379:methods / Q000592:standards / Q000706:statistics & numerical data*; D015588:Observer Variation; D011856:Radiographic Image Enhancement / Q000379:methods / Q000592:standards; D011897:Random Allocation; D012680:Sensitivity and Specificity