title: Observer accuracy in estimating proportions in images: implications for the semiquantitative assessment of staining reactions and a proposal for a new system.
abstract: BACKGROUND
  The results of immunohistochemical staining are often assessed by semiquantitative scoring. However, these scoring systems are usually non-standardised and there has been little evaluation of the accuracy and reliability of this subjective assessment.
  
  
  OBJECTIVE
  To assess the accuracy of observer estimation of proportions of objects in an image.
  
  
  METHODS
  Images were generated that contained known proportions of pink squares in grids of 50 x 50 and 100 x 100 squares. Observers were shown each image for five seconds in random order and either estimated the proportion of pink squares or selected the image (from a pair of images) that contained the greater proportion of pink squares. The observers were four consultant histopathologists, seven trainee histopathologists, and six control non-histopathologists.
  
  
  RESULTS
  The raw estimations of proportions showed a close correlation with the real proportions, with correlation coefficients of 0.94 and 0.95 for consultant and trainee histopathologists on the 50 x 50 grids. However, the performance in the comparison task was much higher, with an almost perfect classification for grids of equal size even when the proportions only differed by 5%.
  
  
  CONCLUSIONS
  Histopathologists can estimate proportions of objects in an image with a reasonable degree of accuracy in this abstract test system. All observers, whether histopathologists or not, can discriminate between proportions that are only 5% different in equal sized image grids. This suggests that the generation and use of carefully calibrated reference images could greatly improve the accuracy and reliability of semiquantitative scoring of immunohistochemical or any other staining.
authors.affiliation: Section of Oncology and Pathology, Division of Genomic Medicine, University of Sheffield Medical School, Beech Hill Road, Sheffield S10 2UL, South Yorkshire, UK. ss.cross@sheffield.ac.uk
authors.forename: S S
authors.initials: SS
authors.lastname: Cross
mesh_terms: D002983:Clinical Competence; D003196:Computer Graphics; D006801:Humans; D007150:Immunohistochemistry / Q000379:methods*; D015588:Observer Variation; D010364:Pattern Recognition, Visual; D015203:Reproducibility of Results; D013194:Staining and Labeling