title: Crowdsourcing scoring of immunohistochemistry images: Evaluating Performance of the Crowd and an Automated Computational Method.
abstract: The assessment of protein expression in immunohistochemistry (IHC) images provides important diagnostic, prognostic and predictive information for guiding cancer diagnosis and therapy. Manual scoring of IHC images represents a logistical challenge, as the process is labor intensive and time consuming. Since the last decade, computational methods have been developed to enable the application of quantitative methods for the analysis and interpretation of protein expression in IHC images. These methods have not yet replaced manual scoring for the assessment of IHC in the majority of diagnostic laboratories and in many large-scale research studies. An alternative approach is crowdsourcing the quantification of IHC images to an undefined crowd. The aim of this study is to quantify IHC images for labeling of ER status with two different crowdsourcing approaches, image-labeling and nuclei-labeling, and compare their performance with automated methods. Crowdsourcing- derived scores obtained greater concordance with the pathologist interpretations for both image-labeling and nuclei-labeling tasks (83% and 87%), as compared to the pathologist concordance achieved by the automated method (81%) on 5,338 TMA images from 1,853 breast cancer patients. This analysis shows that crowdsourcing the scoring of protein expression in IHC images is a promising new approach for large scale cancer molecular pathology studies.
authors.affiliation: Beth Israel Deaconess Medical Center, Harvard Medical School, Department of Pathology, Boston, 02115, USA.
authors.forename: Humayun
authors.initials: H
authors.lastname: Irshad
authors.affiliation: Kaiser-Permanente, Mid-Atlantic Group, Rockville, MD, USA.
authors.forename: Eun-Yeong
authors.initials: EY
authors.lastname: Oh
authors.affiliation: City of Hope National Medical Center, Duarte, CA, USA.
authors.forename: Daniel
authors.initials: D
authors.lastname: Schmolze
authors.affiliation: Beth Israel Deaconess Medical Center, Harvard Medical School, Department of Pathology, Boston, 02115, USA.
authors.forename: Liza M
authors.initials: LM
authors.lastname: Quintana
authors.affiliation: Beth Israel Deaconess Medical Center, Harvard Medical School, Department of Pathology, Boston, 02115, USA.
authors.forename: Laura
authors.initials: L
authors.lastname: Collins
authors.affiliation: Harvard School of Public Health, Department of Epidemiology, Boston, 02115, USA.|Channing Division of Network Medicine, Brigham and Women's Hospital, Boston, 02115, USA.
authors.forename: Rulla M
authors.initials: RM
authors.lastname: Tamimi
authors.affiliation: Beth Israel Deaconess Medical Center, Harvard Medical School, Department of Pathology, Boston, 02115, USA.
authors.forename: Andrew H
authors.initials: AH
authors.lastname: Beck
mesh_terms: D014408:Biomarkers, Tumor / Q000032:analysis*; D063045:Crowdsourcing / Q000379:methods*; D020869:Gene Expression Profiling / Q000379:methods*; D006801:Humans; D007091:Image Processing, Computer-Assisted / Q000379:methods*; D007150:Immunohistochemistry / Q000379:methods*; D009369:Neoplasms / Q000473:pathology; D061848:Optical Imaging / Q000379:methods*
chemical_list: D014408:Biomarkers, Tumor