title: Interobserver agreement in dysplasia grading: toward an enhanced gold standard for clinical pathology trials.
abstract: OBJECTIVE
  Interobserver agreement in the context of oral epithelial dysplasia (OED) grading has been notoriously unreliable and can impose barriers for developing new molecular markers and diagnostic technologies. This paper aimed to report the details of a 3-stage histopathology review and adjudication process with the goal of achieving a consensus histopathologic diagnosis of each biopsy.
  
  
  METHODS
  Two adjacent serial histologic sections of oral lesions from 846 patients were independently scored by 2 different pathologists from a pool of 4. In instances where the original 2 pathologists disagreed, a third, independent adjudicating pathologist conducted a review of both sections. If a majority agreement was not achieved, the third stage involved a face-to-face consensus review.
  
  
  RESULTS
  Individual pathologist pair κ values ranged from 0.251 to 0.706 (fair-good) before the 3-stage review process. During the initial review phase, the 2 pathologists agreed on a diagnosis for 69.9% of the cases. After the adjudication review by a third pathologist, an additional 22.8% of cases were given a consensus diagnosis (agreement of 2 out of 3 pathologists). After the face-to-face review, the remaining 7.3% of cases had a consensus diagnosis.
  
  
  CONCLUSIONS
  The use of the defined protocol resulted in a substantial increase (30%) in diagnostic agreement and has the potential to improve the level of agreement for establishing gold standards for studies based on histopathologic diagnosis.
authors.affiliation: Academic Unit of Oral & Maxillofacial Pathology, University of Sheffield School of Clinical Dentistry, Sheffield, UK.
authors.forename: Paul M
authors.initials: PM
authors.lastname: Speight
authors.affiliation: Rice University, Department of Bioengineering, Houston, TX, USA.
authors.forename: Timothy J
authors.initials: TJ
authors.lastname: Abram
authors.affiliation: University of Texas MD Anderson Cancer Center, Houston, TX, USA.
authors.forename: Pierre N
authors.initials: PN
authors.lastname: Floriano
authors.affiliation: Rho Inc., Chapel Hill, NC, USA.
authors.forename: Robert
authors.initials: R
authors.lastname: James
authors.affiliation: Rho Inc., Chapel Hill, NC, USA.
authors.forename: Julie
authors.initials: J
authors.lastname: Vick
authors.affiliation: Academic Unit of Oral & Maxillofacial Medicine & Surgery, University of Sheffield School of Clinical Dentistry, Sheffield, UK.
authors.forename: Martin H
authors.initials: MH
authors.lastname: Thornhill
authors.affiliation: Academic Unit of Oral & Maxillofacial Medicine & Surgery, University of Sheffield School of Clinical Dentistry, Sheffield, UK.
authors.forename: Craig
authors.initials: C
authors.lastname: Murdoch
authors.affiliation: Academic Unit of Oral & Maxillofacial Medicine & Surgery, University of Sheffield School of Clinical Dentistry, Sheffield, UK.
authors.forename: Christine
authors.initials: C
authors.lastname: Freeman
authors.affiliation: Unit of Oral Medicine, Charles Clifford Dental Hospital, Sheffield Teaching Hospitals National Health Service Foundation Trust, Sheffield, UK.
authors.forename: Anne M
authors.initials: AM
authors.lastname: Hegarty
authors.affiliation: Unit of Oral Medicine, Charles Clifford Dental Hospital, Sheffield Teaching Hospitals National Health Service Foundation Trust, Sheffield, UK.
authors.forename: Katy
authors.initials: K
authors.lastname: D'Apice
authors.affiliation: New York University College of Dentistry, Department of Oral and Maxillofacial Pathology, Radiology & Medicine, New York, NY, USA.
authors.forename: A Ross
authors.initials: AR
authors.lastname: Kerr
authors.affiliation: New York University College of Dentistry, Department of Oral and Maxillofacial Pathology, Radiology & Medicine, New York, NY, USA.
authors.forename: Joan
authors.initials: J
authors.lastname: Phelan
authors.affiliation: New York University College of Dentistry, Bluestone Center for Clinical Research, New York, NY, USA.
authors.forename: Patricia
authors.initials: P
authors.lastname: Corby
authors.affiliation: New York University College of Dentistry, Bluestone Center for Clinical Research, New York, NY, USA.
authors.forename: Ismael
authors.initials: I
authors.lastname: Khouly
authors.affiliation: The University of Texas Health Science Center at Houston, Department of Diagnostic and Biomedical Sciences, Houston, TX, USA.
authors.forename: Nadarajah
authors.initials: N
authors.lastname: Vigneswaran
authors.affiliation: The University of Texas Health Science Center at Houston, Department of Diagnostic and Biomedical Sciences, Houston, TX, USA.
authors.forename: Jerry
authors.initials: J
authors.lastname: Bouquot
authors.affiliation: The University of Texas Health Science Center at Houston, Department of Oral and Maxillofacial Surgery, Houston, TX, USA.
authors.forename: Nagi M
authors.initials: NM
authors.lastname: Demian
authors.affiliation: The University of Texas Health Science Center at Houston, Department of Otolaryngology-Head and Neck Surgery, Houston, TX, USA.
authors.forename: Y Etan
authors.initials: YE
authors.lastname: Weinstock
authors.affiliation: The University of Texas Health Science Center at San Antonio, Department of Comprehensive Dentistry, San Antonio, TX, USA.
authors.forename: Spencer W
authors.initials: SW
authors.lastname: Redding
authors.affiliation: The University of Texas Health Science Center at San Antonio, Department of Comprehensive Dentistry, San Antonio, TX, USA.
authors.forename: Stephanie
authors.initials: S
authors.lastname: Rowan
authors.affiliation: The University of Texas Health Science Center at San Antonio, Department of Comprehensive Dentistry, San Antonio, TX, USA.
authors.forename: Chih-Ko
authors.initials: CK
authors.lastname: Yeh
authors.affiliation: The University of Texas Health Science Center at San Antonio, Department of Pathology, San Antonio, TX, USA.
authors.forename: H Stan
authors.initials: HS
authors.lastname: McGuff
authors.affiliation: The University of Texas Health Science Center at San Antonio, Department of Otolaryngology-Head and Neck Surgery, San Antonio, TX, USA.
authors.forename: Frank R
authors.initials: FR
authors.lastname: Miller
authors.affiliation: Rice University, Department of Bioengineering, Houston, TX, USA; Department Biomaterials, Bioengineering Institute, New York University, New York, NY, USA; Rice University, Department of Chemistry, Houston, TX, USA. Electronic address: mcdevitt@nyu.edu.
authors.forename: John T
authors.initials: JT
authors.lastname: McDevitt
mesh_terms: D001706:Biopsy; D002278:Carcinoma in Situ / Q000473:pathology; D002471:Cell Transformation, Neoplastic / Q000473:pathology; D002986:Clinical Trials as Topic; D006801:Humans; D009061:Mouth Mucosa / Q000473:pathology; D009062:Mouth Neoplasms / Q000473:pathology*; D015588:Observer Variation; D010338:Pathology, Clinical / Q000379:methods*; D011230:Precancerous Conditions / Q000473:pathology