title: Product analysis and initial reliability testing of the total mesorectal excision-quality assessment instrument.
abstract: BACKGROUND
  Product analysis of rectal cancer resection specimens before specimen fixation may provide an immediate and relevant evaluation of surgical performance. We tested the interrater reliability (IRR) of a product analysis tool called the Total Mesorectal Excision-Quality Assessment Instrument (TME-QA).
  
  
  METHODS
  Participants included two gold standard raters, five pathology assistants, and eight pathologists. Domains of the TME-QA reflect total mesorectal excision principles including: (1) completeness of mesorectal margin; (2) completeness of mesorectum; (3) coning of distal mesorectum; (4) physical defects; and (5) overall specimen quality. Specimens were scored independently. We used the generalizability theory to assess the tool's internal consistency and IRR.
  
  
  RESULTS
  There were 39 specimens and 120 ratings. Mean overall specimen quality scores for the gold standard raters, pathologists, and assistants were 4.43, 4.43, and 4.50, respectively (p > 0.85). IRR for the first nine items was 0.68 for the full sample, 0.62 for assistants alone, 0.63 for pathologists alone, and 0.74 for gold standard raters alone. IRR for the item overall specimen quality was 0.67 for the full sample, 0.45 for assistants, 0.80 for pathologists, and 0.86 for gold standard raters. IRR increased for all groups when scores were averaged across two raters.
  
  
  CONCLUSIONS
  Assessment of surgical specimens using the TME-QA may provide rapid and relevant feedback to surgeons about their technical performance. Our results show good internal consistency and IRR when the TME-QA is used by pathologists. However, for pathology assistants, multiple ratings with the averaging of scores may be needed.
authors.affiliation: Department of Surgery, McMaster University, Hamilton, ON, Canada, marko.simunovic@jcc.hhsc.ca.
authors.forename: Marko R
authors.initials: MR
authors.lastname: Simunovic
authors.forename: Franco G
authors.initials: FG
authors.lastname: DeNardi
authors.forename: Angela J
authors.initials: AJ
authors.lastname: Coates
authors.forename: David A
authors.initials: DA
authors.lastname: Szalay
authors.forename: Kevin W
authors.initials: KW
authors.lastname: Eva
mesh_terms: D013505:Digestive System Surgical Procedures / Q000592:standards*; D006801:Humans; D010338:Pathology, Clinical / Q000592:standards*; D017410:Practice Guidelines as Topic / Q000592:standards*; D011379:Prognosis; D019984:Quality Indicators, Health Care / Q000592:standards*; D012004:Rectal Neoplasms / Q000473:pathology / Q000601:surgery*; D015203:Reproducibility of Results