title: A web-based pilot study of inter-pathologist reproducibility using the ISHLT 2004 working formulation for biopsy diagnosis of cardiac allograft rejection: the European experience.
abstract: BACKGROUND
  The aim of this study was to assess, at the European level and using digital technology, the inter-pathologist reproducibility of the ISHLT 2004 system and to compare it with the 1990 system We also assessed the reproducibility of the morphologic criteria for diagnosis of antibody-mediated rejection detailed in the 2004 grading system.
  
  
  METHODS
  The hematoxylin-eosin-stained sections of 20 sets of endomyocardial biopsies were pre-selected and graded by two pathologists (A.A. and M.B.) and digitized using a telepathology digital pathology system (Aperio ImageScope System; for details refer to http://aperio.com/). Their diagnoses were considered the index diagnoses, which covered all grades of acute cellular rejection (ACR), early ischemic lesions, Quilty lesions, late ischemic lesions and (in the 2005 system) antibody-mediated rejection (AMR). Eighteen pathologists from 16 heart transplant centers in 7 European countries participated in the study. Inter-observer reproducibility was assessed using Fleiss's kappa and Krippendorff's alpha statistics.
  
  
  RESULTS
  The combined kappa value of all grades diagnosed by all 18 pathologists was 0.31 for the 1990 grading system and 0.39 for the 2005 grading system, with alpha statistics at 0.57 and 0.55, respectively. Kappa values by grade for 1990/2005, respectively, were: 0 = 0.52/0.51; 1A/1R = 0.24/0.36; 1B = 0.15; 2 = 0.13; 3A/2R = 0.29/0.29; 3B/3R = 0.13/0.23; and 4 = 0.18. For the 2 cases of AMR, 6 of 18 pathologists correctly suspected AMR on the hematoxylin-eosin slides, whereas, in each of 17 of the 18 AMR-negative cases a small percentage of pathologists (range 5% to 33%) overinterpreted the findings as suggestive for AMR.
  
  
  CONCLUSIONS
  Reproducibility studies of cardiac biopsies by pathologists in different centers at the international level were feasible using digitized slides rather than conventional histology glass slides. There was a small improvement in interobserver agreement between pathologists of different European centers when moving from the 1990 ISHLT classification to the "new" 2005 ISHLT classification. Morphologic suspicion of AMR in the 2004 system on hematoxylin-eosin-stained slides only was poor, highlighting the need for better standardization of morphologic criteria for AMR. Ongoing educational programs are needed to ensure standardization of diagnosis of both acute cellular and antibody-mediated rejection.
authors.affiliation: Department of Medical Diagnostic Sciences and Special Therapies, University of Padua, Italy. annalisa.angelini@unipd.it
authors.forename: Annalisa
authors.initials: A
authors.lastname: Angelini
authors.forename: Claus Boegelund
authors.initials: CB
authors.lastname: Andersen
authors.forename: Giovanni
authors.initials: G
authors.lastname: Bartoloni
authors.forename: Fiona
authors.initials: F
authors.lastname: Black
authors.forename: Paul
authors.initials: P
authors.lastname: Bishop
authors.forename: Helen
authors.initials: H
authors.lastname: Doran
authors.forename: Marny
authors.initials: M
authors.lastname: Fedrigo
authors.forename: Jochen W U
authors.initials: JW
authors.lastname: Fries
authors.forename: Martin
authors.initials: M
authors.lastname: Goddard
authors.forename: Heike
authors.initials: H
authors.lastname: Goebel
authors.forename: Desley
authors.initials: D
authors.lastname: Neil
authors.forename: Ornella
authors.initials: O
authors.lastname: Leone
authors.forename: Andrea
authors.initials: A
authors.lastname: Marzullo
authors.forename: Monika
authors.initials: M
authors.lastname: Ortmann
authors.forename: Francois
authors.initials: F
authors.lastname: Paraf
authors.forename: Samuel
authors.initials: S
authors.lastname: Rotman
authors.forename: Nesrin
authors.initials: N
authors.lastname: Turhan
authors.forename: Patrick
authors.initials: P
authors.lastname: Bruneval
authors.forename: Anna Chiara
authors.initials: AC
authors.lastname: Frigo
authors.forename: Francesco
authors.initials: F
authors.lastname: Grigoletto
authors.forename: Alessio
authors.initials: A
authors.lastname: Gasparetto
authors.forename: Roberto
authors.initials: R
authors.lastname: Mencarelli
authors.forename: Gaetano
authors.initials: G
authors.lastname: Thiene
authors.forename: Margaret
authors.initials: M
authors.lastname: Burke
mesh_terms: D000208:Acute Disease; D001706:Biopsy / Q000379:methods*; D003937:Diagnosis, Differential; D005060:Europe; D006084:Graft Rejection / Q000145:classification / Q000473:pathology*; D006331:Heart Diseases / Q000601:surgery; D016027:Heart Transplantation / Q000473:pathology*; D006801:Humans; D020407:Internet; D009206:Myocardium / Q000473:pathology*; D010865:Pilot Projects; D012372:ROC Curve; D015203:Reproducibility of Results; D012189:Retrospective Studies; D014184:Transplantation, Homologous