title: International variation in histologic grading is large, and persistent feedback does not improve reproducibility.
abstract: Histologic grading systems are used to guide diagnosis, therapy, and audit on an international basis. The reproducibility of grading systems is usually tested within small groups of pathologists who have previously worked or trained together. This may underestimate the international variation of scoring systems. We therefore evaluated the reproducibility of an established system, the Banff classification of renal allograft pathology, throughout Europe. We also sought to improve reproducibility by providing individual feedback after each of 14 small groups of cases. Kappa values for all features studied were lower than any previously published, confirming that international variation is greater than interobserver variation as previously assessed. A prolonged attempt to improve reproducibility, using numeric or graphical feedback, failed to produce any detectable improvement. We then asked participants to grade selected photographs, to eliminate variation induced by pathologists viewing different areas of the slide. This produced improved kappa values only for some features. Improvement was influenced by the nature of the grade definitions. Definitions based on "area affected" by a process were not improved. The results indicate the danger of basing decisions on grading systems that may be applied very differently in different institutions.
authors.affiliation: Clinical Sciences Laboratories, Leicester General Hospital, Leicester, UK. peter.furness@le.ac.uk
authors.forename: Peter N
authors.initials: PN
authors.lastname: Furness
authors.forename: Nicholas
authors.initials: N
authors.lastname: Taub
authors.forename: Karel J M
authors.initials: KJ
authors.lastname: Assmann
authors.forename: Giovanni
authors.initials: G
authors.lastname: Banfi
authors.forename: Jean-Pierre
authors.initials: JP
authors.lastname: Cosyns
authors.forename: Anthony M
authors.initials: AM
authors.lastname: Dorman
authors.forename: Claire M
authors.initials: CM
authors.lastname: Hill
authors.forename: Silke K
authors.initials: SK
authors.lastname: Kapper
authors.forename: Rudiger
authors.initials: R
authors.lastname: Waldherr
authors.forename: Aryvdas
authors.initials: A
authors.lastname: Laurinavicius
authors.forename: Niels
authors.initials: N
authors.lastname: Marcussen
authors.forename: Anna Paula
authors.initials: AP
authors.lastname: Martins
authors.forename: Malfada
authors.initials: M
authors.lastname: Nogueira
authors.forename: Heinz
authors.initials: H
authors.lastname: Regele
authors.forename: Daniel
authors.initials: D
authors.lastname: Seron
authors.forename: Marta
authors.initials: M
authors.lastname: Carrera
authors.forename: Ståle
authors.initials: S
authors.lastname: Sund
authors.forename: Eero I
authors.initials: EI
authors.lastname: Taskinen
authors.forename: Timo
authors.initials: T
authors.lastname: Paavonen
authors.forename: Tatjana
authors.initials: T
authors.lastname: Tihomirova
authors.forename: Rafail
authors.initials: R
authors.lastname: Rosenthal
mesh_terms: D005060:Europe; D006801:Humans; D038622:Internationality; D016030:Kidney Transplantation / Q000473:pathology*; D015588:Observer Variation; D010336:Pathology / Q000592:standards*; D015203:Reproducibility of Results; D019737:Transplants