International variation in histologic grading is large, and persistent feedback does not improve reproducibility.
Histologic grading systems are used to guide diagnosis, therapy, and audit on an international basis. The reproducibility of grading systems is usually tested within small groups of pathologists who have previously worked or trained together. This may underestimate the international variation of scoring systems. We therefore evaluated the reproducibility of an established system, the Banff classification of renal allograft pathology, throughout Europe. We also sought to improve reproducibility by providing individual feedback after each of 14 small groups of cases. Kappa values for all features studied were lower than any previously published, confirming that international variation is greater than interobserver variation as previously assessed. A prolonged attempt to improve reproducibility, using numeric or graphical feedback, failed to produce any detectable improvement. We then asked participants to grade selected photographs, to eliminate variation induced by pathologists viewing different areas of the slide. This produced improved kappa values only for some features. Improvement was influenced by the nature of the grade definitions. Definitions based on "area affected" by a process were not improved. The results indicate the danger of basing decisions on grading systems that may be applied very differently in different institutions.
Furness
Peter N
PN
Clinical Sciences Laboratories, Leicester General Hospital, Leicester, UK. peter.furness@le.ac.uk
Taub
Nicholas
N
Assmann
Karel J M
KJ
Banfi
Giovanni
G
Cosyns
Jean-Pierre
JP
Dorman
Anthony M
AM
Hill
Claire M
CM
Kapper
Silke K
SK
Waldherr
Rudiger
R
Laurinavicius
Aryvdas
A
Marcussen
Niels
N
Martins
Anna Paula
AP
Nogueira
Malfada
M
Regele
Heinz
H
Seron
Daniel
D
Carrera
Marta
M
Sund
Ståle
S
Taskinen
Eero I
EI
Paavonen
Timo
T
Tihomirova
Tatjana
T
Rosenthal
Rafail
R
D005060:Europe; D006801:Humans; D038622:Internationality; D016030:Kidney Transplantation / Q000473:pathology*; D015588:Observer Variation; D010336:Pathology / Q000592:standards*; D015203:Reproducibility of Results; D019737:Transplants