title: Training improves the interobserver agreement of the expert positron emission tomography review panel in primary mediastinal B-cell lymphoma: interim analysis in the ongoing International Extranodal Lymphoma Study Group-37 study.
abstract: The International Extranodal Lymphoma Study Group (IELSG)-37 is a prospective randomized trial assessing the role of consolidation mediastinal radiotherapy after immunochemotherapy to patients with newly diagnosed primary mediastinal large B-cell lymphoma (PMBCL). It is a positron emission tomography (PET) response-guided study where patients obtaining a complete metabolic response on an end-of-therapy PET-computed tomography (CT) scan evaluated by a central review are randomized to receive radiotherapy or no further treatment. The aims of this study were to measure agreement between reviewers reporting PET-CT scans for this trial and to determine the effect of training upon concordance rates. The review panel comprised 6 experienced nuclear physicians who read PET-CT scans using the 5-point Deauville scale. Interobserver agreement (IOA) was measured at 4 time points: after a blinded review of a "training set" of 20 patients with PMBCL from the previous IELSG-26 study (phase 1); after the first 10 clinical cases enrolled in the IELSG-37 (phase 2); and after 2 further groups of 50 (phase 3) and 40 clinical cases (phase 4). After feedback from the training set and the first 10 cases, a meeting was held to discuss interpretation, and a detailed set of instructions for the review procedure was agreed and acted upon. Between 2012 and 2014, the first 100 patients were reviewed. Using Deauville score 3 as the cutoff for a complete metabolic response, the overall IOA among the reviewers was good (Krippendorff α = 0.72.) The binary concordance between pairs of reviewers (Cohen κ) ranged from 0.60 to 0.78. The IOA, initially moderate, improved progressively from phase 1 to 4 (Krippendorff α from 0.53 to 0.81; Cohen κ from 0.35-0.72 to 0.77-0.87). Our experience indicates that the agreement among "expert" nuclear physicians reporting PMBCL, even using standardized criteria, was only moderate when the study began. However, agreement improved using a harmonization process, which included a training exercise with discussion of points leading to disagreement and compiling practical rules to sit alongside commonly adopted interpretation criteria.
authors.affiliation: Nuclear Medicine and PET-CT Centre, Oncology Institute of Southern Switzerland, Bellinzona, Switzerland.
authors.forename: Luca
authors.initials: L
authors.lastname: Ceriani
authors.affiliation: Division of Imaging, King's College London, PET Center, Guy & St Thomas' Hospital, London, UK.
authors.forename: Sally
authors.initials: S
authors.lastname: Barrington
authors.affiliation: Nuclear Medicine Unit, Azienda Ospedaliera Santa Croce e Carle, Cuneo, Italy.
authors.forename: Alberto
authors.initials: A
authors.lastname: Biggi
authors.affiliation: Department of Nuclear Medicine, Oncology Center, Bydgoszcz, PET and Molecular Imaging Department, Collegium Medicum, Nicolaus Copernicus University, Toruń, Poland.
authors.forename: Bogdan
authors.initials: B
authors.lastname: Malkowski
authors.affiliation: Division of Molecular Imaging, Joint Department of Medical Imaging, Princess Margaret Cancer Center, Toronto, Ontario, Canada.
authors.forename: Ur
authors.initials: U
authors.lastname: Metser
authors.affiliation: Nuclear Medicine Unit, Arcispedale Santa Maria Nuova-IRCCS, Reggio Emilia, Italy.
authors.forename: Annibale
authors.initials: A
authors.lastname: Versari
authors.affiliation: Department of Cellular Biotechnologies and Haematology, Sapienza University, Rome, Italy.
authors.forename: Maurizio
authors.initials: M
authors.lastname: Martelli
authors.affiliation: Cancer Research UK Centre, University of Southampton, Southampton, UK.
authors.forename: Andrew
authors.initials: A
authors.lastname: Davies
authors.affiliation: Cancer Research UK Centre, University of Southampton, Southampton, UK.
authors.forename: Peter W
authors.initials: PW
authors.lastname: Johnson
authors.affiliation: Oncology Department, Oncology Institute of Southern Switzerland, Bellinzona, Switzerland.
authors.forename: Emanuele
authors.initials: E
authors.lastname: Zucca
authors.affiliation: Medical Physics Unit, Azienda Ospedaliera Santa Croce e Carle, Cuneo, Italy.
authors.forename: Stéphane
authors.initials: S
authors.lastname: Chauvie
mesh_terms: D001402:B-Lymphocytes / Q000473:pathology*; D006801:Humans; D008479:Mediastinal Neoplasms / Q000000981:diagnostic imaging* / Q000473:pathology; D015588:Observer Variation; D000072078:Positron Emission Tomography Computed Tomography / Q000379:methods*; D011446:Prospective Studies
keywords: F-18-FDG PET-CT; Lugano classification; interobserver agreement; primary mediastinal B-cell lymphoma; reading criteria