title: Observer variability in a phase II trial - assessing consistency in RECIST application.
abstract: OBJECTIVE
  To assess the consistency of Response Evaluation Criteria in Solid Tumours (RECIST) application in a phase II trial.
  
  
  METHODS
  Patients with metastatic non-resectable colorectal cancer treated with a combination of an antibody and a chemotherapeutic drug, were included. Computed tomography (CT) scans (thorax, abdomen and pelvis) were performed at baseline and after every fourth treatment cycle. RECIST was intended for response evaluation. The scans were consecutively read by a heterogeneous group of radiologists as a part of daily work and hereafter retrospectively reviewed by a dedicated experienced radiologist. Agreement on best overall response (BOR) between readers and reviewer was quantified using κ-coefficients and the discrepancy rate was correlated with the number of different readers per patient using a χ(2)-test.
  
  
  RESULTS
  One hundred patients with 396 CT scans were included. Discrepancies between the readers and the reviewer were found in 47 patients. The majority of discrepancies concerned the application of RECIST. With the review, BOR changed in 17 patients, although, only in six patients the change was potentially treatment altering. Overall, the κ-coefficient of agreement between readers and reviewer was 0.71 (good). However, in the subgroup of responding patients the κ-coefficient was 0.21 (fair). The number of patients with discrepancies was significantly higher with three or more different readers per patient than with less (p =0.0003).
  
  
  CONCLUSIONS
  RECIST was not consistently applied and the majority of the reader discrepancies were RECIST related. Post review, 17 patients changed BOR; six patients in a potentially treatment altering manner. Additionally, we found that the part of patients with discrepancies increased significantly with more than three different readers per patient. The findings support a peer-review approach where a few dedicated radiologists perform double blinded readings of all the on-going cancer trial patients' CT scans.
authors.affiliation: Department of Oncology, Copenhagen University Hospital Herlev, Denmark. kristinskougaard@dadlnet.dk
authors.forename: Kristin
authors.initials: K
authors.lastname: Skougaard
authors.forename: Mark James Dusgaard
authors.initials: MJ
authors.lastname: McCullagh
authors.forename: Dorte
authors.initials: D
authors.lastname: Nielsen
authors.forename: Helle Westergren
authors.initials: HW
authors.lastname: Hendel
authors.forename: Benny Vittrup
authors.initials: BV
authors.lastname: Jensen
authors.forename: Helle Hjorth
authors.initials: HH
authors.lastname: Johannesen
mesh_terms: D000328:Adult; D000368:Aged; D015179:Colorectal Neoplasms / Q000175:diagnosis* / Q000628:therapy*; D005260:Female; D006801:Humans; D008297:Male; D008875:Middle Aged; D015588:Observer Variation; D017063:Outcome Assessment, Health Care; D011379:Prognosis; D011446:Prospective Studies; D014057:Tomography, X-Ray Computed