Evaluation of reader variability in the interpretation of follow-up CT scans at lung cancer screening.
OBJECTIVE
To measure reader agreement in determining whether lung nodules detected at baseline screening computed tomography (CT) had changed at subsequent screening examinations and to evaluate the variability in recommendations for further follow-up.


METHODS
All subjects were enrolled in the National Lung Screening Trial (NLST), and each participant consented to the use of their de-identified images for research purposes. The authors randomly selected 100 cases of nodules measuring at least 4.0 mm at 1-year screening CT that were considered by the original screening CT reader to be present on baseline CT scans; nodules considered by the original reader to have changed were oversampled. Selected images from each case showing the entire nodule at both examinations were preloaded on a picture archiving and communication system workstation. Nine radiologists served as readers, and they evaluated whether the nodule was present at baseline and recorded the bidimensional measurements and nodule characteristics at each examination, presence or absence of change, results of screening CT, and follow-up recommendations (high-level follow-up, low-level follow-up, no follow-up).


RESULTS
On the basis of reviews during case selection, five nodules seen at follow-up were judged not to have been present at baseline; for 19 of the remaining 95 cases, at least one reader judged the nodule not to have been present at baseline. For the 76 nodules that were unanimously considered to have been present at baseline, 21%-47% (mean ± standard deviation, 30% ± 9) were judged to have grown. The κ values were similar for growth (κ = 0.55) and a positive screening result (κ = 0.51) and were lower for a change in margins and attenuation (κ = 0.27-0.31). The κ value in the recommendation of high- versus low-level follow-up was high (κ = 0.66).


CONCLUSIONS
Reader agreement on nodule growth and screening result was moderate to substantial. Agreement on follow-up recommendations was lower.
Department of Radiology, University of Alabama Hospitals, 619 19th St South, Birmingham, AL 35249, USA.
Satinder
S
Singh
Paul
P
Pinsky
Naomi S
NS
Fineberg
David S
DS
Gierada
Kavita
K
Garg
Yanhui
Y
Sun
P Hrudaya
PH
Nath
D005260:Female; D006801:Humans; D008297:Male; D008403:Mass Screening / Q000706:statistics & numerical data*; D008875:Middle Aged; D015588:Observer Variation; D015995:Prevalence; D012017:Referral and Consultation / Q000706:statistics & numerical data*; D015203:Reproducibility of Results; D012680:Sensitivity and Specificity; D003074:Solitary Pulmonary Nodule / Q000000981:diagnostic imaging*; D014057:Tomography, X-Ray Computed / Q000706:statistics & numerical data*; D014481:United States / Q000453:epidemiology
Radiology. 2009 Apr;251(1):26-37
19332844
AJR Am J Roentgenol. 2006 Apr;186(4):989-94
16554568
Radiology. 2004 May;231(2):453-8
15128990
Radiology. 2000 Oct;217(1):251-6
11012453
AJR Am J Roentgenol. 2005 Oct;185(4):973-8
16177418
Eur J Cancer. 1992;28A(6-7):1054-8
1627374
Radiology. 1999 Aug;212(2):561-6
10429718
Radiology. 2004 Apr;231(1):164-8
14990809
J Clin Oncol. 2003 Jul 1;21(13):2574-82
12829678
N Engl J Med. 1994 Dec 1;331(22):1493-9
7969300
Biometrics. 1977 Jun;33(2):363-74
884196
Acad Radiol. 2003 Jun;10(6):713-5
12809426
Acad Radiol. 2007 Dec;14(12):1475-85
18035277
Radiology. 2008 Jan;246(1):265-72
18024436
Chest. 2004 Jul;126(1):114-21
15249451
Radiology. 2005 Nov;237(2):395-400
16244247
J Natl Cancer Inst. 1998 Dec 2;90(23):1801-9
9839520
Am J Respir Crit Care Med. 2002 Feb 15;165(4):508-13
11850344
Acad Radiol. 2005 Aug;12(8):948-56
16087090