Observer variation in target volume delineation of lung cancer related to radiation oncologist-computer interaction: a 'Big Brother' evaluation.
OBJECTIVE
To evaluate the process of target volume delineation in lung cancer for optimization of imaging, delineation protocol and delineation software.


METHODS
Eleven radiation oncologists (observers) from five different institutions delineated the Gross Tumor Volume (GTV) including positive lymph nodes of 22 lung cancer patients (stages I-IIIB) on CT only. All radiation oncologist-computer interactions were recorded with a tool called 'Big Brother'. For each radiation oncologist and patient the following issues were analyzed: delineation time, number of delineated points and corrections, zoom levels, level and window (L/W) settings, CT slice changes, use of side windows (coronal and sagittal) and software button use.


RESULTS
The mean delineation time per GTV was 16 min (SD 10 min). The mean delineation time for lymph node positive patients was on average 3 min larger (P = 0.02) than for lymph node negative patients. Many corrections (55%) were due to L/W change (e.g. delineating in mediastinum L/W and then correcting in lung L/W). For the lymph node region, a relatively large number of corrections was found (3.7 corr/cm2), indicating that it was difficult to delineate lymph nodes. For the tumor-atelectasis region, a relative small number of corrections was found (1.0 corr/cm2), indicating that including or excluding atelectasis into the GTV was a clinical decision. Inappropriate use of L/W settings was frequently found (e.g. 46% of all delineated points in the tumor-lung region were delineated in mediastinum L/W settings). Despite a large observer variation in cranial and caudal direction of 0.72 cm (1 SD), the coronal and sagittal side windows were not used in 45 and 60% of the cases, respectively. For the more difficult cases, observer variation was smaller when the coronal and sagittal side windows were used.


CONCLUSIONS
With the 'Big Brother' tool a method was developed to trace the delineation process. The differences between observers concerning the delineation style were large. This study led to recommendations on how to improve delineation accuracy by adapting the delineation protocol (guidelines for L/W use) and delineation software (double window with lung and mediastinum L/W settings at the same time, enforced use of coronal and sagittal views) and including FDG-PET information (lymph nodes and atelectasis).
Steenbakkers
Roel J H M
RJ
The Netherlands Cancer Institute, Antoni van Leeuwenhoek Hospital, Amsterdam, The Netherlands.
Duppen
Joop C
JC
Fitton
Isabelle
I
Deurloo
Kirsten E I
KE
Zijp
Lambert
L
Uitterhoeve
Apollonia L J
AL
Rodrigus
Patrick T R
PT
Kramer
Gijsbert W P
GW
Bussink
Johan
J
De Jaeger
Katrien
K
Belderbos
José S A
JS
Hart
Augustinus A M
AA
Nowak
Peter J C M
PJ
van Herk
Marcel
M
Rasch
Coen R N
CR
D000368:Aged; D000369:Aged, 80 and over; D002289:Carcinoma, Non-Small-Cell Lung / Q000473:pathology / Q000532:radiotherapy*; D015331:Cohort Studies; D004307:Dose-Response Relationship, Radiation; D004867:Equipment Design; D004869:Equipment Safety; D005069:Evaluation Studies as Topic; D005260:Female; D006801:Humans; D008175:Lung Neoplasms / Q000473:pathology / Q000532:radiotherapy*; D008297:Male; D008875:Middle Aged; D009367:Neoplasm Staging; D015588:Observer Variation; D010818:Practice Patterns, Physicians'; D018787:Radiation Oncology / Q000592:standards / Q000639:trends; D011879:Radiotherapy Dosage; D011880:Radiotherapy Planning, Computer-Assisted / Q000295:instrumentation* / Q000379:methods; D018570:Risk Assessment; D012680:Sensitivity and Specificity; D014057:Tomography, X-Ray Computed; D016896:Treatment Outcome