Reliability and accuracy assessment of Radiation Therapy Oncology Group-endorsed guidelines for brachial plexus contouring.
OBJECTIVE
The goal of this work was to validate the Radiation Therapy Oncology Group (RTOG)-endorsed guidelines for brachial plexus (BP) contouring by determining the intra- and interobserver agreement. Accuracy of the delineation process was determined using anatomically validated imaging datasets as a gold standard.


METHODS
Five observers delineated the right BP on three cadaver computed tomography (CT) datasets. To assess intraobserver variation, every observer repeated each delineation three times with a time interval of 2 weeks. The BP contours were divided into four regions for detailed analysis. Inter- and intraobserver variation was verified using the Computerized Environment for Radiation Research (CERR) software. Accuracy was measured using anatomically validated fused CT-magnetic resonance imaging (MRI) datasets by measuring the BP inclusion of the delineations.


RESULTS
The overall kappa (κ) values were rather low (mean interobserver overall κ: 0.29, mean intraobserver overall κ: 0.45), indicating poor inter- and intraobserver reliability. In general, the κ coefficient decreased gradually from the medial to lateral BP regions. The total agreement volume (TAV) was much smaller than the union volume (UV) for all delineations, resulting in a low Jaccard index (JI; interobserver agreement 0-0.124; intraobserver agreement 0.004-0.636). The overall accuracy was poor, with an average total BP inclusion of 38%. Inclusions were insufficient for the most lateral regions (region 3: 21.5%; region 4: 12.6%).


CONCLUSIONS
The inter- and intraobserver reliability of the RTOG-endorsed BP contouring guidelines was poor. BP inclusion worsened from the medial to lateral regions. Accuracy assessment of the contours showed an average BP inclusion of 38%. For the first time, this was assessed using the original anatomically validated BP volume. The RTOG-endorsed BP guidelines have insufficient accuracy and reliability, especially for the lateral head-and-neck regions.
Department of Anatomy, Ghent University, De Pintelaan 185, 9000, Ghent, Belgium, joris.vandevelde@ugent.be.
Joris
J
Van de Velde
Tom
T
Vercauteren
Werner
W
De Gersem
Johan
J
Wouters
Katrien
K
Vandecasteele
Philippe
P
Vuye
Frank
F
Vanpachtenbeke
Katharina
K
D'Herde
Ingrid
I
Kerckaert
Wilfried
W
De Neve
Tom
T
Van Hoof
D000368:Aged; D001917:Brachial Plexus / Q000000981:diagnostic imaging*; D002102:Cadaver; D005260:Female; D006801:Humans; D008297:Male; D008495:Medical Oncology / Q000592:standards*; D017410:Practice Guidelines as Topic; D011829:Radiation Dosage; D011835:Radiation Protection / Q000592:standards*; D061089:Radiotherapy, Image-Guided / Q000592:standards*; D015203:Reproducibility of Results; D012680:Sensitivity and Specificity; D014057:Tomography, X-Ray Computed / Q000592:standards*