title: Variation in external beam treatment plan quality: An inter-institutional study of planners and planning systems.
abstract: OBJECTIVE
  This study quantifies variation in radiation treatment plan quality for plans generated by a population of treatment planners given very specific plan objectives.
  
  
  METHODS
  A "Plan Quality Metric" (PQM) with 14 submetrics, each with a unique value function, was defined for a prostate treatment plan, serving as specific goals of a hypothetical "virtual physician." The exact PQM logic was distributed to a population of treatment planners (to remove ambiguity of plan goals or plan assessment methodology) as was a predefined computed tomographic image set and anatomic structure set (to remove anatomy delineation as a variable). Treatment planners used their clinical treatment planning system (TPS) to generate their best plan based on the specified goals and submitted their results for analysis.
  
  
  RESULTS
  One hundred forty datasets were received and 125 plans accepted and analyzed. There was wide variability in treatment plan quality (defined as the ability of the planners and plans to meet the specified goals) quantified by the PQM. Despite the variability, the resulting PQM distributions showed no statistically significant difference between TPS employed, modality (intensity modulated radiation therapy versus arc), or education and certification status of the planner. The PQM results showed negligible correlation to number of beam angles, total monitor units, years of experience of the planner, or planner confidence.
  
  
  CONCLUSIONS
  The ability of the treatment planners to meet the specified plan objectives (as quantified by the PQM) exhibited no statistical dependence on technologic parameters (TPS, modality, plan complexity), nor was the plan quality statistically different based on planner demographics (years of experience, confidence, certification, and education). Therefore, the wide variation in plan quality could be attributed to a general "planner skill" category that would lend itself to processes of continual improvement where best practices could be derived and disseminated to improve the mean quality and minimize the variation in any population of treatment planners.
authors.affiliation: Canis Lupus LLC, Merrimac, Wisconsin; Department of Human Oncology, University of Wisconsin, Madison, Wisconsin. Electronic address: alpha@canislupusllc.com.
authors.forename: Benjamin E
authors.initials: BE
authors.lastname: Nelms
authors.affiliation: Radiation Oncology Resources, Goshen, Indiana.
authors.forename: Greg
authors.initials: G
authors.lastname: Robinson
authors.affiliation: Radiation Oncology Resources, Goshen, Indiana.
authors.forename: Jay
authors.initials: J
authors.lastname: Markham
authors.affiliation: Radiation Oncology Resources, Goshen, Indiana.
authors.forename: Kyle
authors.initials: K
authors.lastname: Velasco
authors.affiliation: Radiation Oncology Resources, Goshen, Indiana.
authors.forename: Steve
authors.initials: S
authors.lastname: Boyd
authors.affiliation: Radiation Oncology Resources, Goshen, Indiana.
authors.forename: Sharath
authors.initials: S
authors.lastname: Narayan
authors.affiliation: Department of Radiation Oncology, Goshen Health System, Goshen, Indiana.
authors.forename: James
authors.initials: J
authors.lastname: Wheeler
authors.affiliation: Fox Chase Cancer Center, Philadelphia, Pennsylvania.
authors.forename: Mark L
authors.initials: ML
authors.lastname: Sobczak