title: Evaluating the quality, clinical relevance, and resident perception of the radiation oncology in-training examination: A national survey.
abstract: OBJECTIVE
  The yearly radiation oncology in-training examination (ITE) by the American College of Radiology is a widely used, norm-referenced educational assessment, with high test reliability and psychometric performance. We distributed a national survey to evaluate the academic radiation oncology community's perception of the ITE.
  
  
  METHODS
  In June 2014, a 7-question online survey was distributed via e-mail to current radiation oncology residents, program directors, and attending physicians who had completed residency in the past 5 years or junior attendings. Survey questions were designed on a 5-point Likert scale. Sign test was performed with P ≤ .05 considered statistically different from neutral.
  
  
  RESULTS
  Thirty-one program directors (33.3%), 114 junior attendings (35.4%), and 225 residents (41.2%) responded. Junior attendings and program directors reported that the ITE directly contributed to their preparation for the American Board of Radiology written certification (P = .050 and .004, respectively). Residents did not perceive the examination as an accurate assessment of relevant clinical and scientific knowledge (P < .0001) and feel the quality assurance is insufficient in its current form (P < .0001). Residents and junior attendings agree that there are factual errors, and unclear questions/answers (P < .0001 and .04, respectively). Free response suggestions included: less questions on rare disease sites (16.4%), more relevance to clinical practice (15.4%), avoiding questions that discriminate between a few percentage points (11.8%), and designing the test similar to the written certification examination (9.2%).
  
  
  CONCLUSIONS
  Despite high examination reliability and psychometric performance, resident and attending physicians report a need for improved quality assurance and clinical relevance in the ITE. Although the current examination allows limited feedback, establishing a venue for individualized feedback may allow continual and timely improvement of the ITE. Adopting a criterion-referenced examination may further increase resident investment in and utilization of this valuable learning tool.
authors.affiliation: Department of Radiation Oncology, Sidney Kimmel Medical College at Thomas Jefferson University, Philadelphia, Pennsylvania. Electronic address: hyun.kim@jefferson.edu.
authors.forename: Hyun
authors.initials: H
authors.lastname: Kim
authors.affiliation: Department of Radiation Oncology, Sidney Kimmel Medical College at Thomas Jefferson University, Philadelphia, Pennsylvania.
authors.forename: Voichita
authors.initials: V
authors.lastname: Bar Ad
authors.affiliation: Jefferson College of Population Health, Thomas Jefferson University, Philadelphia, Pennsylvania.
authors.forename: John
authors.initials: J
authors.lastname: McAna
authors.affiliation: Department of Radiation Oncology, Sidney Kimmel Medical College at Thomas Jefferson University, Philadelphia, Pennsylvania.
authors.forename: Adam P
authors.initials: AP
authors.lastname: Dicker
mesh_terms: D002983:Clinical Competence; D003479:Curriculum; D004521:Educational Measurement; D006801:Humans; D007318:Inservice Training / Q000458:organization & administration*; D007396:Internship and Residency; D015397:Program Evaluation; D018787:Radiation Oncology / Q000193:education* / Q000592:standards / Q000706:statistics & numerical data; D011795:Surveys and Questionnaires