Orion: a web-based application designed to monitor resident and fellow performance on-call.
Radiology residency and fellowship training provides a unique opportunity to evaluate trainee performance and determine the impact of various educational interventions. We have developed a simple software application (Orion) using open-source tools to facilitate the identification and monitoring of resident and fellow discrepancies in on-call preliminary reports. Over a 6-month period, 19,200 on-call studies were interpreted by 20 radiology residents, and 13,953 on-call studies were interpreted by 25 board-certified radiology fellows representing eight subspecialties. Using standard review macros during faculty interpretation, each of these reports was classified as "agreement", "minor discrepancy", and "major discrepancy" based on the potential to impact patient management or outcome. Major discrepancy rates were used to establish benchmarks for resident and fellow performance by year of training, modality, and subspecialty, and to identify residents and fellows demonstrating a significantly higher major discrepancy rate compared with their classmates. Trends in discrepancies were used to identify subspecialty-specific areas of increased major discrepancy rates in an effort to tailor the didactic and case-based curriculum. A series of missed-case conferences were developed based on trends in discrepancies, and the impact of these conferences is currently being evaluated. Orion is a powerful information technology tool that can be used by residency program directors, fellowship programs directors, residents, and fellows to improve radiology education and training.
Hospital of the University of Pennsylvania, 3400 Spruce Street, Philadelphia, PA 19104, USA. Jason.itri@uphs.upenn.edu
Jason N
JN
Itri
Woojin
W
Kim
Mary H
MH
Scanlon
D004650:Employee Performance Appraisal / Q000379:methods*; D006801:Humans; D007396:Internship and Residency / Q000592:standards; D008122:Local Area Networks; D011785:Quality Assurance, Health Care; D011871:Radiology / Q000592:standards*
Emerg Radiol. 2009 Jul;16(4):303-8
19184142
AJR Am J Roentgenol. 2007 Sep;189(3):523-6
17715095
AJNR Am J Neuroradiol. 2002 Jan;23(1):103-7
11827881
AJR Am J Roentgenol. 2003 Aug;181(2):367-73
12876012
Acad Radiol. 2010 Jul;17(7):917-20
20447839
AJR Am J Roentgenol. 2009 Aug;193(2):527-32
19620452
Radiology. 1998 Jul;208(1):125-8
9646802
Radiology. 1996 Jun;199(3):717-20
8637994
Acad Radiol. 2008 Sep;15(9):1198-204
18692761
Am J Emerg Med. 2007 Jan;25(1):45-8
17157681