title: QRSE: a novel metric for the evaluation of trainee radiologist reporting skills.
abstract: Diagnostic radiology training programs must produce highly skilled diagnostic radiologists capable of interpreting radiological examinations and communicating results to clinicians. Established training performance tools evaluate interpretive skills, but trainees' competency in reporting skills is also essential. Our semi-automated passive electronic tool entitled the Quantitative Reporting Skills Evaluation (QRSE) allows radiology training programs to evaluate the quantity of edits made to trainee preliminary reports by attending physicians as a metric to evaluate trainee reporting performance. Consecutive report pairs and metadata extracted from the radiology information system were anonymized and exported to a MySQL database. To perform the QRSE, for each report pair, open source software was first utilized to calculate the Levenshtein Percent (LP), the percent of character changes required to convert each preliminary report to its corresponding final report. The average LP (ALP), ALP for each trainee, and standard deviations were calculated. Eighty-four trainees and 56 attending radiologists interpreted 228,543 radiological examinations during the study period. The overall ALP was 6.38 %. Trainee-specific ALPs ranged from 1.1 to 15.3 %. Among trainee-specific ALPs, the standard deviation was 3.7 %. Our analysis identified five trainees with trainee-specific ALPs above 2 standard deviations from the mean and 14 trainees with trainee-specific ALPs less than 1 standard deviation below the mean. The QRSE methodology allows for the passive, quantitative, and longitudinal evaluation of the reporting skills of trainees during diagnostic radiology residency training. The QRSE identifies trainees with high and low levels of edits to their preliminary reports, as a marker for trainee overall reporting skills, and thus represents a novel performance metric for radiology training programs.
authors: Department of Radiology, Thomas Jefferson University Hospital, 132 South 10th Street, Main Building 10th Floor, Philadelphia, PA 19107, USA.
authors: David
authors: D
authors: Surrey
authors: Richard E
authors: RE
authors: Sharpe
authors: Richard J T
authors: RJ
authors: Gorniak
authors: Levon N
authors: LN
authors: Nazarian
authors: Vijay M
authors: VM
authors: Rao
authors: Adam E
authors: AE
authors: Flanders
mesh_terms: D002983:Clinical Competence / Q000592:standards*; D004503:Education, Medical, Graduate / Q000379:methods*; D057286:Electronic Health Records / Q000458:organization & administration*; D006801:Humans; D007396:Internship and Residency / Q000379:methods*; D008505:Medical Staff, Hospital / Q000592:standards; D011871:Radiology / Q000193:education*; D011873:Radiology Information Systems / Q000458:organization & administration*
references: J Digit Imaging. 2012 Jun;25(3):330-6
references: 21956519
references: AJR Am J Roentgenol. 2003 Aug;181(2):367-73
references: 12876012
references: Acad Radiol. 2002 Jun;9(6):721-6
references: 12061747
references: AJR Am J Roentgenol. 2009 Aug;193(2):527-32
references: 19620452
references: AJR Am J Roentgenol. 1984 Mar;142(3):647-8
references: 6607660
references: J Digit Imaging. 2011 Oct;24(5):897-907
references: 21249419
references: Radiology. 1998 Jul;208(1):125-8
references: 9646802
references: Am J Emerg Med. 2007 Jan;25(1):45-8
references: 17157681
references: Acad Radiol. 2010 Jul;17(7):917-20
references: 20447839
references: AJR Am J Roentgenol. 2000 Nov;175(5):1239-42
references: 11044014