title: Introducing a Novel Applicant Ranking Tool to Predict Future Resident Performance: A Pilot Study.
abstract: The purposes of this study are to (1) introduce our novel Applicant Ranking Tool that aligns with the Accreditation Council for Graduate Medical Education competencies and (2) share our preliminary results comparing applicant rank to current performance. After a thorough literature review and multiple roundtable discussions, an Applicant Ranking Tool was created. Feasibility, satisfaction, and critiques were discussed via open feedback session. Inter-rater reliability was assessed using weighted kappa statistic (κ) and Kendall coefficient of concordance (W). Fisher's exact tests evaluated the ability of the tool to stratify performance into the top or bottom half of their class. Internal medicine and anesthesiology residents served as the pilot cohorts. The tool was considered user-friendly for both data input and analysis. Inter-rater reliability was strongest with intradisciplinary evaluation (W = 0.8-0.975). Resident performance was successfully stratified into those functioning in the upper vs. lower half of their class within the Clinical Anesthesia-3 grouping (p = 0.008). This novel Applicant Ranking Tool lends support for the use of both cognitive and noncognitive traits in predicting resident performance. While the ability of this instrument to accurately predict future resident performance will take years to answer, this pilot study suggests the instrument is worthy of ongoing investigation.
authors.affiliation: San Antonio Uniformed Services Health Education Consortium, San Antonio Military Medical Center, 3551 Roger Brooke Drive, Fort Sam Houston, TX 78234.
authors.forename: Sarah N
authors.initials: SN
authors.lastname: Bowe
authors.affiliation: San Antonio Uniformed Services Health Education Consortium, San Antonio Military Medical Center, 3551 Roger Brooke Drive, Fort Sam Houston, TX 78234.
authors.forename: Erik K
authors.initials: EK
authors.lastname: Weitzel
authors.affiliation: San Antonio Uniformed Services Health Education Consortium, San Antonio Military Medical Center, 3551 Roger Brooke Drive, Fort Sam Houston, TX 78234.
authors.forename: William N
authors.initials: WN
authors.lastname: Hannah
authors.affiliation: San Antonio Uniformed Services Health Education Consortium, San Antonio Military Medical Center, 3551 Roger Brooke Drive, Fort Sam Houston, TX 78234.
authors.forename: Brian M
authors.initials: BM
authors.lastname: Fitzgerald
authors.affiliation: San Antonio Uniformed Services Health Education Consortium, San Antonio Military Medical Center, 3551 Roger Brooke Drive, Fort Sam Houston, TX 78234.
authors.forename: Gregory P
authors.initials: GP
authors.lastname: Kraus
authors.affiliation: San Antonio Uniformed Services Health Education Consortium, San Antonio Military Medical Center, 3551 Roger Brooke Drive, Fort Sam Houston, TX 78234.
authors.forename: Christopher J
authors.initials: CJ
authors.lastname: Nagy
authors.affiliation: San Antonio Uniformed Services Health Education Consortium, San Antonio Military Medical Center, 3551 Roger Brooke Drive, Fort Sam Houston, TX 78234.
authors.forename: Stephen A
authors.initials: SA
authors.lastname: Harrison
mesh_terms: D000076363:Academic Success; D000776:Anesthesiology / Q000193:education; D002983:Clinical Competence / Q000592:standards; D004503:Education, Medical, Graduate / Q000379:methods / Q000592:standards; D004521:Educational Measurement / Q000379:methods*; D006801:Humans; D007388:Internal Medicine / Q000193:education; D007396:Internship and Residency / Q000639:trends*; D007586:Job Application; D010036:Otolaryngology / Q000193:education; D010552:Personality Assessment; D010865:Pilot Projects; D015203:Reproducibility of Results; D012570:School Admission Criteria / Q000639:trends*; D011795:Surveys and Questionnaires; D000066477:Work Performance / Q000592:standards*