title: Medical Knowledge Assessment by Hematology and Medical Oncology In-Training Examinations Are Better Than Program Director Assessments at Predicting Subspecialty Certification Examination Performance.
abstract: The Accreditation Council for Graduate Medical Education's Next Accreditation System requires training programs to demonstrate that fellows are achieving competence in medical knowledge (MK), as part of a global assessment of clinical competency. Passing American Board of Internal Medicine (ABIM) certification examinations is recognized as a metric of MK competency. This study examines several in-training MK assessment approaches and their ability to predict performance on the ABIM Hematology or Medical Oncology Certification Examinations. Results of a Hematology In-Service Examination (ISE) and an Oncology In-Training Examination (ITE), program director (PD) ratings, demographic variables, United States Medical Licensing Examination (USMLE), and ABIM Internal Medicine (IM) Certification Examination were compared. Stepwise multiple regression and logistic regression analyses evaluated these assessment approaches as predictors of performance on the Hematology or Medical Oncology Certification Examinations. Hematology ISE scores were the strongest predictor of Hematology Certification Examination scores (β = 0.41) (passing odds ratio [OR], 1.012; 95 % confidence interval [CI], 1.008-1.015), and the Oncology ITE scores were the strongest predictor of Medical Oncology Certification Examination scores (β = 0.45) (passing OR, 1.013; 95 % CI, 1.011-1.016). PD rating of MK was the weakest predictor of Medical Oncology Certification Examination scores (β = 0.07) and was not significantly predictive of Hematology Certification Examination scores. Hematology and Oncology ITEs are better predictors of certification examination performance than PD ratings of MK, reinforcing the effectiveness of ITEs for competency-based assessment of MK.
authors.affiliation: Division of Hematology and Oncology, University of North Carolina-Chapel Hill, Physicians Office Building, 3rd Floor 170 Manning Drive, CB# 7305, Chapel Hill, NC, 27599, USA.
authors.forename: Frances A
authors.initials: FA
authors.lastname: Collichio
authors.affiliation: Hess Consulting, 272 Rue du Replat, Lévis, Quebec, G7A 5E4, Canada.
authors.forename: Brian J
authors.initials: BJ
authors.lastname: Hess
authors.affiliation: Division of Hematology/Oncology, University of California-San Diego School of Medicine and Veterans Affairs San Diego Healthcare System, 9500 Gilman Drive #9111-E, La Jolla, CA, 92093, USA.
authors.forename: Elaine A
authors.initials: EA
authors.lastname: Muchmore
authors.affiliation: American Board of Internal Medicine, 510 Walnut St, Suite 1700, Philadelphia, PA, 19106, USA.
authors.forename: Lauren
authors.initials: L
authors.lastname: Duhigg
authors.affiliation: American Board of Internal Medicine, 510 Walnut St, Suite 1700, Philadelphia, PA, 19106, USA.
authors.forename: Rebecca S
authors.initials: RS
authors.lastname: Lipner
authors.affiliation: Test Development Services, National Board of Medical Examiners, 3750 Market St, Philadelphia, PA, 19104, USA.
authors.forename: Steven
authors.initials: S
authors.lastname: Haist
authors.affiliation: National Board of Medical Examiners, 3750 Market St, Philadelphia, PA, 19104, USA.
authors.forename: Janine L
authors.initials: JL
authors.lastname: Hawley
authors.affiliation: National Board of Medical Examiners, 3750 Market St, Philadelphia, PA, 19104, USA.
authors.forename: Carol A
authors.initials: CA
authors.lastname: Morrison
authors.affiliation: Education and Training, American Society of Hematology, 2021 L Street NW, Suite 900, Washington, DC, 20036, USA.
authors.forename: Charles P
authors.initials: CP
authors.lastname: Clayton
authors.affiliation: American Society of Clinical Oncology, 2318 Mill Road, Suite 800, Alexandria, VA, 22314, USA.
authors.forename: Marilyn J
authors.initials: MJ
authors.lastname: Raymond
authors.affiliation: Education and Training, American Society of Hematology, 2021 L Street NW, Suite 900, Washington, DC, 20036, USA.
authors.forename: Karen M
authors.initials: KM
authors.lastname: Kayoumi
authors.affiliation: Division of Hematology/Oncology, University of Michigan Health System and Veterans Affairs Ann Arbor Health System, C345 Med Inn Building/SPC 5848, 1500 E. Medical Center Drive, Ann Arbor, MI, 48109, USA. sgitlin@umich.edu.|University of Michigan Health System, C345 Med Inn Building/SPC 5848, 1500 E. Medical Center Drive, Ann Arbor, MI, 48109-5848, USA. sgitlin@umich.edu.
authors.forename: Scott D
authors.initials: SD
authors.lastname: Gitlin
mesh_terms: D002568:Certification / Q000592:standards*; D002983:Clinical Competence / Q000592:standards / Q000706:statistics & numerical data*; D004503:Education, Medical, Graduate; D004521:Educational Measurement / Q000706:statistics & numerical data*; D005257:Fellowships and Scholarships; D005260:Female; D006405:Hematology / Q000193:education*; D006801:Humans; D007396:Internship and Residency; D008297:Male; D008495:Medical Oncology / Q000193:education*
keywords: Assessment; Certification; Competency; In-service examination; In-training examination; Medical knowledge; Outcomes