Development of a test to evaluate residents' knowledge of medical procedures.
OBJECTIVE
Knowledge of core medical procedures is required by the American Board of Internal Medicine (ABIM) for certification. Efforts to improve the training of residents in these procedures have been limited by the absence of a validated tool for the assessment of knowledge. In this study we aimed to develop a standardized test of procedural knowledge in 3 medical procedures associated with potentially serious complications.


METHODS
Placement of an arterial line, central venous catheter, and thoracentesis were selected for test development. Learning objectives and multiple-choice questions were constructed for each topic. Content evidence was evaluated by critical care subspecialists. Item test characteristics were evaluated by administering the test to students, residents and specialty clinicians. Reliability of the 32-item instrument was established through its administration to 192 medical residents in 4 hospitals.


RESULTS
Reliability of the instrument as measured by Cronbach's alpha was 0.79 and its test-retest reliability was 0.82. Median score was 53% on a test comprising elements deemed important by critical care subspecialists. Increasing number of procedures attempted, higher self-reported confidence, and increasing seniority were predictors of overall test scores. Procedural confidence correlated significantly with increasing seniority and experience. Residents performed few procedures.


CONCLUSIONS
We have successfully developed a standardized instrument to assess residents' cognitive competency for 3 common procedures. Residents' overall knowledge about procedures is poor. Experiential learning is the dominant source for knowledge improvement, but these experiences are increasingly rare.
Grover
Shilpa
S
Department of Medicine, Brigham and Women's Hospital, Boston, Massachusetts 02114, USA. gmcmahon@partners.org
Currier
Paul F
PF
Elinoff
Jason M
JM
Mouchantaf
Kevin J
KJ
Katz
Joel T
JT
McMahon
Graham T
GT
D000328:Adult; D000704:Analysis of Variance; D002405:Catheterization, Central Venous / Q000592:standards*; D002406:Catheterization, Peripheral / Q000592:standards*; D002983:Clinical Competence; D004503:Education, Medical, Graduate / Q000379:methods*; D004521:Educational Measurement / Q000379:methods*; D005260:Female; D007722:Health Knowledge, Attitudes, Practice; D006801:Humans; D007388:Internal Medicine / Q000193:education*; D007396:Internship and Residency; D008297:Male; D019152:Paracentesis / Q000592:standards*; D015203:Reproducibility of Results; D018709:Statistics, Nonparametric