Use of competency-based metrics to determine effectiveness of a postgraduate thoracoscopy course.
BACKGROUND
Despite the paradigm shift from process to competency-based education, no study has explored how competency-based metrics might be used to assess short-term effectiveness of thoracoscopy-related postgraduate medical education.


OBJECTIVE
To assess the use of a single-group, pre-/post-test model comprised of multiple-choice questions (MCQ) and psychomotor skill measures to ascertain the effectiveness of a postgraduate thoracoscopy program.


METHODS
A 37-item MCQ test of cognitive knowledge was administered to 17 chest physicians before and after a 2-day continued medical education-approved program. Pre- and post-course technical skills were assessed using rigid videothoracoscopy simulation stations. Competency-based metrics (mean relative gain, mean absolute gain, and class-average normalized gain <g>) were calculated. A <g> >30% was used to determine curricular effectiveness.


RESULTS
Mean cognitive knowledge score improved significantly from 20.9 to 28.7 (7.8 ± 1.3 points, p < 0.001), representing a relative gain of 37% and an absolute gain of 21%. Mean technical skill score improved significantly from 5.20 to 7.82 (2.62 ± 0.33 points, p < 0.001), representing a relative gain of 50% and an absolute gain of 33%. Non-parametric testing confirmed t test results (p < 0.001). Class-average normalized gains were 48 and 92%, respectively.


CONCLUSIONS
Competency-based metrics, including class-average normalized gain, can be used to assess course effectiveness and to determine if a program meets predesignated objectives of knowledge acquisition and psychomotor technical skill.
Colt
Henri G
HG
Division of Pulmonary and Critical Care Medicine, University of California, Irvine, Calif. 92868-3217, USA. hcolt@uci.edu
Davoudi
Mohsen
M
Quadrelli
Silvia
S
Zamanian Rohani
Nazanin
N
D003162:Competency-Based Education / Q000706:statistics & numerical data*; D032882:Comprehension; D006801:Humans; D011446:Prospective Studies; D011597:Psychomotor Performance; D013906:Thoracoscopy / Q000193:education* / Q000592:standards