title: Use of competency-based metrics to determine effectiveness of a postgraduate thoracoscopy course.
abstract: BACKGROUND
  Despite the paradigm shift from process to competency-based education, no study has explored how competency-based metrics might be used to assess short-term effectiveness of thoracoscopy-related postgraduate medical education.
  
  
  OBJECTIVE
  To assess the use of a single-group, pre-/post-test model comprised of multiple-choice questions (MCQ) and psychomotor skill measures to ascertain the effectiveness of a postgraduate thoracoscopy program.
  
  
  METHODS
  A 37-item MCQ test of cognitive knowledge was administered to 17 chest physicians before and after a 2-day continued medical education-approved program. Pre- and post-course technical skills were assessed using rigid videothoracoscopy simulation stations. Competency-based metrics (mean relative gain, mean absolute gain, and class-average normalized gain <g>) were calculated. A <g> >30% was used to determine curricular effectiveness.
  
  
  RESULTS
  Mean cognitive knowledge score improved significantly from 20.9 to 28.7 (7.8 ± 1.3 points, p < 0.001), representing a relative gain of 37% and an absolute gain of 21%. Mean technical skill score improved significantly from 5.20 to 7.82 (2.62 ± 0.33 points, p < 0.001), representing a relative gain of 50% and an absolute gain of 33%. Non-parametric testing confirmed t test results (p < 0.001). Class-average normalized gains were 48 and 92%, respectively.
  
  
  CONCLUSIONS
  Competency-based metrics, including class-average normalized gain, can be used to assess course effectiveness and to determine if a program meets predesignated objectives of knowledge acquisition and psychomotor technical skill.
authors: Division of Pulmonary and Critical Care Medicine, University of California, Irvine, Calif. 92868-3217, USA. hcolt@uci.edu
authors: Henri G
authors: HG
authors: Colt
authors: Mohsen
authors: M
authors: Davoudi
authors: Silvia
authors: S
authors: Quadrelli
authors: Nazanin
authors: N
authors: Zamanian Rohani
mesh_terms: D003162:Competency-Based Education / Q000706:statistics & numerical data*; D032882:Comprehension; D006801:Humans; D011446:Prospective Studies; D011597:Psychomotor Performance; D013906:Thoracoscopy / Q000193:education* / Q000592:standards