title: Incorporation of core competency questions into an annual national self-assessment examination for residents in physical medicine and rehabilitation: results and implications.
abstract: OBJECTIVE
  To determine the performance and change over time when incorporating questions in the core competency domains of practice-based learning and improvement (PBLI), systems-based practice (SBP), and professionalism (PROF) into the national PM&R Self-Assessment Examination for Residents (SAER).
  
  
  METHODS
  Prospective, longitudinal analysis.
  
  
  METHODS
  The national Self-Assessment Examination for Residents (SAER) in Physical Medicine and Rehabilitation, which is administered annually.
  
  
  METHODS
  Approximately 1100 PM&R residents who take the examination annually.
  
  
  METHODS
  Inclusion of progressively more challenging questions in the core competency domains of PBLI, SBP, and PROF.
  
  
  METHODS
  Individual test item level of difficulty (P value) and discrimination (point biserial index).
  
  
  RESULTS
  Compared with the overall test, questions in the subtopic areas of PBLI, SBP, and PROF were relatively easier and less discriminating (correlation of resident performance on these domains compared with that on the total test). These differences became smaller during the 3-year time period. The difficulty level of the questions in each of the subtopic domains was raised during the 3 year period to a level close to the overall exam. Discrimination of the test items improved or remained stable.
  
  
  CONCLUSIONS
  This study demonstrates that, with careful item writing and review, multiple-choice items in the PBLI, SBP, and PROF domains can be successfully incorporated into an annual, national self-assessment examination for residents. The addition of these questions had value in assessing competency while not compromising the overall validity and reliability of the exam. It is yet to be determined if resident performance on these questions corresponds to performance on other measures of competency in the areas of PBLI, SBP, and PROF.
authors.affiliation: American Academy of PM&R Self-Assessment Committee, Division of PM&R, University of Utah, Salt Lake City, UT, USA. joseph.webster@hsc.utah.edu
authors.forename: Joseph B
authors.initials: JB
authors.lastname: Webster
mesh_terms: D003162:Competency-Based Education; D032962:Consent Forms; D004521:Educational Measurement; D004992:Ethics, Medical / Q000193:education; D006801:Humans; D007396:Internship and Residency; D008137:Longitudinal Studies; D010810:Physical and Rehabilitation Medicine / Q000193:education*; D011361:Professional Competence; D011785:Quality Assurance, Health Care; D012647:Self-Assessment