Quantitative validation of a general competency composite assessment evaluation.
OBJECTIVE
The authors sought to modify and validate a composite assessment evaluation process that assesses resident acquisition of the Accreditation Council for Graduate Medical Education (ACGME) general competencies (GCs).


METHODS
This study critically analyzed the evaluation process used in a multicenter study (150 emergency medicine resident evaluations) to determine whether the procedure was psychometrically valid. For each GC, principal component analysis (PCA) was used to determine whether certain evaluation items could be eliminated, as well as to determine the magnitude of variability explained by up to three linear combinations or "principal components." The factor proportions (factor loadings) of various eigenvectors were measured to determine the degree of variability (determined by the square of the factor proportion) within a data or item set. The factor proportions essentially measure the length of the eigenvector as determined from a correlation matrix.


RESULTS
The first three principal components are reported as factor proportion sum (% of total variability) as follows: patient care 0.91 (83%), medical knowledge 0.87 (76%), practice-based learning and improvement 0.90 (81%), interpersonal and communication skills 0.84 (71%), professionalism 0.74 (55%), and systems-based practice 0.80 (64%). PCA showed that evaluating certain traditional categories such as medical knowledge seemed to capture a single element, whereas professionalism appeared to measure a more complex, multidimensional phenomenon.


CONCLUSIONS
By using a structured development process, the authors were able to create valid evaluation items for determining resident acquisition of the ACGME GCs.
Reisdorff
Earl J
EJ
Michigan State University Emergency Medicine Residency Lansing, Lansing, MI 48910, USA. reisdoe@irmcmail.irmc.org
Carlson
Dale J
DJ
Reeves
Mat
M
Walker
Gregory
G
Hayes
Oliver W
OW
Reynolds
Brian
B
D002983:Clinical Competence; D003430:Cross-Sectional Studies; D004503:Education, Medical, Graduate / Q000379:methods*; D004521:Educational Measurement / Q000379:methods*; D004635:Emergency Medicine / Q000193:education*; D006801:Humans; D007396:Internship and Residency / Q000379:methods*; D025341:Principal Component Analysis; D015397:Program Evaluation / Q000592:standards*; D011594:Psychometrics; D014481:United States