Necessary steps in factor analysis: enhancing validation studies of educational instruments. The PHEEM applied to clerks as an example.
BACKGROUND
The validation of educational instruments, in particular the employment of factor analysis, can be improved in many instances.


OBJECTIVE
To demonstrate the superiority of a sophisticated method of factor analysis, implying an integration of recommendations described in the factor analysis literature, over often employed limited applications of factor analysis. We demonstrate the essential steps, focusing on the Postgraduate Hospital Educational Environment Measure (PHEEM).


METHODS
The PHEEM was completed by 279 clerks. We performed Principal Component Analysis (PCA) with varimax rotation. A combination of three psychometric criteria was applied: scree plot, eigenvalues >1.5 and a minimum percentage of additionally explained variance of approximately 5%. Furthermore, four interpretability criteria were used. Confirmatory factor analysis was performed to verify the original scale structure.


RESULTS
Our method yielded three interpretable and practically useful dimensions: learning content and coaching, beneficial affective climate and external regulation. Additionally, combining several criteria reduced the risk of overfactoring and underfactoring. Furthermore, the resulting dimensions corresponded with three learning functions essential to high-quality learning, thus strengthening our findings. Confirmatory factor analysis disproved the original scale structure.


CONCLUSIONS
Our sophisticated approach yielded several advantages over methods applied in previous validation studies. Therefore, we recommend this method in validation studies to achieve best practice.
Schönrock-Adema
Johanna
J
Center for Research and Innovation in Medical Education, University of Groningen and University Medical Center Groningen, Groningen, The Netherlands. j.schonrockadema@med.umcg.nl
Heijne-Penninga
Marjolein
M
Van Hell
Elisabeth A
EA
Cohen-Schotanus
Janke
J
D002982:Clinical Clerkship / Q000706:statistics & numerical data*; D004521:Educational Measurement / Q000379:methods*; D005163:Factor Analysis, Statistical; D006801:Humans; D007858:Learning; D017145:Models, Educational; D015233:Models, Statistical; D009426:Netherlands; D025341:Principal Component Analysis; D011594:Psychometrics; D054928:Validation Studies as Topic