Using factor analysis to evaluate checklist items.
BACKGROUND
Relatively little research has been published examining the use of factor analytic techniques to improve the psychometric qualities of performance assessments. The purpose of the current investigation is to illustrate the potential use of structural equation modeling (SEM) with scores from a standardized patient examination.


METHODS
Checklist items for a single case were examined to compare two models within a SEM framework: a single, underlying "data gathering" construct and a two-factor model consisting of medical history interviewing and physical examination maneuvers.


RESULTS
The chi2 statistics obtained were statistically significant for both the single factor (chi2=1645.8, p<.001) and the two-factor model, (chi2=1373.1; p<.001), indicating marginal model-data fit. A difference statistic was calculated (chi2=320.9; p<.001), suggesting that the two-factor model has better fit.


CONCLUSIONS
The parameters estimated by each model could be used to evaluate checklist item performance. The results of this investigation illustrate the utility of this method in the analysis of checklist item data.
McKinley
Danette W
DW
Research and Evaluation Educational Commission for Foreign Medical Graduates, 3624 Market Street, 4th Floor, Philadelphia, PA 19104, USA. dmckinley@ecfmg.org
Boulet
John R
JR
D002983:Clinical Competence; D004521:Educational Measurement / Q000379:methods; D005163:Factor Analysis, Statistical; D006801:Humans; D008487:Medical History Taking / Q000592:standards*; D010808:Physical Examination / Q000592:standards*; D011594:Psychometrics; D011795:Surveys and Questionnaires / Q000592:standards*; D014481:United States