Scoring and analysis of performance examinations: a comparison of methods and interpretations.
The purpose of this study was to compare the results and interpretation of the data from a performance examination when four methods of analysis are used. Methods are 1) traditional summary statistics, 2) inter-judge correlations, 3) generalizability theory, and 4) the multi-facet Rasch model. Results indicated that similar sources of variance were identified using each method; however, the multi-facet Rasch model is the only method that linearized the scores and accounts for differences in the particular examination challenged by a candidate before ability estimates are calculated.
Lunz
M E
ME
American Society of Clinical Pathologists, Chicago, IL 60612-0277, USA.
Schumacker
R E
RE
D003627:Data Interpretation, Statistical; D006801:Humans; D015588:Observer Variation; D011594:Psychometrics; D012984:Software; D013647:Task Performance and Analysis