Measurement precision of spoken English proficiency scores on the USMLE step 2 clinical skills examination.
BACKGROUND
Previous research has shown that ratings of English proficiency on the United States Medical Licensing Examination Clinical Skills Examination are highly reliable. However, the score distributions for native and nonnative speakers of English are sufficiently different to suggest that reliability should be investigated separately for each group.


METHODS
Generalizability theory was used to obtain reliability indices separately for native and nonnative speakers of English (N = 29,084). Conditional standard errors of measurement were also obtained for both groups to evaluate measurement precision for each group at specific score levels.


RESULTS
Overall indices of reliability (phi) exceeded 0.90 for both native and nonnative speakers, and both groups were measured with nearly equal precision throughout the score distribution. However, measurement precision decreased at lower levels of proficiency for all examinees.


CONCLUSIONS
The results of this and future studies may be helpful in understanding and minimizing sources of measurement error at particular regions of the score distribution.
Raymond
Mark R
MR
National Board of Medical Examiners, 3750 Market Street, Philadelphia, PA 19104, USA. mraymond@nbme.org
Clauser
Brian E
BE
Swygert
Kimberly
K
van Zanten
Marta
M
D002983:Clinical Competence / Q000706:statistics & numerical data; D004521:Educational Measurement / Q000706:statistics & numerical data; D007802:Language; D008007:Licensure, Medical; D014481:United States