Analysis of the relationship between score components on a standardized patient clinical skills examination.
OBJECTIVE
This work investigated the reliability of and relationships between individual case and composite scores on a standardized patient clinical skills examination.


METHODS
Four hundred ninety two fourth-year U.S. medical students received three scores [data gathering (DG), interpersonal skills (IPS), and written communication (WC)] for each of 10 standardized patient cases. mGENOVA software was used for all analyses.


RESULTS
Estimated generalizability coefficients were 0.69, 0.80, and 0.70 for the DG, IPS, and WC scores, respectively. The universe-score correlation between DG and WC was high (.83); those for DG/IPS and IPS/WC were not as strong (0.51 and 0.37, respectively). Task difficulty appears to be modestly but positively related across the three scores. Correlations between the person-by-task effects for DG/IPS and DG/WC were positive yet modest. The estimated generalizability coefficient for a ten-case test using an equally weighted composite DG/WC score was 0.78.


CONCLUSIONS
This work allows for interpretation of correlations between (1) proficiencies measured by multiple scores and (2) sources of error that affect those scores as well as for estimation of the reliability of composite scores. Results have important implications for test construction and test validity.
Margolis
Melissa J
MJ
NBME, Philadelphia, PA 19104, USA. mmargolis@nbme.org
Clauser
Brian E
BE
Swanson
David B
DB
Boulet
John R
JR
D002983:Clinical Competence / Q000706:statistics & numerical data; D006801:Humans; D008007:Licensure, Medical / Q000706:statistics & numerical data*; D008487:Medical History Taking / Q000706:statistics & numerical data*; D015999:Multivariate Analysis; D010808:Physical Examination / Q000706:statistics & numerical data*; D010817:Physician-Patient Relations; D013337:Students, Medical; D014481:United States