Assessing cardiac physical examination skills using simulation technology and real patients: a comparison study.
OBJECTIVE
High-stakes assessments of doctors' physical examination skills often employ standardised patients (SPs) who lack physical abnormalities. Simulation technology provides additional opportunities to assess these skills by mimicking physical abnormalities. The current study examined the relationship between internists' cardiac physical examination competence as assessed with simulation technology compared with that assessed with real patients (RPs).


METHODS
The cardiac physical examination skills and bedside diagnostic accuracy of 28 internists were assessed during an objective structured clinical examination (OSCE). The OSCE included 3 modalities of cardiac patients: RPs with cardiac abnormalities; SPs combined with computer-based, audio-video simulations of auscultatory abnormalities, and a cardiac patient simulator (CPS) manikin. Four cardiac diagnoses and their associated cardiac findings were matched across modalities. At each station, 2 examiners independently rated a participant's physical examination technique and global clinical competence. Two investigators separately scored diagnostic accuracy.


RESULTS
Inter-rater reliability between examiners for global ratings (GRs) ranged from 0.75-0.78 for the different modalities. Although there was no significant difference between participants' mean GRs for each modality, the correlations between participants' performances on each modality were low to modest: RP versus SP, r = 0.19; RP versus CPS, r = 0.22; SP versus CPS, r = 0.57 (P < 0.01).


CONCLUSIONS
Methodological limitations included variability between modalities in the components contributing to examiners' GRs, a paucity of objective outcome measures and restricted case sampling. No modality provided a clear 'gold standard' for the assessment of cardiac physical examination competence. These limitations need to be addressed before determining the optimal patient modality for high-stakes assessment purposes.
Hatala
Rose
R
Department of Medicine, University of British Columbia, Vancouver, British Columbia, Canada. rhatala@mac.com
Issenberg
S Barry
SB
Kassen
Barry
B
Cole
Gary
G
Bacchus
C Maria
CM
Scalese
Ross J
RJ
D002170:Canada; D002309:Cardiology / Q000193:education*; D002983:Clinical Competence / Q000592:standards*; D006331:Heart Diseases / Q000175:diagnosis*; D006801:Humans; D007396:Internship and Residency; D015588:Observer Variation; D016544:Patient Simulation; D010808:Physical Examination / Q000592:standards*; D011897:Random Allocation