The reliability of long and short cases undertaken as practice for a summative examination.
BACKGROUND
This study explores whether long and short cases performed in the workplace during training could be integrated into an overall summative assessment. Less examiner training and a less formalized structure might compromise reliability, but increased testing time might improve it.


METHODS
Results of practice long and short cases, undertaken in preparation for the Royal Australasian College of Physicians clinical examination, were compared with actual examination results. The effects on reliability of the examination were compared by modelling varying combinations of practice and examination long and short cases.


RESULTS
Fifty-nine candidates in two centres undertook 256 practice long cases and 154 practice short cases. Two practice long cases correlated with two examination long cases (r= 0.46). The reliability of a single long case was 0.22 under practice conditions and 0.36 under examination conditions. The reliability of a single short case was similar under either condition (0.18 vs 0.21). Reliability of over 0.80 could be achieved by assimilating two examination long cases and four examination short cases with varying combinations of seven practice cases.


CONCLUSIONS
Long cases undertaken in the workplace are not as reliable those undertaken under examination conditions, but short cases have similar reliability under either condition.
Wilkinson
T J
TJ
Department of Medicine, University of Otago, Christchurch, New Zealand. tim.wilkinson@otago.ac.nz
D'Orsogna
L J
LJ
Nair
B R
BR
Judd
S J
SJ
Frampton
C M
CM
D044362:Australasia; D002983:Clinical Competence / Q000592:standards; D003191:Comprehensive Health Care / Q000592:standards; D004503:Education, Medical, Graduate / Q000379:methods*; D005069:Evaluation Studies as Topic; D006801:Humans; D007388:Internal Medicine / Q000193:education*; D015203:Reproducibility of Results; D013647:Task Performance and Analysis; D013663:Teaching / Q000379:methods; D058013:Test Taking Skills / Q000379:methods / Q000523:psychology; D013997:Time Factors