Short answer questions in an undergraduate qualifying examination: a study of examiner variability.
Scripts of a short answer Final MB examination paper were re-marked by the original examiner and three colleagues. While individual question marks varied considerably, most examiners agreed quite well as to the overall rank order of candidates--although not about absolute total marks--but a few disagreed considerably. Examiners exhibited remarkably consistent idiosyncratic behaviour in terms of the distribution of marks awarded. These factors led to haphazard pass/fail and pass/distinction decisions on this particular component of the examination. Recommendations are put forward to improve the marking consistency and use of short answer question papers.
D004504:Education, Medical, Undergraduate; D004521:Educational Measurement; D004739:England