A situational judgement test of professional behaviour: development and validation.
OBJECTIVE
To describe the development of a single best answer multiple choice situational judgement test (SJT) to assess medical students' ability to judge professional behaviour; to present results of an expert validation of the SJT; to describe experiences with two different validation formats.


METHODS
Based on educational objectives and work situations concerning professional behaviour, a SJT with 17 vignettes and 35 questions was developed. Best answers were developed according to available evidence. Forty-four experts validated the answers using either a rank order or a rating scale.


RESULTS
For sixteen questions in the rating group, thirteen questions in the ranking group and twelve questions in both groups more than two thirds of the experts agreed on a best answer. In the ranking group some experts found it difficult to commit themselves to a single best answer as instructed. In the rating group some experts did not mark any of the proposed answer options as adequate.


CONCLUSIONS
The definition of the construct of and experts for professional behaviour remain a challenge. Both validation scales have advantages and disadvantages. The two thirds criterion is somewhat arbitrary and lower or higher agreement may be acceptable or necessary depending on the purpose of the assessment.
Schubert
Sebastian
S
Assessment Division and Reformed Medical Curriculum Working Group, Deanery of Student Affairs, Charité, Universitätsmedizin Berlin. sebastian.schubert@charite.de
Ortwein
Heiderose
H
Dumitsch
Antje
A
Schwantes
Ulrich
U
Wilhelm
Oliver
O
Kiessling
Claudia
C
D000328:Adult; D000368:Aged; D001519:Behavior; D005260:Female; D006801:Humans; D007600:Judgment; D008297:Male; D008875:Middle Aged; D011364:Professional Practice; D013337:Students, Medical; D011795:Surveys and Questionnaires / Q000592:standards*