Evaluation of procedure-based assessment for assessing trainees' skills in the operating theatre.
BACKGROUND
Procedure-based assessment (PBA) is used within most UK surgical training programmes for assessing trainees' procedural skills in the operating theatre. All postgraduate assessment methods require evidence to support their implementation. The aims were to evaluate the validity, reliability and acceptability of PBA.


METHODS
Eighty-one trainees in six surgical specialties were assessed performing common procedures; 749 PBAs were provided across 348 operations by 57 clinical supervisors and four independent assessors.


RESULTS
Construct validity was demonstrated by correlation of PBA scores with measures of surgical training and experience. Reliability (G exceeding 0·8) of the adjusted total item score and global summary score for a given procedure was achieved using four and three assessor judgements respectively; assessing a mix of procedures required more cases/assessors because performance is procedure specific. The acceptability of PBA for assessment and feedback within surgical training was predominantly positive among clinical supervisors and trainees.


CONCLUSIONS
PBA demonstrated good overall validity and acceptability, and exceptionally high reliability. Trainees should be assessed adequately for each given procedure.
Department of Reproductive and Developmental Medicine, University of Sheffield, Sheffield, UK.
J
J
Marriott
H
H
Purdie
J
J
Crossley
J D
JD
Beard
D000328:Adult; D002983:Clinical Competence / Q000592:standards*; D004503:Education, Medical, Graduate; D005246:Feedback; D005260:Female; D006801:Humans; D008297:Male; D010549:Personal Satisfaction; D013043:Specialties, Surgical / Q000193:education*; D013514:Surgical Procedures, Operative / Q000193:education*; D013997:Time Factors