Enhancing reliability in portfolio assessment: discussions between assessors.
This paper reports the reliability in assessments of a series of portfolios assembled by a cohort of participants attending a course for prospective general practice trainers. Initial individual assessments are compared with open discussion between random pairs of assessors to produce paired composite scores, and analysed using kappa statistics. Overall reliability of a global pass/refer judgement improved from a kappa of 0.26 (fair) using individual assessment, to 0.5 (moderate) with paired discussants.
Pitts
John
J
NHS Executive South East Region (Wessex Deanery) Winchester, King Alfred's College Winchester, UK. jpitts@chilgrove.u-net.com
Coles
Colin
C
Thomas
Peter
P
Smith
Frank
F
D015331:Cohort Studies; D004521:Educational Measurement; D005180:Faculty, Medical / Q000706:statistics & numerical data; D005194:Family Practice / Q000193:education* / Q000706:statistics & numerical data; D015588:Observer Variation; D015203:Reproducibility of Results; D006113:United Kingdom