title: Inter-examiner agreement in clinical evaluation.
abstract: BACKGROUND
  The reliability of assessment is an important issue in the evaluation of competence in medical and allied health practice, particularly when assessments are conducted by multiple examiners. The purpose of this study was to examine the agreement between multiple examiners in the assessment of a postgraduate physiotherapy student using a specifically designed performance evaluation system.
  
  
  METHODS
  Seven examiners simultaneously watched a recording of a postgraduate student's examination and treatment of one patient. The Postgraduate Physiotherapy Performance Assessment (PPPA) form was used to guide the assessment of performance in key areas of patient examination and management. Each examiner independently recorded a grade for each of five performance categories, and these scores were used to guide the global performance grade and mark.
  
  
  RESULTS
  Five examiners agreed on the global performance grade and four of the performance categories. The level of pass grade awarded was more variable, with scores in the performance categories spanning two grades, and in one case, three grades. The two examiners who were not in agreement with the majority consistently awarded higher grades across most performance categories.
  
  
  CONCLUSIONS
  This preliminary study has demonstrated majority agreement in global performance between multiple examiners when physiotherapy clinical practice is assessed against specific performance standards. Not all examiners awarded global grades consistent with the majority, and there was greater variability between examiners when grading performance in specific aspects of practice. These findings highlight the importance of examiner training and review sessions to improve inter-examiner agreement in assessments of clinical performance that require multiple examiners.
authors.affiliation: School of Physiotherapy and Curtin Health Innovation Research Institute, Curtin University, Perth, Australia.
authors.forename: Alan
authors.initials: A
authors.lastname: Reubenson
authors.forename: Tanis
authors.initials: T
authors.lastname: Schnepf
authors.forename: Robert
authors.initials: R
authors.lastname: Waller
authors.forename: Stephen
authors.initials: S
authors.lastname: Edmondston
mesh_terms: D002983:Clinical Competence / Q000592:standards* / Q000706:statistics & numerical data*; D004500:Education, Graduate; D004521:Educational Measurement / Q000379:methods*; D006801:Humans; D015588:Observer Variation; D026761:Physical Therapy Specialty / Q000193:education; D015203:Reproducibility of Results