title: Examining the inter-rater and test-retest reliability of the Student Practice Evaluation Form-Revised (SPEF-R) for occupational therapy students.
abstract: OBJECTIVE
  The Student Practice Evaluation Form-Revised Edition (SPEF-R) is widely used in Australian universities to evaluate occupational therapy students' performance in practice education. Reliable completion of the SPEF-R by practice educators is critical for students and universities from a quality assurance perspective. This study used standardised video vignettes to examine the inter-rater and test-retest reliability of practice educators when completing the SPEF-R.
  
  
  METHODS
  Nine vignettes were developed with three levels of student performance (below expectations, adequate-to-proficient, and outstanding) for each of three scenarios which depicted a multidisciplinary team meeting, a supervision session, and a home visit respectively. Seventy-five occupational therapy practice educators viewed the vignettes and completed an online survey to rate student performance in each vignette using five selected SPEF-R items and using the five-point rating scale. Twenty of these practice educators completed the rating process twice for test-retest reliability purposes. Percentage agreement, t-tests and Rasch Measurement Model were used for analysis.
  
  
  RESULTS
  Practice educators exhibited consistent ratings on most of the SPEF-R items, except for rating adequate-to-proficient student performance on three items for inter-rater reliability and eight items for test-retest reliability. Rasch analysis found that the majority of practice educators (96-98.7%) exhibited consistency in their use of the SPEF-R rating scales. Practice educators also demonstrated satisfactory test-retest agreement of severity/leniency in rating student performance in the multidisciplinary team meeting scenario.
  
  
  CONCLUSIONS
  The study findings suggest that the SPEF-R could be used reliably and interpreted consistently by practice educators with diverse backgrounds and levels of experience.
authors.affiliation: Occupational Therapy Division, School of Health and Rehabilitation Sciences, The University of Queensland, Brisbane, Queensland.
authors.forename: Sylvia
authors.initials: S
authors.lastname: Rodger
authors.forename: Allison
authors.initials: A
authors.lastname: Coleman
authors.forename: Anne-Maree
authors.initials: AM
authors.lastname: Caine
authors.forename: Chi-Wen
authors.initials: CW
authors.lastname: Chien
authors.forename: Jodie
authors.initials: J
authors.lastname: Copley
authors.forename: Merrill
authors.initials: M
authors.lastname: Turpin
authors.forename: Ted
authors.initials: T
authors.lastname: Brown
mesh_terms: D001315:Australia; D002983:Clinical Competence; D004521:Educational Measurement / Q000379:methods* / Q000592:standards*; D006801:Humans; D015588:Observer Variation; D009788:Occupational Therapy / Q000193:education*; D015203:Reproducibility of Results
keywords: education; occupational therapy; professional; reproducibility of results