title: Validity and reliability of the Standardized Orthopedic Assessment Tool (SOAT): a variation of the traditional objective structured clinical examination.
abstract: BACKGROUND
  Health care professions have replaced traditional multiple choice tests or essays with structured and practical, performance-based examinations with the hope of eliminating rater bias and measuring clinical competence.
  
  
  OBJECTIVE
  To establish the validity and reliability of the Standardized Orthopedic Assessment Tool (SOAT) as a measure of clinical competence of orthopaedic injury evaluation.
  
  
  METHODS
  Descriptive laboratory study.
  
  
  METHODS
  University.
  
  
  METHODS
  A total of 60 undergraduate students and 11 raters from 3 Canadian universities and 1 standardized patient.
  
  
  METHODS
  Students were required to complete a 30-minute musculoskeletal evaluation in 1 of 2 randomly assigned mock scenarios involving the knee (second-degree medial collateral ligament sprain) or the shoulder (third-degree supraspinatus muscle strain).
  
  
  METHODS
  We measured interreliability with an intraclass correlation coefficient (ICC) (2,k) and stability of the tool with standard error of measurement and confidence intervals. Agreement was measured using Bland-Altman plots. Concurrent validity was measured using a Pearson product moment correlation coefficient whereby the raters' global rating of a student was matched to the cumulative mean grade score.
  
  
  RESULTS
  The ICCs were 0.75 and 0.82 for the shoulder and knee cases, respectively. Bland-Altman plots indicated no systematic bias between raters. In addition, Pearson product moment correlation analysis demonstrated a strong relationship between the overall cumulative mean grade score and the global rating score of the examinees' performances.
  
  
  CONCLUSIONS
  This study demonstrated good interrater reliability of the SOAT with a standard error of measurement that indicated very modest stability, strong agreement between raters, and correlation indicative of concurrent validity.
authors.affiliation: Department of Physical Education and Recreation Studies, Mount Royal University, Calgary, AB, Canada.
authors.forename: Mark R
authors.initials: MR
authors.lastname: Lafave
authors.forename: Larry
authors.initials: L
authors.lastname: Katz
mesh_terms: D000328:Adult; D002170:Canada; D002983:Clinical Competence / Q000592:standards*; D005260:Female; D006801:Humans; D007718:Knee Injuries / Q000175:diagnosis*; D008297:Male; D009985:Orthopedics / Q000379:methods* / Q000592:standards*; D010808:Physical Examination / Q000592:standards*; D015203:Reproducibility of Results
keywords: clinical competence; health professionals; psychometrics