title: An assessment of the inter-rater reliability of the ASA physical status score in the orthopaedic trauma population.
abstract: OBJECTIVE
  Although recent literature has demonstrated the utility of the ASA score in predicting postoperative length of stay, complication risk and potential utilization of other hospital resources, the ASA score has been inconsistently assigned by anaesthesia providers. This study tested the reliability of assignment of the ASA score classification by both attending anaesthesiologists and anaesthesia residents specifically among the orthopaedic trauma patient population.
  
  
  METHODS
  Nine case-based scenarios were created involving preoperative patients with isolated operative orthopaedic trauma injuries. The cases were created and assigned a reference score by both an attending anaesthesiologist and orthopaedic trauma surgeon. Attending and resident anaesthesiologists were asked to assign an ASA score for each case. Rater versus reference and inter-rater agreement amongst respondents was then analyzed utilizing Fleiss's Kappa and weighted and unweighted Cohen's Kappa.
  
  
  RESULTS
  Thirty three individuals provided ASA scores for each of the scenarios. The average rater versus reference reliability was substantial (Kw=0.78, SD=0.131, 95% CI=0.73-0.83). The average rater versus reference Kuw was also substantial (Kuw=0.64, SD=0.21, 95% CI=0.56-0.71). The inter-rater reliability as evaluated by Fleiss's Kappa was moderate (K=0.51, p<.001). An inter-rater comparison within the group of attendings (K=0.50, p<.001) and within the group of residents were both moderate (K=0.55, p<.001). There was a significant increase in the level of inter-rater reliability from the self-reported 'very uncomfortable' participants to the 'very comfortable' participants (uncomfortable K=0.43, comfortable K=0.59, p<.001).
  
  
  CONCLUSIONS
  This study shows substantial agreement strength for reliability of the ASA score among anaesthesiologists when evaluating orthopaedic trauma patients. The significant increase in inter-rater reliability based on anaesthesiologists' comfort with the ASA scoring method implies a need for further evaluation of ASA assessment training and routine use on the ground. These findings support the use of the ASA score as a statistically reliable tool in orthopaedic trauma.
authors.affiliation: The Vanderbilt Orthopaedic Institute Center for Health Policy, Vanderbilt University, Suite 4200, South Tower, MCE, Nashville, TN 37221, United States.
authors.forename: Rivka C
authors.initials: RC
authors.lastname: Ihejirika
authors.affiliation: The Vanderbilt Orthopaedic Institute Center for Health Policy, Vanderbilt University, Suite 4200, South Tower, MCE, Nashville, TN 37221, United States.
authors.forename: Rachel V
authors.initials: RV
authors.lastname: Thakore
authors.affiliation: The Vanderbilt Orthopaedic Institute Center for Health Policy, Vanderbilt University, Suite 4200, South Tower, MCE, Nashville, TN 37221, United States.
authors.forename: Vasanth
authors.initials: V
authors.lastname: Sathiyakumar
authors.affiliation: The Vanderbilt Orthopaedic Institute Center for Health Policy, Vanderbilt University, Suite 4200, South Tower, MCE, Nashville, TN 37221, United States.
authors.forename: Jesse M
authors.initials: JM
authors.lastname: Ehrenfeld
authors.affiliation: The Vanderbilt Orthopaedic Institute Center for Health Policy, Vanderbilt University, Suite 4200, South Tower, MCE, Nashville, TN 37221, United States.
authors.forename: William T
authors.initials: WT
authors.lastname: Obremskey
authors.affiliation: The Vanderbilt Orthopaedic Institute Center for Health Policy, Vanderbilt University, Suite 4200, South Tower, MCE, Nashville, TN 37221, United States. Electronic address: Manish.K.Sethi@vanderbilt.edu.
authors.forename: Manish K
authors.initials: MK
authors.lastname: Sethi
mesh_terms: D000776:Anesthesiology / Q000706:statistics & numerical data*; D050723:Fractures, Bone / Q000175:diagnosis* / Q000453:epidemiology; D006305:Health Status Indicators; D006801:Humans; D007902:Length of Stay / Q000706:statistics & numerical data*; D009104:Multiple Trauma / Q000175:diagnosis* / Q000453:epidemiology; D015588:Observer Variation; D019637:Orthopedic Procedures / Q000706:statistics & numerical data*; D011857:Radiographic Image Interpretation, Computer-Assisted; D015203:Reproducibility of Results; D012107:Research Design; D012955:Societies, Medical; D014481:United States / Q000453:epidemiology
keywords: ASA physical status score; Inter-rater reliability; Orthopaedic trauma