title: Inter-rater evaluation of a clinical scoring system in children with asthma.
abstract: BACKGROUND
  Many clinical scores that measure the degree of asthma are used without adequate evaluation of inter-rater reliability. When reliability is tested, most often the Cohen K statistic is used, which limits the comparative results of only two raters at a time.
  
  
  OBJECTIVE
  To evaluate inter-rater agreement of a clinical asthma score using a multi-rater K statistic.
  
  
  METHODS
  Four raters administered a clinical asthma score to 17 children with clinical asthma. Five items were evaluated: O2 requirement, inspiratory breath sounds, accessory muscle use, expiratory wheeze, and cerebral function. For each, a score of zero indicated a normal state; one, moderate impairment; two, severe impairment. A multi-rater kappa statistic was used as a measure of agreement among all four raters simultaneously. This was applied using hand calculations then cross-checked by using a standard statistical syntax, a component of the Statistical Package for Social Sciences (SPSS 9.0).
  
  
  RESULTS
  Application of the multi-rater K statistic revealed strong agreement among raters on oxygenation (K = 0.759), moderate agreement for expiratory wheeze and cerebral function (K = 0.698), and poor agreement for accessory muscle use (K = 0.528) and inspiratory breath sounds (K = 0.316).
  
  
  CONCLUSIONS
  The level of agreement varied by item with the least subjective item, O2 requirement, demonstrating the highest inter-rater correlation. A multi-rater kappa statistic can be applied to data obtained from a clinical scoring instrument either manually or by using statistical syntax provided by SPSS.
authors.affiliation: Wayne State University School of Medicine, Pediatrics, Detroit, Michigan, USA. mangeli@med.wayne.edu
authors.forename: Mary Lu
authors.initials: ML
authors.lastname: Angelilli
authors.forename: Ronald
authors.initials: R
authors.lastname: Thomas
mesh_terms: D000293:Adolescent; D001249:Asthma / Q000175:diagnosis* / Q000503:physiopathology; D002648:Child; D006801:Humans; D015588:Observer Variation; D015203:Reproducibility of Results; D012720:Severity of Illness Index; D013223:Statistics as Topic / Q000379:methods