title: Interrater reliability and convergent validity of the American Academy for Cerebral Palsy and Developmental Medicine methodology for conducting systematic reviews.
abstract: OBJECTIVE
  The aim of this study was to evaluate the interrater reliability and convergent validity of the American Academy for Cerebral Palsy and Developmental Medicine's (AACPDM) methodology for conducting systematic reviews (group design studies).
  
  
  METHODS
  Four clinicians independently rated 24 articles for the level of evidence and conduct using AACPDM methodology. Study conduct was also assessed using the Effective Public Health Practice Project scale. Raters were randomly assigned to one of two pairs to resolve discrepancies. The level of agreement between individual raters and pairs was calculated using kappa (α=0.05) and intraclass correlations (ICCs; α=0.05). Spearman's rank correlation coefficient was calculated to evaluate the relationship between raters' categorization of quality categories using the two tools.
  
  
  RESULTS
  There was acceptable agreement between raters (κ=0.77; p<0.001; ICC=0.90) and between assigned pairs (κ=0.83; p<0.001; ICC=0.96) for the level of evidence ratings. There was acceptable agreement between pairs for four of the seven conduct questions (κ=0.53-0.87). ICCs (all raters) for conduct category ratings (weak, moderate, and strong) also indicated good agreement (ICC=0.76). Spearman's rho indicated a significant positive correlation for the overall quality category comparisons of the two tools (0.52; p<0.001).
  
  
  CONCLUSIONS
  The AACPDM rating system has acceptable interrater reliability. Evaluation of its study quality ratings demonstrated reasonable agreement when compared with a similar tool.
authors.affiliation: Glenrose Rehabilitation Hospital, Edmonton, AB, Canada.  lesley.wiart@ualberta.ca
authors.forename: Lesley
authors.initials: L
authors.lastname: Wiart
authors.forename: Kat
authors.initials: K
authors.lastname: Kolaski
authors.forename: Charlene
authors.initials: C
authors.lastname: Butler
authors.forename: Laura
authors.initials: L
authors.lastname: Vogtle
authors.forename: Lynne Romeiser
authors.initials: LR
authors.lastname: Logan
authors.forename: Robbin
authors.initials: R
authors.lastname: Hickman
authors.forename: Jamie
authors.initials: J
authors.lastname: Romeiser
authors.forename: Lisa
authors.initials: L
authors.lastname: Samson-Fang
authors.forename: Carey A
authors.initials: CA
authors.lastname: Matsuba
authors.forename: Micah W
authors.initials: MW
authors.lastname: Baird
authors.forename: Lori
authors.initials: L
authors.lastname: Roxborough
authors.forename: Tanja
authors.initials: T
authors.lastname: Mayson
authors.forename: Irina
authors.initials: I
authors.lastname: Dinu
mesh_terms: D002547:Cerebral Palsy; D002652:Child Behavior; D002657:Child Development; D002675:Child, Preschool; D006801:Humans; D007223:Infant; D015588:Observer Variation; D015203:Reproducibility of Results; D012196:Review Literature as Topic; D012955:Societies, Medical; D011795:Surveys and Questionnaires / Q000592:standards*