title: Development of a quality assessment scale for retrospective clinical studies in pediatric surgery.
abstract: OBJECTIVE
  More than 97% of the clinical research in pediatric surgery consists of retrospective data. Although these studies have significant limitations, they fundamentally shape clinical practice within the field. In this report, the authors describe the development and potential applications of a standardized quality assessment scale designed for retrospective studies in pediatric surgery.
  
  
  METHODS
  The authors developed a comprehensive quality assessment instrument incorporating 30 items within 3 subscales. These subscales were designed to assess 3 integral components of study design: clinical relevance, reporting methodology, and the strength of conclusions. Global quality ratings (poor, fair, or good) were derived by combining scores from each subscale. To examine inter-rater reliability, 10 retrospective studies from the pediatric surgery literature were assessed with the instrument by 6 independent reviewers.
  
  
  RESULTS
  Inter-rater reliability was excellent as assessed by the level of agreement for all items within the instrument (84.6% concordance, n = 1,573 total item-choices) and for individual subscales (range, 73.3% to 85.8%, n = 60 to 1,258). The extent of agreement among reviewers was 82% for classifying reports into qualitative categories by global quality scores (n = 60). The ranking of papers by individual reviewers was highly predictive of overall ranking by mean quality scores (n = 60 rank pairs, r = 0.83; P <.01).
  
  
  CONCLUSIONS
  The authors have developed a standardized and reliable quality assessment scale for the analysis of retrospective data in pediatric surgery. Potential applications include: (1) providing the practicing surgeon with a knowledge base to critically evaluate published retrospective data; (2) providing a standardized methodology for the systematic review of existing retrospective data; and (3) developing standardized reporting guidelines for use in peer-reviewed journals.
authors.affiliation: Division of Pediatric Surgery and Neonatology, and the Department of Epidemiology, Stanford University School of Medicine, Stanford, California, USA.
authors.forename: Shawn J
authors.initials: SJ
authors.lastname: Rangel
authors.forename: Jennifer
authors.initials: J
authors.lastname: Kelsey
authors.forename: Christopher E
authors.initials: CE
authors.lastname: Colby
authors.forename: JoDee
authors.initials: J
authors.lastname: Anderson
authors.forename: R Lawrence
authors.initials: RL
authors.lastname: Moss
mesh_terms: D013502:General Surgery / Q000592:standards*; D006801:Humans; D015588:Observer Variation; D010372:Pediatrics; D011785:Quality Assurance, Health Care / Q000592:standards*; D015203:Reproducibility of Results; D012107:Research Design / Q000592:standards; D012189:Retrospective Studies; D011795:Surveys and Questionnaires