Development of a quality assessment scale for retrospective clinical studies in pediatric surgery.
OBJECTIVE
More than 97% of the clinical research in pediatric surgery consists of retrospective data. Although these studies have significant limitations, they fundamentally shape clinical practice within the field. In this report, the authors describe the development and potential applications of a standardized quality assessment scale designed for retrospective studies in pediatric surgery.


METHODS
The authors developed a comprehensive quality assessment instrument incorporating 30 items within 3 subscales. These subscales were designed to assess 3 integral components of study design: clinical relevance, reporting methodology, and the strength of conclusions. Global quality ratings (poor, fair, or good) were derived by combining scores from each subscale. To examine inter-rater reliability, 10 retrospective studies from the pediatric surgery literature were assessed with the instrument by 6 independent reviewers.


RESULTS
Inter-rater reliability was excellent as assessed by the level of agreement for all items within the instrument (84.6% concordance, n = 1,573 total item-choices) and for individual subscales (range, 73.3% to 85.8%, n = 60 to 1,258). The extent of agreement among reviewers was 82% for classifying reports into qualitative categories by global quality scores (n = 60). The ranking of papers by individual reviewers was highly predictive of overall ranking by mean quality scores (n = 60 rank pairs, r = 0.83; P <.01).


CONCLUSIONS
The authors have developed a standardized and reliable quality assessment scale for the analysis of retrospective data in pediatric surgery. Potential applications include: (1) providing the practicing surgeon with a knowledge base to critically evaluate published retrospective data; (2) providing a standardized methodology for the systematic review of existing retrospective data; and (3) developing standardized reporting guidelines for use in peer-reviewed journals.
Rangel
Shawn J
SJ
Division of Pediatric Surgery and Neonatology, and the Department of Epidemiology, Stanford University School of Medicine, Stanford, California, USA.
Kelsey
Jennifer
J
Colby
Christopher E
CE
Anderson
JoDee
J
Moss
R Lawrence
RL
D013502:General Surgery / Q000592:standards*; D006801:Humans; D015588:Observer Variation; D010372:Pediatrics; D011785:Quality Assurance, Health Care / Q000592:standards*; D015203:Reproducibility of Results; D012107:Research Design / Q000592:standards; D012189:Retrospective Studies; D011795:Surveys and Questionnaires