title: A Systematic Review of Surgical Randomized Controlled Trials: Part I. Risk of Bias and Outcomes: Common Pitfalls Plastic Surgeons Can Overcome.
abstract: BACKGROUND
  The authors investigated the methodological validity of plastic surgery randomized controlled trials that compared surgical interventions.
  
  
  METHODS
  An electronic search identified randomized controlled trials published between 2000 and 2013. Reviewers, independently and in duplicate, assessed manuscripts and performed data extraction. Methodological safeguards (randomization, allocation concealment, blinding, and incomplete outcome data) were examined using the Cochrane risk of bias tool. Regression analysis was used to identify trial characteristics associated with risk of bias.
  
  
  RESULTS
  Of 1664 potentially eligible studies, 173 randomized controlled trials were included. Proper randomization and allocation concealment methods were described in 61 of 173 (35 percent) and 21 of 173 (12 percent), respectively. Outcome assessors were blinded in 58 of 173 (34 percent) trials, and patients were blinded in 45 of 173 (26 percent). Follow-up rates were high, with 99 of 173 (57 percent) randomized controlled trials appearing to have complete follow-up. An intention-to-treat analysis was used in 19 of 173 (11 percent) trials. One-third (58 of 173, 34 percent) did not state their primary outcomes. The most common type of primary outcome used was a symptom/quality of life, class III, outcome (73 of 173, 42 percent). Multinomial regression demonstrated trials reporting an a priori sample size as more likely to have a low risk of bias (p = 0.001).
  
  
  CONCLUSIONS
  This article highlights methodological safeguards that plastic surgeons should consider when interpreting results of a surgical randomized controlled trial. Allocation concealment, outcome assessor blinding, and patient blinding were identified as areas of concern. Valid and reliable outcome measures are being used in plastic surgery. This analysis provides strong rationale for continued focus on the performance and reporting of clinical trials within our specialty.
authors.affiliation: Hamilton and Toronto, Ontario, Canada; and Ann Arbor, Mich. From the Divisions of Plastic and Reconstructive Surgery and Orthopedic Surgery, Department of Surgery, the Surgical Outcomes Research Center, the Health Sciences Library, and the Department of Clinical Epidemiology and Biostatistics, McMaster University; the Division of Plastic and Reconstructive Surgery, Department of Surgery, University of Toronto; and the Section of Plastic Surgery, Department of Surgery, The University of Michigan Health System.
authors.forename: Sophocles H
authors.initials: SH
authors.lastname: Voineskos
authors.forename: Christopher J
authors.initials: CJ
authors.lastname: Coroneos
authors.forename: Natalia I
authors.initials: NI
authors.lastname: Ziolkowski
authors.forename: Manraj N
authors.initials: MN
authors.lastname: Kaur
authors.forename: Laura
authors.initials: L
authors.lastname: Banfield
authors.forename: Maureen O
authors.initials: MO
authors.lastname: Meade
authors.forename: Achilleas
authors.initials: A
authors.lastname: Thoma
authors.forename: Kevin C
authors.initials: KC
authors.lastname: Chung
authors.forename: Mohit
authors.initials: M
authors.lastname: Bhandari
mesh_terms: D015982:Bias; D017063:Outcome Assessment, Health Care; D016032:Randomized Controlled Trials as Topic / Q000706:statistics & numerical data*; D012306:Risk; D013518:Surgery, Plastic
references.citation: Slim K. Limits of evidence-based surgery. World J Surg. 2005;29:606–609
references.citation: Karri V. Randomised clinical trials in plastic surgery: Survey of output and quality of reporting. J Plast Reconstr Aesthet Surg. 2006;59:787–796
references.citation: McCarthy JE, Chatterjee A, McKelvey TG, Jantzen EM, Kerrigan CL. A detailed analysis of level I evidence (randomized controlled trials and meta-analyses) in five plastic surgery journals to date: 1978 to 2009. Plast Reconstr Surg. 2010;126:1774–1778
references.citation: Becker A, Blümle A, Antes G, et al. Controlled trials in aesthetic plastic surgery: A 16-year analysis. Aesthetic Plast Surg. 2008;32:359–362
references.citation: Momeni A, Becker A, Antes G, et al. Evidence-based plastic surgery: Controlled trials in three plastic surgical journals (1990-2005). Ann Plast Surg. 2008;61:221–225
references.citation: Taghinia AH, Liao EC, May JW Jr. Randomized controlled trials in plastic surgery: A 20-year review of reporting standards, methodologic quality, and impact. Plast Reconstr Surg. 2008;122:1253–1263
references.citation: Veiga Filho J, Castro AA, Veiga DF, et al. Quality of reports of randomized clinical trials in plastic surgery. Plast Reconstr Surg. 2005;115:320–323
references.citation: Veiga DF, Veiga-Filho J, Pellizzon RF, Juliano Y, Ferreira LM. Evolution of reports of randomised clinical trials in plastic surgery. J Plast Reconstr Aesthet Surg. 2011;64:703–709
references.citation: Guyatt GH, Oxman AD, Vist G, et al. GRADE guidelines: 4. Rating the quality of evidence–study limitations (risk of bias). J Clin Epidemiol. 2011;64:407–415
references.citation: Slobogean GP, Sprague S, Bhandari M. The tactics of large randomized trials. J Bone Joint Surg Am. 2012;94(Suppl 1):19–23
references.citation: Stanley K. Design of randomized controlled trials. Circulation. 2007;115:1164–1169
references.citation: Rohrich RJ, Sullivan D. The role of the journal impact factor: Choosing the optimal source of peer-reviewed plastic surgery information. Plast Reconstr Surg. 2006;117:2495–2498
references.citation: Landis JR, Koch GG. The measurement of observer agreement for categorical data. Biometrics. 1977;33:159–174
references.citation: Akl EA, Sun X, Busse JW, et al. Specific instructions for estimating unclearly reported blinding status in randomized trials were reliable and valid. J Clin Epidemiol. 2012;65:262–267
references.citation: Akl EA, Briel M, You JJ, et al. Potential impact on estimated treatment effects of information lost to follow-up in randomised controlled trials (LOST-IT): Systematic review. BMJ. 2012;344:e2809
references.citation: Bala MM, Akl EA, Sun X, et al. Randomized trials published in higher vs. lower impact journals differ in design, conduct, and analysis. J Clin Epidemiol. 2013;66:286–295
references.citation: Sears ED, Burns PB, Chung KC. The outcomes of outcome studies in plastic surgery: A systematic review of 17 years of plastic surgery research. Plast Reconstr Surg. 2007;120:2059–2065
references.citation: Akl EA, Briel M, You JJ, et al. LOST to follow-up Information in Trials (LOST-IT): A protocol on the potential impact. Trials. 2009;10:40
references.citation: Gillgren P, Drzewiecki KT, Niin M, et al. 2-cm versus 4-cm surgical excision margins for primary cutaneous melanoma thicker than 2 mm: A randomised, multicentre trial. Lancet. 2011;378:1635–1642
references.citation: Saffle JR, Morris SE, Edelman L. Early tracheostomy does not improve outcome in burn patients. J Burn Care Rehabil. 2002;23:431–438
references.citation: Schulz KF, Grimes DA. Allocation concealment in randomised trials: Defending against deciphering. Lancet. 2002;359:614–618
references.citation: Pildal J, Hróbjartsson A, Jørgensen KJ, Hilden J, Altman DG, Gøtzsche PC. Impact of allocation concealment on conclusions drawn from meta-analyses of randomized trials. Int J Epidemiol. 2007;36:847–857
references.citation: Wood L, Egger M, Gluud LL, et al. Empirical evidence of bias in treatment effect estimates in controlled trials with different interventions and outcomes: Meta-epidemiological study. BMJ. 2008;336:601–605
references.citation: Devereaux PJ, Bhandari M, Clarke M, et al. Need for expertise based randomised controlled trials. BMJ. 2005;330:88
references.citation: Alshurafa M, Briel M, Akl EA, et al. Inconsistent definitions for intention-to-treat in relation to missing outcome data: Systematic review of the methods literature. PLoS One. 2012;7:e49163
references.citation: Farrokhyar F, Karanicolas PJ, Thoma A, et al. Randomized controlled trials of surgical interventions. Ann Surg. 2010;251:409–416
references.citation: Chung KC, Pusic AL. Patient-reported outcomes instruments. Outcomes measures in plastic surgery. Clin Plast Surg. 2013;40:xi–xii
references.citation: Guyatt GH, Oxman AD, Kunz R, et al. GRADE guidelines: 2. Framing the question and deciding on important outcomes. J Clin Epidemiol. 2011;64:395–400
references.citation: Agha RA, Camm CF, Edison E, Orgill DP. The methodological quality of randomized controlled trials in plastic surgery needs improvement: A systematic review. J Plast Reconstr Aesthet Surg. 2013;66:447–452
references.citation: Linde K, Ramirez G, Mulrow CD, Pauls A, Weidenhammer W, Melchart D. St John’s wort for depression–an overview and meta-analysis of randomised clinical trials. BMJ. 1996;313:253–258
references.citation: Hartling L, Ospina M, Liang Y, et al. Risk of bias versus quality assessment of randomised controlled trials: Cross sectional study. BMJ. 2009;339:b4012
references.citation: Gurusamy KS, Gluud C, Nikolova D, Davidson BR. Assessment of risk of bias in randomized clinical trials in surgery. Br J Surg. 2009;96:342–349
references.citation: Huemer GM, Bauer T, Gurunluoglu R, Sakho C, Oehlbauer M, Dunst KM. Analysis of publications in three plastic surgery journals for the year 2002. Plast Reconstr Surg. 2004;114:1147–1154
references.citation: Liberati A, Altman DG, Tetzlaff J, et al. The PRISMA statement for reporting systematic reviews and meta-analyses of studies that evaluate health care interventions: Explanation and elaboration. PLoS Med. 2009;6:e1000100
references.citation: Devereaux PJ, Choi PT, El-Dika S, et al. An observational study found that authors of randomized controlled trials frequently use concealment of randomization and blinding, despite the failure to report these methods. J Clin Epidemiol. 2004;57:1232–1236
references.citation: Hill CL, LaValley MP, Felson DT. Discrepancy between published report and actual conduct of randomized clinical trials. J Clin Epidemiol. 2002;55:783–786