title: Inconsistency in the items included in tools used in general health research and physical therapy to evaluate the methodological quality of randomized controlled trials: a descriptive analysis.
abstract: BACKGROUND
  Assessing the risk of bias of randomized controlled trials (RCTs) is crucial to understand how biases affect treatment effect estimates. A number of tools have been developed to evaluate risk of bias of RCTs; however, it is unknown how these tools compare to each other in the items included. The main objective of this study was to describe which individual items are included in RCT quality tools used in general health and physical therapy (PT) research, and how these items compare to those of the Cochrane Risk of Bias (RoB) tool.
  
  
  METHODS
  We used comprehensive literature searches and a systematic approach to identify tools that evaluated the methodological quality or risk of bias of RCTs in general health and PT research. We extracted individual items from all quality tools. We calculated the frequency of quality items used across tools and compared them to those in the RoB tool. Comparisons were made between general health and PT quality tools using Chi-squared tests.
  
  
  RESULTS
  In addition to the RoB tool, 26 quality tools were identified, with 19 being used in general health and seven in PT research. The total number of quality items included in general health research tools was 130, compared with 48 items across PT tools and seven items in the RoB tool. The most frequently included items in general health research tools (14/19, 74%) were inclusion and exclusion criteria, and appropriate statistical analysis. In contrast, the most frequent items included in PT tools (86%, 6/7) were: baseline comparability, blinding of investigator/assessor, and use of intention-to-treat analysis. Key items of the RoB tool (sequence generation and allocation concealment) were included in 71% (5/7) of PT tools, and 63% (12/19) and 37% (7/19) of general health research tools, respectively.
  
  
  CONCLUSIONS
  There is extensive item variation across tools that evaluate the risk of bias of RCTs in health research. Results call for an in-depth analysis of items that should be used to assess risk of bias of RCTs. Further empirical evidence on the use of individual items and the psychometric properties of risk of bias tools is needed.
authors.affiliation: Postdoctoral Fellow, CLEAR (Connecting Leadership and Research) Outcomes Research Program, University of Alberta, 5-115A Edmonton Clinic Health Academy (ECHA), 11405 - 87 Avenue, Edmonton, Alberta T6G 1C9 Canada. susanarmijo@gmail.com.
authors.forename: Susan
authors.initials: S
authors.lastname: Armijo-Olivo
authors.forename: Jorge
authors.initials: J
authors.lastname: Fuentes
authors.forename: Maria
authors.initials: M
authors.lastname: Ospina
authors.forename: Humam
authors.initials: H
authors.lastname: Saltaji
authors.forename: Lisa
authors.initials: L
authors.lastname: Hartling
mesh_terms: D015982:Bias; D006262:Health; D006801:Humans; D015233:Models, Statistical; D026741:Physical Therapy Modalities; D011785:Quality Assurance, Health Care / Q000379:methods; D016032:Randomized Controlled Trials as Topic; D012107:Research Design; D012306:Risk
references.citation: J Clin Epidemiol. 2006 Dec;59(12):1239-48
references.pmid: 17098566
references.citation: JAMA. 1999 Sep 15;282(11):1054-60
references.pmid: 10493204
references.citation: Epidemiology. 1996 Sep;7(5):561
references.pmid: 8862998
references.citation: JAMA. 2002 Jun 5;287(21):2801-4
references.pmid: 12038917
references.citation: J Adv Nurs. 1997 Jun;25(6):1262-8
references.pmid: 9181425
references.citation: BMJ. 2011 Oct 18;343:d5928
references.pmid: 22008217
references.citation: Control Clin Trials. 1981 May;2(1):31-49
references.pmid: 7261638
references.citation: J Clin Psychopharmacol. 2007 Aug;27(4):352-6
references.pmid: 17632218
references.citation: Aust J Physiother. 2002;48(1):43-9
references.pmid: 11869164
references.citation: J Orthop Sports Phys Ther. 2003 Jan;33(1):4-20
references.pmid: 12570282
references.citation: Pediatrics. 1989 Nov;84(5):815-27
references.pmid: 2797977
references.citation: J Clin Epidemiol. 2001 Jul;54(7):651-4
references.pmid: 11438404
references.citation: J Eval Clin Pract. 2012 Feb;18(1):12-8
references.pmid: 20698919
references.citation: BMC Complement Altern Med. 2009 Apr 23;9:11
references.pmid: 19389240
references.citation: Acta Radiol Diagn (Stockh). 1984;25(1):55-8
references.pmid: 6731012
references.citation: Ann Intern Med. 1990 Aug 15;113(4):299-307
references.pmid: 2142869
references.citation: Med Care. 1995 Jul;33(7):687-99
references.pmid: 7596208
references.citation: Control Clin Trials. 1995 Feb;16(1):62-73
references.pmid: 7743790
references.citation: BMC Med Res Methodol. 2004 Sep 16;4:22
references.pmid: 15369598
references.citation: Arch Phys Med Rehabil. 2002 Dec;83(12):1745-52
references.pmid: 12474181
references.citation: Ann Intern Med. 2006 Mar 21;144(6):427-37
references.pmid: 16549855
references.citation: Eur J Orthod. 1999 Oct;21(5):503-15
references.pmid: 10565091
references.citation: Control Clin Trials. 1996 Feb;17(1):1-12
references.pmid: 8721797
references.citation: J Epidemiol Community Health. 1998 Jun;52(6):377-84
references.pmid: 9764259
references.citation: Spine (Phila Pa 1976). 2012 Mar 15;37(6):515-22
references.pmid: 21673624
references.citation: Int J Epidemiol. 2007 Aug;36(4):847-57
references.pmid: 17517809
references.citation: J Clin Epidemiol. 1992 Mar;45(3):255-65
references.pmid: 1569422
references.citation: BMJ. 2001 Jul 7;323(7303):42-6
references.pmid: 11440947
references.citation: Int J Epidemiol. 2007 Jun;36(3):666-76
references.pmid: 17470488
references.citation: J Clin Epidemiol. 2011 Feb;64(2):136-44
references.pmid: 20705426
references.citation: JAMA. 1995 Feb 1;273(5):408-12
references.pmid: 7823387
references.citation: J Clin Epidemiol. 1998 Dec;51(12):1235-41
references.pmid: 10086815
references.citation: Spine (Phila Pa 1976). 1997 Oct 15;22(20):2323-30
references.pmid: 9355211
references.citation: J Epidemiol Community Health. 2004 Aug;58(8):635-41
references.pmid: 15252064
references.citation: BMJ. 2004 Jan 3;328(7430):22-4
references.pmid: 14703540
references.citation: J Clin Epidemiol. 2012 Feb;65(2):163-78
references.pmid: 21959223
references.citation: JAMA. 1994 Jul 13;272(2):101-4
references.pmid: 8015115
references.citation: Man Ther. 2000 Nov;5(4):223-6
references.pmid: 11052901
references.citation: PLoS One. 2011 Feb 24;6(2):e17242
references.pmid: 21390219
references.citation: J Clin Epidemiol. 2007 Jan;60(1):34-42
references.pmid: 17161752
references.citation: Radiology. 2000 Oct;217(1):69-74
references.pmid: 11012425
references.citation: J Clin Epidemiol. 2005 Dec;58(12):1233-40
references.pmid: 16291467
references.citation: J Chronic Dis. 1979;32(1-2):51-63
references.pmid: 447779
references.citation: Ann Intern Med. 2004 Feb 3;140(3):189-202
references.pmid: 14757617
references.citation: BMJ. 2008 Mar 15;336(7644):601-5
references.pmid: 18316340
references.citation: Pain. 2005 Oct;117(3):314-325
references.pmid: 16154704
references.citation: Compr Psychiatry. 2010 May-Jun;51(3):319-24
references.pmid: 20399343
references.citation: Am J Psychiatry. 2011 Jan;168(1):19-28
references.pmid: 20843868
references.citation: Spine (Phila Pa 1976). 2003 Jun 15;28(12):1290-9
references.pmid: 12811274
references.citation: Lancet. 1998 Aug 22;352(9128):609-13
references.pmid: 9746022
references.citation: J Clin Epidemiol. 2013 Sep;66(9):973-81
references.pmid: 22981249
references.citation: BMJ. 2009 Oct 19;339:b4012
references.pmid: 19841007
references.citation: J Clin Epidemiol. 2006 Dec;59(12):1249-56
references.pmid: 17098567
references.citation: Pain. 2000 May;86(1-2):119-32
references.pmid: 10779669
references.citation: Phys Ther. 2008 Feb;88(2):156-75
references.pmid: 18073267