title: Quality assessment of observational studies in a drug-safety systematic review, comparison of two tools: the Newcastle-Ottawa Scale and the RTI item bank.
abstract: BACKGROUND
  The study objective was to compare the Newcastle-Ottawa Scale (NOS) and the RTI item bank (RTI-IB) and estimate interrater agreement using the RTI-IB within a systematic review on the cardiovascular safety of glucose-lowering drugs.
  
  
  METHODS
  We tailored both tools and added four questions to the RTI-IB. Two reviewers assessed the quality of the 44 included studies with both tools, (independently for the RTI-IB) and agreed on which responses conveyed low, unclear, or high risk of bias. For each question in the RTI-IB (n=31), the observed interrater agreement was calculated as the percentage of studies given the same bias assessment by both reviewers; chance-adjusted interrater agreement was estimated with the first-order agreement coefficient (AC1) statistic.
  
  
  RESULTS
  The NOS required less tailoring and was easier to use than the RTI-IB, but the RTI-IB produced a more thorough assessment. The RTI-IB includes most of the domains measured in the NOS. Median observed interrater agreement for the RTI-IB was 75% (25th percentile [p25] =61%; p75 =89%); median AC1 statistic was 0.64 (p25 =0.51; p75 =0.86).
  
  
  CONCLUSIONS
  The RTI-IB facilitates a more complete quality assessment than the NOS but is more burdensome. The observed agreement and AC1 statistic in this study were higher than those reported by the RTI-IB's developers.
authors.affiliation: RTI Health Solutions, Barcelona, Spain.
authors.forename: Andrea V
authors.initials: AV
authors.lastname: Margulis
authors.affiliation: RTI Health Solutions, Barcelona, Spain.
authors.forename: Manel
authors.initials: M
authors.lastname: Pladevall
authors.affiliation: RTI Health Solutions, Barcelona, Spain.
authors.forename: Nuria
authors.initials: N
authors.lastname: Riera-Guardia
authors.affiliation: RTI Health Solutions, Barcelona, Spain.
authors.forename: Cristina
authors.initials: C
authors.lastname: Varas-Lorenzo
authors.affiliation: Drug Safety Research Unit, Southampton, UK ; Associate Department of the School of Pharmacy and Biomedical Sciences, University of Portsmouth, Portsmouth, UK.
authors.forename: Lorna
authors.initials: L
authors.lastname: Hazell
authors.affiliation: RTI International, Research Triangle Park, NC, USA.
authors.forename: Nancy D
authors.initials: ND
authors.lastname: Berkman
authors.affiliation: RTI International, Research Triangle Park, NC, USA.
authors.forename: Meera
authors.initials: M
authors.lastname: Viswanathan
authors.affiliation: RTI Health Solutions, Barcelona, Spain.
authors.forename: Susana
authors.initials: S
authors.lastname: Perez-Gutthann
keywords: AC1; meta-analysis; quality assessment; systematic review