title: Methodological shortcomings predicted lower harm estimates in one of two sets of studies of clinical interventions.
abstract: OBJECTIVE
  High quality harms data are necessary to appropriately assess the balance between benefits and harms of interventions. Little is known, however, about whether perceived methodological shortcomings are associated with lower estimates of harms.
  
  
  METHODS
  Studies reporting harms associated with carotid endarterectomy (CEA) and rofecoxib were identified using published systematic reviews. A standardized abstraction form, including eight predefined criteria for assessing the quality of harms reporting, was used to extract data. Univariate and multivariate analyses were performed to empirically evaluate the association between quality criteria and estimates of harms.
  
  
  RESULTS
  In 111 studies of CEA, meeting five of the eight-quality criteria was associated with significantly higher adverse event rates. A quality-rating instrument with four criteria predicted adverse events (5.7% in studies rated "adequate," compared to 3.9% in studies rated "inadequate" [P=0.0003]). In multivariate analyses, the quality-rating assignment remained significant when controlling for other clinical and study-related variables. Different quality criteria, however, predicted estimates of risk for myocardial infarction in 16 trials of rofecoxib.
  
  
  CONCLUSIONS
  The presence of methodological shortcomings can predict lower estimates of serious harms. Clinicians and researchers should consider methodological shortcomings when evaluating estimates of harms associated with clinical interventions.
authors.affiliation: Oregon Evidence-Based Practice Center, Portland, OR, USA. chour@ohsu.edu
authors.forename: Roger
authors.initials: R
authors.lastname: Chou
authors.forename: Rongwei
authors.initials: R
authors.lastname: Fu
authors.forename: Susan
authors.initials: S
authors.lastname: Carson
authors.forename: Somnath
authors.initials: S
authors.lastname: Saha
authors.forename: Mark
authors.initials: M
authors.lastname: Helfand
mesh_terms: D035843:Biomedical Research / Q000379:methods / Q000592:standards; D052246:Cyclooxygenase 2 Inhibitors / Q000009:adverse effects*; D016894:Endarterectomy, Carotid / Q000009:adverse effects*; D006801:Humans; D007783:Lactones / Q000009:adverse effects*; D012107:Research Design / Q000592:standards; D018570:Risk Assessment / Q000379:methods; D020521:Stroke / Q000209:etiology; D013450:Sulfones / Q000009:adverse effects*
chemical_list: D052246:Cyclooxygenase 2 Inhibitors; D007783:Lactones; D013450:Sulfones; C116926:rofecoxib