Methodological shortcomings predicted lower harm estimates in one of two sets of studies of clinical interventions.
OBJECTIVE
High quality harms data are necessary to appropriately assess the balance between benefits and harms of interventions. Little is known, however, about whether perceived methodological shortcomings are associated with lower estimates of harms.


METHODS
Studies reporting harms associated with carotid endarterectomy (CEA) and rofecoxib were identified using published systematic reviews. A standardized abstraction form, including eight predefined criteria for assessing the quality of harms reporting, was used to extract data. Univariate and multivariate analyses were performed to empirically evaluate the association between quality criteria and estimates of harms.


RESULTS
In 111 studies of CEA, meeting five of the eight-quality criteria was associated with significantly higher adverse event rates. A quality-rating instrument with four criteria predicted adverse events (5.7% in studies rated "adequate," compared to 3.9% in studies rated "inadequate" [P=0.0003]). In multivariate analyses, the quality-rating assignment remained significant when controlling for other clinical and study-related variables. Different quality criteria, however, predicted estimates of risk for myocardial infarction in 16 trials of rofecoxib.


CONCLUSIONS
The presence of methodological shortcomings can predict lower estimates of serious harms. Clinicians and researchers should consider methodological shortcomings when evaluating estimates of harms associated with clinical interventions.
Chou
Roger
R
Oregon Evidence-Based Practice Center, Portland, OR, USA. chour@ohsu.edu
Fu
Rongwei
R
Carson
Susan
S
Saha
Somnath
S
Helfand
Mark
M
D035843:Biomedical Research / Q000379:methods / Q000592:standards; D052246:Cyclooxygenase 2 Inhibitors / Q000009:adverse effects*; D016894:Endarterectomy, Carotid / Q000009:adverse effects*; D006801:Humans; D007783:Lactones / Q000009:adverse effects*; D012107:Research Design / Q000592:standards; D018570:Risk Assessment / Q000379:methods; D020521:Stroke / Q000209:etiology; D013450:Sulfones / Q000009:adverse effects*
D052246:Cyclooxygenase 2 Inhibitors; D007783:Lactones; D013450:Sulfones; C116926:rofecoxib