The sensitivity of review results to methods used to appraise and incorporate trial quality into data synthesis.
METHODS
Systematic review.


OBJECTIVE
To determine whether results and conclusions on the effectiveness of exercise for workers with neck pain vary with the Cochrane Back Review Group Guidelines and best-evidence synthesis review methods. To identify methodologic weaknesses associated with these review methods that may impact on the validity of their results.


BACKGROUND
The Cochrane Back Review Group Guidelines and best-evidence synthesis have different approaches to appraising trial quality and incorporating quality into data synthesis. The impact of different review methods on the reproducibility and validity of review results is unknown.


RESULTS
Systematic search of Medline, Embase, CINAHL, and Cochrane databases, without language restrictions. Twelve trials were selected. Two review methods were used to appraise trial quality and to incorporate quality into data synthesis. As recommended by the Cochrane Back Review Group Guidelines, trials were assigned quality scores using a scale. Results of all 12 trials were stratified into levels of evidence according to their scores. Based on these results, no treatment recommendation could be formulated. Best-evidence synthesis critically appraised methodology; trials were accepted on the strength of their scientific merit or rejected due to risk of bias. According to the 4 trials accepted for best-evidence synthesis, workers should be activated with exercise given its beneficial effect on patient-perceived recovery. Both the Cochrane Back Review Group Guidelines and best-evidence synthesis reviews were found to have weaknesses associated with their methods.


CONCLUSIONS
Review results and conclusions are sensitive to methods for appraising trial quality and incorporating quality into data synthesis when the evidence consists largely of low-quality trials. Both the Cochrane Back Review Group Guidelines and best-evidence synthesis methods were found to have strengths and methodologic weaknesses that healthcare decision-makers should be aware of when interpreting systematic reviews.
van der Velde
Gabrielle
G
Institute for Work and Health, Toronto, Ontario, Canada. gvdvelde@iwh.on.ca
van Tulder
Maurits
M
Côté
Pierre
P
Hogg-Johnson
Sheilah
S
Aker
Peter
P
Cassidy
J David
JD
Carragee
Eugene
E
Carroll
Linda
L
Guzman
Jaime
J
Haldeman
Scott
S
Holm
Lena
L
Hurwitz
Eric
E
Nordin
Margareta
M
Peloso
Paul
P
D002986:Clinical Trials as Topic / Q000592:standards*; D003625:Data Collection / Q000379:methods*; D003657:Decision Making; D019317:Evidence-Based Medicine; D005081:Exercise Therapy; D017408:Guidelines as Topic; D006801:Humans; D015201:Meta-Analysis as Topic; D019547:Neck Pain / Q000628:therapy*; D016272:Occupational Health; D011786:Quality Control; D015203:Reproducibility of Results; D012680:Sensitivity and Specificity; D016896:Treatment Outcome