Is quality control of Cochrane reviews in controversial areas sufficient?
OBJECTIVE
Some conclusions from the Cochrane Database of Systematic Reviews (CDSR) seem volatile in areas of controversy and have recently changed conclusions. With this perspective in mind we decided to test the validity and sensitivity of negative conclusions in a systematic review of low-level laser therapy (LLLT) for osteoarthritis (OA) from the Cochrane Library was valid and robust.


METHODS
None as this was a systematic review of literature.


METHODS
Validity was tested against a 9-item checklist for systematic reviews. Review selections were analyzed for possible discrepancies between trial and review reports and omissions of relevant trials and data. Data from discrepancies and omissions were then entered into sensitivity and subgroup analyses.


METHODS
Continuous and dichotomous data for pain


METHODS
Patients with osteoarthritis.


METHODS
LLLT.


RESULTS
Only clinicians who had performed LLLT trials, and had negative results were invited into the review group. The review was oblivious to findings published after 1993 about physiologic mechanisms and dose response patterns for LLLT. We found 18 questionable selections that favored a negative review conclusion in 17 of 18 cases. These were largely omissions of relevant positive data and selective inclusion of negative data from trials with small, ineffective doses. When existing and omitted data from relevant trials were combined, results changed from negative to significantly positive for continuous and categorical data. Subgrouping trials by location and recommended doses taken from current guidelines, revealed a highly significant effect of LLLT for treating knee OA.


CONCLUSIONS
The results of this CDSR was not robust and seems to be colored by questionable selections or omissions. For alternative pain therapies, it should be considered if lack of expertise on therapy mechanisms, or conflicts of interests with competing pain drug manufacturers, may bias conclusions. Review groups should recruit a balanced mix of current views and expertise and expand the use of sensitivity analyses to improve quality of CDSRs in areas of controversy.
Bjordal
Jan Magnus
JM
Section of Physiotherapy Science, University of Bergen and Institute of Physiotherapy, Bergen University College, Bergen, Norway. jan.bjordal@hib.no
Lopes-Martins
Rodrigo Alvaro Brandao
RA
Klovning
Atle
A
D026683:Advisory Committees; D016206:Databases, Bibliographic / Q000592:standards*; D019317:Evidence-Based Medicine / Q000592:standards*; D006801:Humans; D016247:Information Storage and Retrieval; D028022:Low-Level Light Therapy; D010003:Osteoarthritis / Q000532:radiotherapy; D017712:Peer Review, Research; D011786:Quality Control; D016032:Randomized Controlled Trials as Topic