There were large discrepancies in risk of bias tool judgments when a randomized controlled trial appeared in more than one systematic review.
To assess the consistency in risk of bias (RoB) judgments across Cochrane reviews for studies appearing in more than one Cochrane review in the field of subfertility.



We retrieved any study that had been used more than once in systematic reviews present on the Cochrane Database of Systematic Reviews in the area of subfertility. We then retrieved the recorded RoB assessments for these studies and looked at the consistency of judgments made between different authoring teams on the same trials.



From the 156 bias judgments that were completed by at least two separate groups of authors, 45% of these judgments differed. For the domains of random sequence generation and incomplete outcome data, there was reasonably high level of agreement (71% and 79%, respectively). However, for the domain of blinding, agreement was reached in only 35% of cases.



This assessment of how consistently the RoB is being applied in Cochrane reviews has shown that, especially in some domains, there are large discrepancies in how RoB is being evaluated. Further work needs to be undertaken to improve the application of this tool.
Department Obstetrics and Gynaecology, Auckland University, Private Bag 92019, Auckland 1142, New Zealand. Electronic address: v.jordan@auckland.ac.nz.
Vanessa M B
VM
Jordan
Department Obstetrics and Gynaecology, Auckland University, Private Bag 92019, Auckland 1142, New Zealand.
Sarah F
SF
Lensen
Department Obstetrics and Gynaecology, Auckland University, Private Bag 92019, Auckland 1142, New Zealand.
Cynthia M
CM
Farquhar
D015982:Bias; D006801:Humans; D007246:Infertility / Q000453:epidemiology*; D007600:Judgment; D016032:Randomized Controlled Trials as Topic / Q000706:statistics & numerical data*; D012196:Review Literature as Topic
Cochrane; Internal validity; Meta-analysis; Quality; Reliability; Risk of bias; Systematic reviews