An algorithm was developed to assign GRADE levels of evidence to comparisons within systematic reviews.
OBJECTIVE
One recommended use of the Grading of Recommendations Assessment, Development and Evaluation (GRADE) approach is supporting quality assessment of evidence of comparisons included within a Cochrane overview of reviews. Within our overview, reviewers found that current GRADE guidance was insufficient to make reliable and consistent judgments. To support our ratings, we developed an algorithm to grade quality of evidence using concrete rules.


METHODS
Using a pragmatic, exploratory approach, we explored the challenges of applying GRADE levels of evidence and developed an algorithm to applying GRADE levels of evidence in a consistent and transparent approach. Our methods involved application of algorithms and formulas to samples of reviews, expert panel discussion, and iterative refinement and revision.


RESULTS
The developed algorithm incorporated four key criteria: number of participants, risk of bias of trials, heterogeneity, and methodological quality of the review. A formula for applying GRADE level of evidence from the number of downgrades assigned by the algorithm was agreed.


CONCLUSIONS
Our algorithm which assigns GRADE levels of evidence using a set of concrete rules was successfully applied within our Cochrane overview. We propose that this methodological approach has implications for assessment of quality of evidence within future evidence syntheses.
Nursing, Midwifery and Allied Health Professions Research Unit, Glasgow Caledonian University, Buchanan House, Cowcaddens Road, Glasgow G4 0BA, UK. Electronic address: alex.pollock@gcu.ac.uk.
Alex
A
Pollock
Nursing, Midwifery and Allied Health Professions Research Unit, Glasgow Caledonian University, Buchanan House, Cowcaddens Road, Glasgow G4 0BA, UK.
Sybil E
SE
Farmer
Nursing, Midwifery and Allied Health Professions Research Unit, Glasgow Caledonian University, Buchanan House, Cowcaddens Road, Glasgow G4 0BA, UK.
Marian C
MC
Brady
Academic Section of Geriatric Medicine, Level 2, New Lister Building, Glasgow Royal Infirmary, Glasgow G31 2ER, UK.
Peter
P
Langhorne
Centre for Clinical Brain Sciences, Royal Infirmary of Edinburgh, Little France Crescent, Edinburgh EH16 5NN, UK.
Gillian E
GE
Mead
Wissenschaftliches Institut, Private Europäische Medizinische Akademie der Klinik Bavaria in Kreischa GmbH, An der Wolfsschlucht 1-2, 01731 Kreischa, Germany.
Jan
J
Mehrholz
Institute for Applied Health Research and the School of Health and Life Sciences, Glasgow Caledonian University, Cowcaddens Road, Glasgow G4 0BA, UK.
Frederike
F
van Wijck
Pain Research and Nuffield Department of Clinical Neurosciences (Nuffield Division of Anaesthetics), University of Oxford, Churchill Hospital, Oxford OX3 7LE, UK.
Philip J
PJ
Wiffen
D000465:Algorithms; D002986:Clinical Trials as Topic; D055317:Evidence-Based Practice; D006801:Humans; D012107:Research Design; D012196:Review Literature as Topic
Algorithm; GRADE; Methodology; Overview; Quality of evidence; Review