Tools for assessing quality and susceptibility to bias in observational studies in epidemiology: a systematic review and annotated bibliography.
BACKGROUND
Assessing quality and susceptibility to bias is essential when interpreting primary research and conducting systematic reviews and meta-analyses. Tools for assessing quality in clinical trials are well-described but much less attention has been given to similar tools for observational epidemiological studies.


METHODS
Tools were identified from a search of three electronic databases, bibliographies and an Internet search using Google. Two reviewers extracted data using a pre-piloted extraction form and strict inclusion criteria. Tool content was evaluated for domains potentially related to bias and was informed by the STROBE guidelines for reporting observational epidemiological studies.


RESULTS
A total of 86 tools were reviewed, comprising 41 simple checklists, 12 checklists with additional summary judgements and 33 scales. The number of items ranged from 3 to 36 (mean 13.7). One-third of tools were designed for single use in a specific review and one-third for critical appraisal. Half of the tools provided development details, although most were proposed for future use in other contexts. Most tools included items for selection methods (92%), measurement of study variables (86%), design-specific sources of bias (86%), control of confounding (78%) and use of statistics (78%); only 4% addressed conflict of interest. The distribution and weighting of domains across tools was variable and inconsistent.


CONCLUSIONS
A number of useful assessment tools have been identified by this report. Tools should be rigorously developed, evidence-based, valid, reliable and easy to use. There is a need to agree on critical elements for assessing susceptibility to bias in observational epidemiology and to develop appropriate evaluation tools.
Sanderson
Simon
S
Primary Care Genetics, General Practice and Primary Care Research Unit, University of Cambridge and Public Health Genetics Unit, Strangeways Research Labs, Worts Causeway, Cambridge, UK. simon.sanderson@sri.cam.ac.uk
Tatt
Iain D
ID
Higgins
Julian P T
JP
D015982:Bias; D016021:Epidemiologic Studies; D019317:Evidence-Based Medicine / Q000379:methods / Q000592:standards; D006801:Humans; D019370:Observation; D012107:Research Design