Development quality criteria to evaluate nontherapeutic studies of incidence, prevalence, or risk factors of chronic diseases: pilot study of new checklists.
OBJECTIVE
To develop two checklists for the quality of observational studies of incidence or risk factors of diseases.


METHODS
Initial development of the checklists was based on a systematic literature review. The checklists were refined after pilot trials of validity and reliability were conducted by seven experts, who tested the checklists on 10 articles.


RESULTS
The checklist for studies of incidence or prevalence of chronic disease had six criteria for external validity and five for internal validity. The checklist for risk factor studies had six criteria for external validity, 13 criteria for internal validity, and two aspects of causality. A Microsoft Access database produced automated standardized reports about external and internal validities. Pilot testing demonstrated face and content validities and discrimination of reporting vs. methodological qualities. Interrater agreement was poor. The experts suggested future reliability testing of the checklists in systematic reviews with preplanned protocols, a priori consensus about research-specific quality criteria, and training of the reviewers.


CONCLUSIONS
We propose transparent and standardized quality assessment criteria of observational studies using the developed checklists. Future testing of the checklists in systematic reviews is necessary to develop reliable tools that can be used with confidence.
Division of Health Policy and Management, University of Minnesota School of Public Health, Minneapolis, MN, USA.
Tatyana A
TA
Shamliyan
Robert L
RL
Kane
Mohammed T
MT
Ansari
Gowri
G
Raman
Nancy D
ND
Berkman
Mark
M
Grant
Gail
G
Janes
Margaret
M
Maglione
David
D
Moher
Mona
M
Nasser
Karen A
KA
Robinson
Jodi B
JB
Segal
Sophia
S
Tsouros
D015982:Bias; D057189:Checklist / Q000379:methods / Q000592:standards*; D002908:Chronic Disease / Q000453:epidemiology*; D005260:Female; D006801:Humans; D015994:Incidence; D008297:Male; D010865:Pilot Projects; D015995:Prevalence; D011786:Quality Control; D015203:Reproducibility of Results; D012107:Research Design / Q000592:standards; D012307:Risk Factors