title: A newly developed tool for classifying study designs in systematic reviews of interventions and exposures showed substantial reliability and validity.
abstract: OBJECTIVE
  To develop a study Design Algorithm for Medical Literature on Intervention (DAMI) and test its interrater reliability, construct validity, and ease of use.
  
  
  METHODS
  We developed and then revised the DAMI to include detailed instructions. To test the DAMI's reliability, we used a purposive sample of 134 primary, mainly nonrandomized studies. We then compared the study designs as classified by the original authors and through the DAMI. Unweighted kappa statistics were computed to test interrater reliability and construct validity based on the level of agreement between the original and DAMI classifications. Assessment time was also recorded to evaluate ease of use.
  
  
  RESULTS
  The DAMI includes 13 study designs, including experimental and observational studies of interventions and exposure. Both the interrater reliability (unweighted kappa = 0.67; 95% CI [0.64-0.75]) and construct validity (unweighted kappa = 0.63, 95% CI [0.52-0.67]) were substantial. Mean classification time using the DAMI was 4.08 ± 2.44 minutes (range, 0.51-10.92).
  
  
  CONCLUSIONS
  The DAMI showed substantial interrater reliability and construct validity. Furthermore, given its ease of use, it could be used to accurately classify medical literature for systematic reviews of interventions although minimizing disagreement between authors of such reviews.
authors.affiliation: Department of Nursing, College of Medicine, Chosun University, Gwangju, Republic of Korea.
authors.forename: Hyun-Ju
authors.initials: HJ
authors.lastname: Seo
authors.affiliation: Department of Family Medicine, College of Medicine, Hallym University, Seoul, Republic of Korea. Electronic address: hallymfm@gmail.com.
authors.forename: Soo Young
authors.initials: SY
authors.lastname: Kim
authors.affiliation: Department of Oriental Gynecology, CHA Bundang Medical Center, CHA University, Seongnam, Republic of Korea.
authors.forename: Yoon Jae
authors.initials: YJ
authors.lastname: Lee
authors.affiliation: Department of Preventive Medicine, College of Korean Medicine, Kyung Hee University, Seoul, Republic of Korea.
authors.forename: Bo-Hyoung
authors.initials: BH
authors.lastname: Jang
authors.affiliation: Department of Health Care Management and Policy, School of Public Health, Seoul National University, Seoul, Republic of Korea.
authors.forename: Ji-Eun
authors.initials: JE
authors.lastname: Park
authors.affiliation: Department of Pulmonology, Ajou University Hospital, Suwon, Republic of Korea.
authors.forename: Seung-Soo
authors.initials: SS
authors.lastname: Sheen
authors.affiliation: College of Medicine, Seoul National University, Seoul, Republic of Korea.
authors.forename: Seo Kyung
authors.initials: SK
authors.lastname: Hahn
mesh_terms: D000465:Algorithms; D032921:Consensus; D019317:Evidence-Based Medicine / Q000145:classification*; D006801:Humans; D017712:Peer Review, Research / Q000592:standards*; D011786:Quality Control; D015203:Reproducibility of Results; D012107:Research Design / Q000592:standards*; D012196:Review Literature as Topic
keywords: Algorithm; Intervention studies; Reliability; Research design; Systematic review; Validity