title: Improving the reliability of clinical practice guideline appraisals: effects of the Korean AGREE II scoring guide.
abstract: The Korean translated Appraisal of Guidelines for Research and Evaluation II (Korean AGREE II) instrument was distributed into Korean medical societies in 2011. However, inter-rater disagreement issues still exist. The Korean AGREE II scoring guide was therefore developed to reduce inter-rater differences. This study examines the effects of the Korean AGREE II scoring guide to reduce inter-rater differences. Appraisers were randomly assigned to two groups (Scoring Guide group and Non-Scoring Guide group). The Korean AGREE II instrument was provided to both groups. However, the scoring guide was offered to Scoring Guide group only. Total 14 appraisers were participated and each guideline was assessed by 8 appraisers. To evaluate the reliability of the Korean AGREE II scoring guide, correlation of scores among appraisers and domain-specific intra-class correlation (ICC) were compared. Most scores of two groups were comparable. Scoring Guide group showed higher reliability at all guidelines. They showed higher correlation among appraisers and higher ICC values at almost all domains. The scoring guide reduces the inter-rater disagreement and improves the overall reliability of the Korean-AGREE II instrument.
authors.affiliation: Department of Preventive Medicine, Kangwon National University Hospital, Chuncheon, Korea.
authors.forename: Moo-Kyung
authors.identifier: 0000-0002-2011-5708
authors.initials: MK
authors.lastname: Oh
authors.affiliation: Department of Preventive Medicine, Kangwon National University Hospital, Chuncheon, Korea. ; Department of Health Management and Policy, Kangwon National University School of Medicine, Chuncheon, Korea. ; The Executive Committee for Clinical Practice Guideline, The Korean Academy of Medical Sciences, Seoul, Korea.
authors.forename: Heuisug
authors.identifier: 0000-0003-0245-3583
authors.initials: H
authors.lastname: Jo
authors.affiliation: The Executive Committee for Clinical Practice Guideline, The Korean Academy of Medical Sciences, Seoul, Korea. ; Department of Laboratory Medicine and Genetics, Soonchunhyang University Bucheon Hospital, Bucheon, Korea. ; Department of Laboratory Medicine and Genetics, Soonchunhyang University College of Medicine, Cheonan, Korea.
authors.forename: You Kyoung
authors.identifier: 0000-0003-1835-2007
authors.initials: YK
authors.lastname: Lee
mesh_terms: D044466:Asian People; D019317:Evidence-Based Medicine; D006801:Humans; D017410:Practice Guidelines as Topic; D015203:Reproducibility of Results; D056910:Republic of Korea; D012955:Societies, Medical; D014175:Translating
keywords: Clinical Practice Guideline; Observer Variation; Reproducibility of Results