title: Assessment on homogeneity tests for kappa statistics under equal prevalence across studies in reliability.
abstract: In this paper, we assess the performance of homogeneity tests for two or more kappa statistics when prevalence rates across reliability studies are assumed to be equal. The likelihood score method and the chi-square goodness-of-fit (GOF) test provide type 1 error rates that are satisfactorily close to the nominal level, but a Fleiss-like test is not satisfactory for small or moderate sample sizes. Simulations show that the score test is more powerful than the chi-square GOF test and the approximate sample size required for a specific power of the former is substantially smaller than the latter. In addition, the score test is robust to deviations from the equal prevalence assumption, while the GOF test is highly sensitive and it may give a grossly misleading type 1 error rate when the assumption of equal prevalence is violated. We conclude that the homogeneity score test is the preferred method.
authors.affiliation: Biostatistics Branch, Division of Cancer Epidemiology and Genetics, National Cancer Institute, National Institutes of Health, Department of Health & Human Services, Executive Plaza South, Rockville, MD 20892-7240, USA. namj@mail.nih.gov
authors.forename: Jun-mo
authors.initials: JM
authors.lastname: Nam
mesh_terms: D000428:Alcohol Drinking; D016009:Chi-Square Distribution; D003198:Computer Simulation; D003627:Data Interpretation, Statistical; D005260:Female; D006801:Humans; D016013:Likelihood Functions; D008297:Male; D018401:Sample Size; D016008:Statistical Distributions; D014429:Twins, Dizygotic; D014430:Twins, Monozygotic