The effect of collapsing multinomial data when assessing agreement.
BACKGROUND
In epidemiological studies researchers often depend on proxies to obtain information when primary subjects are unavailable. However, relatively few studies have performed formal statistical inference to assess agreement among proxy informants and primary study subjects. In this paper, we consider inference procedures for studies of interobserver agreement characterized by two raters and three or more outcome categories. Of particular interest is the consequence of dichotomizing such data on the expected confidence interval width for the kappa coefficient. The effect of dichotomization on sample size requirements for testing hypotheses concerning kappa is also evaluated.


METHODS
Simulation studies were used to compare coverage levels and widths for constructing confidence intervals. Sample size requirements were compared for multinomial and dichotomous data. We illustrate our results using a published data set on drinking habits that assesses agreement among primary and proxy respondents.


RESULTS
Our results show that when multinomial data are treated as dichotomous, not only do the expected confidence interval widths become greater, but the penalty in terms of larger sample size requirements for hypothesis testing can be severe.


CONCLUSIONS
We conclude that there are clear advantages in preserving multinomial data on the original scale rather than collapsing the data into a binary trait.
Bartfay
E
E
Department of Epidemiology and Biostatistics, The University of Western Ontario, London, Ontario, Canada. emma.bartfay@krcc.on.ca
Donner
A
A
D000428:Alcohol Drinking; D016001:Confidence Intervals; D006801:Humans; D009203:Myocardial Infarction / Q000453:epidemiology; D018401:Sample Size; D013223:Statistics as Topic