Pisces did not have increased heart failure: data-driven comparisons of binary proportions between levels of a categorical variable can result in incorrect statistical significance levels.
OBJECTIVE
We examined the impact on statistical inference when a chi(2) test is used to compare the proportion of successes in the level of a categorical variable that has the highest observed proportion of successes with the proportion of successes in all other levels of the categorical variable combined.


METHODS
Monte Carlo simulations and a case study examining the association between astrological sign and hospitalization for heart failure.


RESULTS
A standard chi(2) test results in an inflation of the type I error rate, with the type I error rate increasing as the number of levels of the categorical variable increases. Using a standard chi(2) test, the hospitalization rate for Pisces was statistically significantly different from that of the other 11 astrological signs combined (P=0.026). After accounting for the fact that the selection of Pisces was based on it having the highest observed proportion of heart failure hospitalizations, subjects born under the sign of Pisces no longer had a significantly higher rate of heart failure hospitalization compared to the other residents of Ontario (P=0.152).


CONCLUSIONS
Post hoc comparisons of the proportions of successes across different levels of a categorical variable can result in incorrect inferences.
Austin
Peter C
PC
Institute for Clinical Evaluative Sciences, Toronto, Ontario, Canada. peter.austin@ices.on.ca
Goldwasser
Meredith A
MA
D001255:Astrology; D015986:Confounding Factors, Epidemiologic; D004812:Epidemiologic Methods; D006333:Heart Failure / Q000453:epidemiology* / Q000209:etiology; D006760:Hospitalization / Q000706:statistics & numerical data; D006801:Humans; D009864:Ontario / Q000453:epidemiology