A note on testing for intervention effects on binary responses.
OBJECTIVE
In some circumstances controlled trials are not feasible and treatments can only be evaluated using clinical databases. Here we consider the situation where treatment is introduced at a particular calendar time and can only be evaluated by comparison with historical controls. In these circumstances Heuer and Abel recommended using change-point methods to search for change in characteristics over the whole study period rather than simply comparing treated and untreated patients. Their recommendation is to only conclude that the intervention had an effect if a change-point could be demonstrated close in time to the introduction of the new treatment. This reduces the risk of false positives caused by confounding changes in population characteristics or changes in patient management. For binary data we develop a method that follows their philosophy and apply it to an observational study in the treatment of pin sites after orthopaedic surgery.


METHODS
Tests for change in binomial probabilities based on Brownian bridge and Hansen's approximation for maximally selected chi(2) statistics are compared to an exact test by Worsley. The approximate method is generalized to logistic regression models allowing for covariates.


RESULTS
The agreement of the exact and approximate method is good for sample sizes of 100 or more. The actual test size of the Hansen approximate test allowing for covariates is close to the nominal level, whereas the Brownian bridge approximation is slightly conservative. The change in pin site treatment significantly reduces the risk of infection for both adults and children.


CONCLUSIONS
We consider the Hansen approximation to provide a very good and very simple method for obtaining the p-value when testing for a change in binary data event probabilities, with or without covariates.
Friede
T
T
Biostatics and Statistical Reporting, Novartis Pharma AG, 4002 Basel, Switzerland. tim.friede@novartis.com
Henderson
R
R
Kao
C-F
CF
D000328:Adult; D016010:Binomial Distribution; D016009:Chi-Square Distribution; D002648:Child; D002985:Clinical Protocols; D016001:Confidence Intervals; D015986:Confounding Factors, Epidemiologic; D003627:Data Interpretation, Statistical; D016267:External Fixators / Q000382:microbiology*; D050723:Fractures, Bone / Q000601:surgery*; D006801:Humans; D016013:Likelihood Functions; D016015:Logistic Models; D016555:Matched-Pair Analysis; D019370:Observation; D017063:Outcome Assessment, Health Care / Q000379:methods* / Q000706:statistics & numerical data; D013530:Surgical Wound Infection / Q000453:epidemiology / Q000517:prevention & control*; D013997:Time Factors