Evaluating structured care for diabetes: can calibration on margins help to avoid overestimation of the benefits? An illustration from French diabetes provider networks using data from the ENTRED Survey.
OBJECTIVE
While there is growing evidence on the effectiveness of structured care for diabetic patients in trial settings, standard population level evaluations may misestimate intervention benefits due to patient selection. In order to account for potential biases in measuring intervention benefits, we tested the impact of calibration on margins as a novel adjustment method in an evaluation context compared with simple poststratification.


METHODS
We compared the results of a before-after evaluation on HbA1c levels after 1 year of enrollment in a French diabetes provider network (DPN) using an unadjusted sample and samples adjusted by simple poststratification to results obtained after adjustment via calibration on margins to the general diabetic population's characteristics using a national cross-sectional sample of diabetic patients.


RESULTS
Both with and without adjustment, patients in the DPN had significantly lower HbA1c levels after 1 year of enrollment. However, the reductions in HbA1c levels among the adjusted samples were 22-183% lower than those measured in the unadjusted sample, regardless of the poststratification method and characteristics used. Compared with simple poststratification, estimations using calibration on margins exhibited higher performance.


CONCLUSIONS
Evaluations of diabetes management interventions based on uncontrolled before-after experiments may overestimate the actual benefit for patients. This can be corrected by using poststratification approaches when data on the ultimate target population for the intervention are available. In order to more accurately estimate the effect an intervention would have if extended to the target population, calibration on margins seems to be preferable over simple poststratification in terms of performance and usability.
Université Paris Est Créteil, Université Paris Diderot, Sorbonne Paris Cité, Paris, FranceInserm, ECEVE U1123, Paris, FranceAP-HP, URC-Eco, Paris, France karine.chevreul@urc-eco.fr.
Karine
K
Chevreul
Université Paris Est Créteil, Université Paris Diderot, Sorbonne Paris Cité, Paris, FranceInserm, ECEVE U1123, Paris, FranceAP-HP, URC-Eco, Paris, France.
Matthias
M
Brunn
Université Paris Est Créteil, Université Paris Diderot, Sorbonne Paris Cité, Paris, FranceInserm, ECEVE U1123, Paris, FranceAP-HP, URC-Eco, Paris, France.
Benjamin
B
Cadier
RAND Europe, Health and Healthcare Research Programme, Cambridge, U.K.
Ellen
E
Nolte
Université Paris Est Créteil, Université Paris Diderot, Sorbonne Paris Cité, Paris, FranceInserm, ECEVE U1123, Paris, FranceAP-HP, URC-Eco, Paris, France.
Isabelle
I
Durand-Zaleski
D000368:Aged; D002138:Calibration; D003430:Cross-Sectional Studies; D003625:Data Collection; D003920:Diabetes Mellitus / Q000628:therapy*; D005260:Female; D005602:France; D006442:Glycated Hemoglobin / Q000378:metabolism*; D006801:Humans; D008297:Male; D008875:Middle Aged
D006442:Glycated Hemoglobin