Performance Comparison of CGM Systems: MARD Values Are Not Always a Reliable Indicator of CGM System Accuracy.
BACKGROUND
The ongoing progress of continuous glucose monitoring (CGM) systems results in an increasing interest in comparing their performance, in particular in terms of accuracy, that is, matching CGM readings with reference values measured at the same time. Most often accuracy is evaluated by the mean absolute relative difference (MARD). It is frequently overseen that MARD does not only reflect accuracy, but also the study protocol and evaluation procedure, making a cross-study comparison problematic.


METHODS
We evaluate the effect of several factors on the MARD statistical properties: number of paired reference and CGM values, distribution of the paired values, accuracy of the reference measurement device itself and the time delay between data pairs. All analysis is done using clinical data from 12 patients wearing 6 sensors each.


RESULTS
We have found that a few paired points can have a potentially high impact on MARD. Leaving out those points for evaluation thus reduces the MARD. Similarly, accuracy of the reference measurements greatly affects the MARD as numerical and graphical data show. Results also show that a log-normal distribution of the paired references provides a significantly different MARD than, for example, a uniform distribution.


CONCLUSIONS
MARD is a reasonable parameter to characterize the performance of CGM systems when keeping its limitations in mind. To support clinicians and patients in selecting which CGM system to use in a clinical setting, care should be taken to make MARD more comparable by employing a standardized evaluation procedure.
Institute for Design and Control of Mechatronical Systems, Johannes Kepler University, Linz, Austria harald.kirchsteiger@jku.at.
Harald
H
Kirchsteiger
Science & Co, Düsseldorf, Germany.
Lutz
L
Heinemann
Institute for Diabetes-Technology GmbH, at Ulm University, Ulm, Germany.
Guido
G
Freckmann
Roche Diagnostics GmbH, Mannheim, Germany.
Volker
V
Lodwig
Roche Diagnostics GmbH, Mannheim, Germany.
Günther
G
Schmelzeisen-Redeker
Roche Diagnostics GmbH, Mannheim, Germany.
Michael
M
Schoemaker
Institute for Design and Control of Mechatronical Systems, Johannes Kepler University, Linz, Austria.
Luigi
L
Del Re
D015374:Biosensing Techniques / Q000295:instrumentation*; D001786:Blood Glucose / Q000032:analysis*; D015190:Blood Glucose Self-Monitoring / Q000295:instrumentation*; D003922:Diabetes Mellitus, Type 1 / Q000097:blood*; D006801:Humans; D008422:Materials Testing; D015203:Reproducibility of Results
D001786:Blood Glucose
CGM; MARD; accuracy; continuous glucose monitoring; performance comparison; performance evaluation; precision