title: Assessing performance of closed-loop insulin delivery systems by continuous glucose monitoring: drawbacks and way forward.
abstract: BACKGROUND
  We investigated whether continuous glucose monitoring (CGM) levels can accurately assess glycemic control while directing closed-loop insulin delivery.
  
  
  METHODS
  Data were analyzed retrospectively from 33 subjects with type 1 diabetes who underwent closed-loop and conventional pump therapy on two separate nights. Glycemic control was evaluated by reference plasma glucose and contrasted against three methods based on Navigator (Abbott Diabetes Care, Alameda, CA) CGM levels.
  
  
  RESULTS
  Glucose mean and variability were estimated by unmodified CGM levels with acceptable clinical accuracy. Time when glucose was in target range was overestimated by CGM during closed-loop nights (CGM vs. plasma glucose median [interquartile range], 86% [65-97%] vs. 75% [59-91%]; P=0.04) but not during conventional pump therapy (57% [32-72%] vs. 51% [29-68%]; P=0.82) providing comparable treatment effect (mean [SD], 28% [29%] vs. 23% [21%]; P=0.11). Using the CGM measurement error of 15% derived from plasma glucose-CGM pairs (n=4,254), stochastic interpretation of CGM gave unbiased estimate of time in target during both closed-loop (79% [62-86%] vs. 75% [59-91%]; P=0.24) and conventional pump therapy (54% [33-66%] vs. 51% [29-68%]; P=0.44). Treatment effect (23% [24%] vs. 23% [21%]; P=0.96) and time below target were accurately estimated by stochastic CGM. Recalibrating CGM using reference plasma glucose values taken at the start and end of overnight closed-loop was not superior to stochastic CGM.
  
  
  CONCLUSIONS
  CGM is acceptable to estimate glucose mean and variability, but without adjustment it may overestimate benefit of closed-loop. Stochastic CGM provided unbiased estimate of time when glucose is in target and below target and may be acceptable for assessment of closed-loop in the outpatient setting.
authors.affiliation: Metabolic Research Laboratories, Institute of Metabolic Science, Cambridge, United Kingdom. rh347@cam.ac.uk
authors.forename: Roman
authors.initials: R
authors.lastname: Hovorka
authors.forename: Marianna
authors.initials: M
authors.lastname: Nodale
authors.forename: Ahmad
authors.initials: A
authors.lastname: Haidar
authors.forename: Malgorzata E
authors.initials: ME
authors.lastname: Wilinska
mesh_terms: D000293:Adolescent; D000465:Algorithms; D015374:Biosensing Techniques / Q000379:methods* / Q000639:trends; D001786:Blood Glucose / Q000378:metabolism; D015190:Blood Glucose Self-Monitoring / Q000379:methods* / Q000639:trends; D002648:Child; D003922:Diabetes Mellitus, Type 1 / Q000097:blood / Q000188:drug therapy*; D005260:Female; D006442:Glycated Hemoglobin / Q000378:metabolism; D006801:Humans; D007004:Hypoglycemic Agents / Q000008:administration & dosage*; D007328:Insulin / Q000008:administration & dosage*; D007332:Insulin Infusion Systems / Q000592:standards / Q000639:trends; D008297:Male; D018670:Monitoring, Ambulatory / Q000379:methods* / Q000639:trends; D015203:Reproducibility of Results; D012189:Retrospective Studies; D016896:Treatment Outcome
chemical_list: D001786:Blood Glucose; D006442:Glycated Hemoglobin; D007004:Hypoglycemic Agents; D007328:Insulin; C517652:hemoglobin A1c protein, human