title: Metrics for external model evaluation with an application to the population pharmacokinetics of gliclazide.
abstract: OBJECTIVE
  The aim of this study is to define and illustrate metrics for the external evaluation of a population model.
  
  
  METHODS
  In this paper, several types of metrics are defined: based on observations (standardized prediction error with or without simulation and normalized prediction distribution error); based on hyperparameters (with or without simulation); based on the likelihood of the model. All the metrics described above are applied to evaluate a model built from two phase II studies of gliclazide. A real phase I dataset and two datasets simulated with the real dataset design are used as external validation datasets to show and compare how metrics are able to detect and explain potential adequacies or inadequacies of the model.
  
  
  RESULTS
  Normalized prediction errors calculated without any approximation, and metrics based on hyperparameters or on objective function have good theoretical properties to be used for external model evaluation and showed satisfactory behaviour in the simulation study.
  
  
  CONCLUSIONS
  For external model evaluation, prediction distribution errors are recommended when the aim is to use the model to simulate data. Metrics through hyperparameters should be preferred when the aim is to compare two populations and metrics based on the objective function are useful during the model building process.
authors.affiliation: INSERM U738, Paris, France. karl.brendel@bch.aphp.fr
authors.forename: Karl
authors.initials: K
authors.lastname: Brendel
authors.forename: Emmanuelle
authors.initials: E
authors.lastname: Comets
authors.forename: Céline
authors.initials: C
authors.lastname: Laffont
authors.forename: Christian
authors.initials: C
authors.lastname: Laveille
authors.forename: France
authors.initials: F
authors.lastname: Mentré
mesh_terms: D000465:Algorithms; D001185:Artificial Intelligence; D001682:Biological Availability; D017322:Clinical Trials, Phase II as Topic; D003198:Computer Simulation; D003627:Data Interpretation, Statistical; D005907:Gliclazide / Q000008:administration & dosage / Q000493:pharmacokinetics*; D006801:Humans; D007004:Hypoglycemic Agents / Q000008:administration & dosage / Q000493:pharmacokinetics*; D015233:Models, Statistical; D011153:Population; D015203:Reproducibility of Results
chemical_list: D007004:Hypoglycemic Agents; D005907:Gliclazide