title: Assessing models for genetic prediction of complex traits: a comparison of visualization and quantitative methods.
abstract: BACKGROUND
  In silico models have recently been created in order to predict which genetic variants are more likely to contribute to the risk of a complex trait given their functional characteristics. However, there has been no comprehensive review as to which type of predictive accuracy measures and data visualization techniques are most useful for assessing these models.
  
  
  METHODS
  We assessed the performance of the models for predicting risk using various methodologies, some of which include: receiver operating characteristic (ROC) curves, histograms of classification probability, and the novel use of the quantile-quantile plot. These measures have variable interpretability depending on factors such as whether the dataset is balanced in terms of numbers of genetic variants classified as risk variants versus those that are not.
  
  
  RESULTS
  We conclude that the area under the curve (AUC) is a suitable starting place, and for models with similar AUCs, violin plots are particularly useful for examining the distribution of the risk scores.
authors.affiliation: Campbell Family Mental Health Research Institute, Centre for Addiction and Mental Health, Toronto, Ontario, Canada. sarah.gagliano@camh.ca.|Institute of Medical Science, University of Toronto, Toronto, Ontario, Canada. sarah.gagliano@camh.ca.|Department of Psychiatry, University of Toronto, Toronto, Ontario, Canada. sarah.gagliano@camh.ca.
authors.forename: Sarah A
authors.initials: SA
authors.lastname: Gagliano
authors.affiliation: Institute of Medical Science, University of Toronto, Toronto, Ontario, Canada. andrew.paterson@utoronto.ca.|Department of Psychiatry, University of Toronto, Toronto, Ontario, Canada. andrew.paterson@utoronto.ca.|Program in Genetics and Genome Biology, The Hospital for Sick Children, Toronto, Ontario, Canada. andrew.paterson@utoronto.ca.|Biostatistics Division, Dalla Lana School of Public Health, University of Toronto, Toronto, Ontario, Canada. andrew.paterson@utoronto.ca.|Epidemiology Division, Dalla Lana School of Public Health, University of Toronto, Toronto, Ontario, Canada. andrew.paterson@utoronto.ca.
authors.forename: Andrew D
authors.initials: AD
authors.lastname: Paterson
authors.affiliation: Department of Medical & Molecular Genetics, King's College London, Guy's Hospital, London, UK. michael.weale@kcl.ac.uk.
authors.forename: Michael E
authors.initials: ME
authors.lastname: Weale
authors.affiliation: Campbell Family Mental Health Research Institute, Centre for Addiction and Mental Health, Toronto, Ontario, Canada. jo.knight@camh.ca.|Institute of Medical Science, University of Toronto, Toronto, Ontario, Canada. jo.knight@camh.ca.|Department of Psychiatry, University of Toronto, Toronto, Ontario, Canada. jo.knight@camh.ca.|Biostatistics Division, Dalla Lana School of Public Health, University of Toronto, Toronto, Ontario, Canada. jo.knight@camh.ca.
authors.forename: Jo
authors.initials: J
authors.lastname: Knight
mesh_terms: D000465:Algorithms; D019540:Area Under Curve; D003198:Computer Simulation; D030541:Databases, Genetic; D015894:Genome, Human; D006801:Humans; D008957:Models, Genetic; D040641:Quantitative Trait Loci; D012372:ROC Curve; D012306:Risk