title: Confidence limits, error bars and method comparison in molecular modeling. Part 2: comparing methods.
abstract: The calculation of error bars for quantities of interest in computational chemistry comes in two forms: (1) Determining the confidence of a prediction, for instance of the property of a molecule; (2) Assessing uncertainty in measuring the difference between properties, for instance between performance metrics of two or more computational approaches. While a former paper in this series concentrated on the first of these, this second paper focuses on comparison, i.e. how do we calculate differences in methods in an accurate and statistically valid manner. Described within are classical statistical approaches for comparing widely used metrics such as enrichment, area under the curve and Pearson's product-moment coefficient, as well as generic measures. These are considered of over single and multiple sets of data and for two or more methods that evince either independent or correlated behavior. General issues concerning significance testing and confidence limits from a Bayesian perspective are discussed, along with size-of-effect aspects of evaluation.
authors.affiliation: OpenEye Scientific Software, Inc., Santa Fe, NM, USA. anthony@eyesopen.com.
authors.forename: A
authors.identifier: http://orcid.org/0000-0001-8488-9641
authors.initials: A
authors.lastname: Nicholls
mesh_terms: D000465:Algorithms; D001499:Bayes Theorem; D019295:Computational Biology / Q000706:statistics & numerical data*; D008958:Models, Molecular; D008962:Models, Theoretical
keywords: Bayes; Computational methods; Confidence intervals; Correlation; Error bars; Evaluations; Significance; Statistics