title: Quantitative imaging biomarkers: a review of statistical methods for computer algorithm comparisons.
abstract: Quantitative biomarkers from medical images are becoming important tools for clinical diagnosis, staging, monitoring, treatment planning, and development of new therapies. While there is a rich history of the development of quantitative imaging biomarker (QIB) techniques, little attention has been paid to the validation and comparison of the computer algorithms that implement the QIB measurements. In this paper we provide a framework for QIB algorithm comparisons. We first review and compare various study designs, including designs with the true value (e.g. phantoms, digital reference images, and zero-change studies), designs with a reference standard (e.g. studies testing equivalence with a reference standard), and designs without a reference standard (e.g. agreement studies and studies of algorithm precision). The statistical methods for comparing QIB algorithms are then presented for various study types using both aggregate and disaggregate approaches. We propose a series of steps for establishing the performance of a QIB algorithm, identify limitations in the current statistical literature, and suggest future directions for research.
authors.affiliation: Cleveland Clinic Foundation, Cleveland, OH, USA obuchon@ccf.org.
authors.forename: Nancy A
authors.initials: NA
authors.lastname: Obuchowski
authors.affiliation: Cornell University, Ithaca, NY, USA.
authors.forename: Anthony P
authors.initials: AP
authors.lastname: Reeves
authors.affiliation: National Institutes of Health, Rockville, MD, USA.
authors.forename: Erich P
authors.initials: EP
authors.lastname: Huang
authors.affiliation: Cleveland Clinic Foundation, Cleveland, OH, USA.
authors.forename: Xiao-Feng
authors.initials: XF
authors.lastname: Wang
authors.affiliation: Elucid Bioimaging Inc., Wenham, MA, USA.
authors.forename: Andrew J
authors.initials: AJ
authors.lastname: Buckler
authors.affiliation: University of California, Los Angeles, CA, USA.
authors.forename: Hyun J Grace
authors.initials: HJ
authors.lastname: Kim
authors.affiliation: Duke University, Durham, NC, USA.
authors.forename: Huiman X
authors.initials: HX
authors.lastname: Barnhart
authors.affiliation: University of Wisconsin-Madison, Madison, WI, USA.
authors.forename: Edward F
authors.initials: EF
authors.lastname: Jackson
authors.affiliation: University of Chicago, Chicago, IL, USA.
authors.forename: Maryellen L
authors.initials: ML
authors.lastname: Giger
authors.affiliation: Food and Drug Administration/CDRH, Silver Spring, MD, USA.
authors.forename: Gene
authors.initials: G
authors.lastname: Pennello
authors.affiliation: Biostatistics Consulting, LLC, Kensington, MD, USA.
authors.forename: Alicia Y
authors.initials: AY
authors.lastname: Toledano
authors.affiliation: MGH/Harvard Medical School, Boston, MA, USA.
authors.forename: Jayashree
authors.initials: J
authors.lastname: Kalpathy-Cramer
authors.affiliation: George Washington University, NW Washington, DC, USA.
authors.forename: Tatiyana V
authors.initials: TV
authors.lastname: Apanasovich
authors.affiliation: University of Washington, Seattle, WA, USA.
authors.forename: Paul E
authors.initials: PE
authors.lastname: Kinahan
authors.affiliation: Food and Drug Administration/CDRH, Silver Spring, MD, USA.
authors.forename: Kyle J
authors.initials: KJ
authors.lastname: Myers
authors.affiliation: University of South Florida, Tampa, FL, USA.
authors.forename: Dmitry B
authors.initials: DB
authors.lastname: Goldgof
authors.affiliation: Duke University, Durham, NC, USA.
authors.forename: Daniel P
authors.initials: DP
authors.lastname: Barboriak
authors.affiliation: H. Moffitt Cancer Center, Tampa, FL, USA.
authors.forename: Robert J
authors.initials: RJ
authors.lastname: Gillies
authors.affiliation: Columbia University, New York, NY, USA.
authors.forename: Lawrence H
authors.initials: LH
authors.lastname: Schwartz
authors.affiliation: Duke University, Durham, NC, USA.
authors.forename: Daniel C
authors.initials: DC
authors.lastname: Sullivan
mesh_terms: D000465:Algorithms; D015982:Bias; D015415:Biomarkers; D003198:Computer Simulation; D003952:Diagnostic Imaging; D006801:Humans; D019047:Phantoms, Imaging; D012015:Reference Standards; D015203:Reproducibility of Results; D012107:Research Design; D013223:Statistics as Topic
chemical_list: D015415:Biomarkers
keywords: agreement; bias; image metrics; imaging biomarkers; precision; quantitative imaging; repeatability; reproducibility