title: Statistical issues in the comparison of quantitative imaging biomarker algorithms using pulmonary nodule volume as an example.
abstract: Quantitative imaging biomarkers are being used increasingly in medicine to diagnose and monitor patients' disease. The computer algorithms that measure quantitative imaging biomarkers have different technical performance characteristics. In this paper we illustrate the appropriate statistical methods for assessing and comparing the bias, precision, and agreement of computer algorithms. We use data from three studies of pulmonary nodules. The first study is a small phantom study used to illustrate metrics for assessing repeatability. The second study is a large phantom study allowing assessment of four algorithms' bias and reproducibility for measuring tumor volume and the change in tumor volume. The third study is a small clinical study of patients whose tumors were measured on two occasions. This study allows a direct assessment of six algorithms' performance for measuring tumor change. With these three examples we compare and contrast study designs and performance metrics, and we illustrate the advantages and limitations of various common statistical methods for quantitative imaging biomarker studies.
authors.affiliation: Cleveland Clinic Foundation, Cleveland, OH, USA obuchon@ccf.org.
authors.forename: Nancy A
authors.initials: NA
authors.lastname: Obuchowski
authors.affiliation: Duke University, Durham, NC, USA.
authors.forename: Huiman X
authors.initials: HX
authors.lastname: Barnhart
authors.affiliation: Elucid Bioimaging Inc., Wenham, MA, USA.
authors.forename: Andrew J
authors.initials: AJ
authors.lastname: Buckler
authors.affiliation: Food and Drug Administration/CDRH, Washington, DC, USA.
authors.forename: Gene
authors.initials: G
authors.lastname: Pennello
authors.affiliation: Cleveland Clinic Foundation, Cleveland, OH, USA.
authors.forename: Xiao-Feng
authors.initials: XF
authors.lastname: Wang
authors.affiliation: MGH/Harvard Medical School, Boston, MA, USA.
authors.forename: Jayashree
authors.initials: J
authors.lastname: Kalpathy-Cramer
authors.affiliation: UCLA, Los Angeles, CA, USA.
authors.forename: Hyun J Grace
authors.initials: HJ
authors.lastname: Kim
authors.affiliation: Cornell University, Ithaca, NY, USA.
authors.forename: Anthony P
authors.initials: AP
authors.lastname: Reeves
mesh_terms: D000465:Algorithms; D015982:Bias; D015415:Biomarkers; D003952:Diagnostic Imaging; D006801:Humans; D019047:Phantoms, Imaging; D015203:Reproducibility of Results; D012107:Research Design; D003074:Solitary Pulmonary Nodule / Q000175:diagnosis*; D013223:Statistics as Topic
chemical_list: D015415:Biomarkers
keywords: agreement; bias; coverage probability; intraclass correlation coefficient; limits of agreement; repeatability; reproducibility