title: Meta-analysis of the technical performance of an imaging procedure: guidelines and statistical methodology.
abstract: Medical imaging serves many roles in patient care and the drug approval process, including assessing treatment response and guiding treatment decisions. These roles often involve a quantitative imaging biomarker, an objectively measured characteristic of the underlying anatomic structure or biochemical process derived from medical images. Before a quantitative imaging biomarker is accepted for use in such roles, the imaging procedure to acquire it must undergo evaluation of its technical performance, which entails assessment of performance metrics such as repeatability and reproducibility of the quantitative imaging biomarker. Ideally, this evaluation will involve quantitative summaries of results from multiple studies to overcome limitations due to the typically small sample sizes of technical performance studies and/or to include a broader range of clinical settings and patient populations. This paper is a review of meta-analysis procedures for such an evaluation, including identification of suitable studies, statistical methodology to evaluate and summarize the performance metrics, and complete and transparent reporting of the results. This review addresses challenges typical of meta-analyses of technical performance, particularly small study sizes, which often causes violations of assumptions underlying standard meta-analysis techniques. Alternative approaches to address these difficulties are also presented; simulation studies indicate that they outperform standard techniques when some studies are small. The meta-analysis procedures presented are also applied to actual [18F]-fluorodeoxyglucose positron emission tomography (FDG-PET) test-retest repeatability data for illustrative purposes.
authors.affiliation: Division of Cancer Treatment and Diagnosis, National Cancer Institute, NIH, Bethesda, MD, USA erich.huang@nih.gov.
authors.forename: Erich P
authors.initials: EP
authors.lastname: Huang
authors.affiliation: Department of Quantitative Health Sciences, Cleveland Clinic Foundation, Cleveland, OH, USA.
authors.forename: Xiao-Feng
authors.initials: XF
authors.lastname: Wang
authors.affiliation: Department of Biostatistics and Bioinformatics/Department of Radiology, Duke University Medical School, Durham, NC, USA.
authors.forename: Kingshuk Roy
authors.initials: KR
authors.lastname: Choudhury
authors.affiliation: Division of Cancer Treatment and Diagnosis, National Cancer Institute, NIH, Bethesda, MD, USA.
authors.forename: Lisa M
authors.initials: LM
authors.lastname: McShane
authors.affiliation: Department of Epidemiology and Biostatistics, Memorial Sloan Kettering Cancer Center, New York, NY, USA.
authors.forename: Mithat
authors.initials: M
authors.lastname: Gönen
authors.affiliation: Division of Biostatistics, Center of Devices and Radiological Health, Food and Drug Administration, Silver Spring, MD, USA.
authors.forename: Jingjing
authors.initials: J
authors.lastname: Ye
authors.affiliation: Elucid Biomedical Imaging Inc., Wenham, MA, USA.
authors.forename: Andrew J
authors.initials: AJ
authors.lastname: Buckler
authors.affiliation: Department of Radiology, University of Washington, Seattle, WA, USA.
authors.forename: Paul E
authors.initials: PE
authors.lastname: Kinahan
authors.affiliation: School of Electrical and Computer Engineering, Cornell University, Ithaca, NY, USA.
authors.forename: Anthony P
authors.initials: AP
authors.lastname: Reeves
authors.affiliation: Department of Medical Physics, University of Wisconsin, Madison, WI, USA.
authors.forename: Edward F
authors.initials: EF
authors.lastname: Jackson
authors.affiliation: Massachusetts General Hospital, Boston, MA, USA.
authors.forename: Alexander R
authors.initials: AR
authors.lastname: Guimaraes
authors.affiliation: F. Hoffmann-La Roche, Ltd., Basel, Switzerland.
authors.forename: Gudrun
authors.initials: G
authors.lastname: Zahlmann
mesh_terms: D015415:Biomarkers; D003952:Diagnostic Imaging; D017408:Guidelines as Topic; D006801:Humans; D015201:Meta-Analysis as Topic; D015203:Reproducibility of Results; D012107:Research Design; D013223:Statistics as Topic
chemical_list: D015415:Biomarkers
keywords: imaging biomarkers; meta-analysis; meta-regression; quantitative imaging; repeatability; reproducibility; systematic review; technical performance