title: A holistic comparative analysis of diagnostic tests for urothelial carcinoma: a study of Cxbladder Detect, UroVysion® FISH, NMP22® and cytology based on imputation of multiple datasets.
abstract: BACKGROUND
  Comparing the relative utility of diagnostic tests is challenging when available datasets are small, partial or incomplete. The analytical leverage associated with a large sample size can be gained by integrating several small datasets to enable effective and accurate across-dataset comparisons. Accordingly, we propose a methodology for a holistic comparative analysis and ranking of cancer diagnostic tests through dataset integration and imputation of missing values, using urothelial carcinoma (UC) as a case study.
  
  
  METHODS
  Five datasets comprising samples from 939 subjects, including 89 with UC, where up to four diagnostic tests (cytology, NMP22®, UroVysion® Fluorescence In-Situ Hybridization (FISH) and Cxbladder Detect) were integrated into a single dataset containing all measured records and missing values. The tests were firstly ranked using three criteria: sensitivity, specificity and a standard variable (feature) ranking method popularly known as signal-to-noise ratio (SNR) index derived from the mean values for all subjects clinically known to have UC versus healthy subjects. Secondly, step-wise unsupervised and supervised imputation (the latter accounting for the 'clinical truth' as determined by cystoscopy) was performed using personalized modelling, k-nearest-neighbour methods, multiple logistic regression and multilayer perceptron neural networks. All imputation models were cross-validated by comparing their post-imputation predictive accuracy for UC with their pre-imputation accuracy. Finally, the post-imputation tests were re-ranked using the same three criteria.
  
  
  RESULTS
  In both measured and imputed data sets, Cxbladder Detect ranked higher for sensitivity, and urine cytology a higher specificity, when compared with other UC tests. Cxbladder Detect consistently ranked higher than FISH and all other tests when SNR analyses were performed on measured, unsupervised and supervised imputed datasets. Supervised imputation resulted in a smaller cross-validation error. Cxbladder Detect was robust to imputation showing a 2% difference in its predictive versus clinical accuracy, outperforming FISH, NMP22 and cytology.
  
  
  CONCLUSIONS
  All data analysed, pre- and post-imputation showed that Cxbladder Detect had higher SNR and outperformed all other comparator tests, including FISH. The methodology developed and validated for comparative ranking of the diagnostic tests for detecting UC, may be further applied to other cancer diagnostic datasets across population groups and multiple datasets.
authors.affiliation: Auckland University of Technology, Auckland, New Zealand. vbreen@aut.ac.nz.
authors.forename: Vivienne
authors.initials: V
authors.lastname: Breen
authors.affiliation: Auckland University of Technology, Auckland, New Zealand. nkasabov@aut.ac.nz.
authors.forename: Nikola
authors.initials: N
authors.lastname: Kasabov
authors.affiliation: M. D. Anderson Cancer Center, University of Texas, Houston, TX, USA. akamat@mdanderson.org.
authors.forename: Ashish M
authors.initials: AM
authors.lastname: Kamat
authors.affiliation: Pacific Edge Limited, Dunedin, New Zealand. elsie.jacobson@pacificedge.co.nz.
authors.forename: Elsie
authors.initials: E
authors.lastname: Jacobson
authors.affiliation: Pacific Edge Limited, Dunedin, New Zealand. jimmy.suttie@pelnz.com.
authors.forename: James M
authors.initials: JM
authors.lastname: Suttie
authors.affiliation: Pacific Edge Limited, Dunedin, New Zealand. paul.osullivan@pacificedge.co.nz.
authors.forename: Paul J
authors.initials: PJ
authors.lastname: O'Sullivan
authors.affiliation: Pacific Edge Limited, Dunedin, New Zealand. laimonis.kavalieris@pacificedge.co.nz.
authors.forename: Laimonis
authors.initials: L
authors.lastname: Kavalieris
authors.affiliation: Pacific Edge Limited, Dunedin, New Zealand. dave.darling@pelnz.com.
authors.forename: David G
authors.initials: DG
authors.lastname: Darling
mesh_terms: D000465:Algorithms; D002295:Carcinoma, Transitional Cell / Q000175:diagnosis* / Q000235:genetics; D003581:Cytodiagnosis; D016208:Databases, Factual / Q000706:statistics & numerical data; D003955:Diagnostic Tests, Routine / Q000379:methods* / Q000592:standards / Q000706:statistics & numerical data; D006801:Humans; D017404:In Situ Hybridization, Fluorescence; D015203:Reproducibility of Results; D012680:Sensitivity and Specificity; D001749:Urinary Bladder Neoplasms / Q000175:diagnosis* / Q000235:genetics