title: Testing for interchangeability of imaging tests.
abstract: OBJECTIVE
  New tests are typically assessed by estimating their technical and diagnostic performance through comparisons with a reference standard. A valid reference standard, however, is not always available and is not required for assessing the interchangeability of a new test with an existing one.
  
  
  METHODS
  To show interchangeability of a new test with an existing test, one compares the differences in diagnoses between the new and existing tests to differences between diagnoses made with the existing test on several occasions. We illustrate the test for interchangeability with two studies. In a transcatheter aortic valve replacement study, we test whether semiautomated analysis can be used interchangeably with manual reconstructions from three-dimensional computed tomography (CT) images. In patients with femoroacetabular impingement, we test whether magnetic resonance imaging (MRI) can replace CT to measure acetabular version.
  
  
  RESULTS
  Although the semiautomated method agreed often with the manual measurement of aortic valve size (87.6%), interchanging the semiautomated method with manual measurements by an expert would lead to a 1.7%-12.2% increase in the frequency of disagreement. Interchanging MRI for CT to measure acetabular version would lead to differences in angle measurements of 2.0° to 3.1° in excess of the differences we would expect to see with CT alone.
  
  
  CONCLUSIONS
  Testing for agreement or correlation between a new and an existing test is not sufficient evidence of the performance of a new test. A formal evaluation of interchangeability can be conducted in the absence of a reference standard.
authors.affiliation: Department of Quantitative Health Sciences, Cleveland Clinic Foundation, 9500 Euclid Ave, Cleveland, OH 44195; Department of Diagnostic Radiology, Cleveland Clinic Foundation, Cleveland, OH. Electronic address: obuchon@ccf.org.
authors.forename: Nancy A
authors.initials: NA
authors.lastname: Obuchowski
authors.affiliation: Department of Diagnostic Radiology, Cleveland Clinic Foundation, Cleveland, OH.
authors.forename: Naveen
authors.initials: N
authors.lastname: Subhas
authors.affiliation: Department of Diagnostic Radiology, Cleveland Clinic Foundation, Cleveland, OH; Department of Cardiovascular Medicine, Cleveland Clinic Foundation, Cleveland, OH.
authors.forename: Paul
authors.initials: P
authors.lastname: Schoenhagen
mesh_terms: D019983:Guideline Adherence; D006801:Humans; D007090:Image Interpretation, Computer-Assisted / Q000592:standards*; D018810:Magnetic Resonance Angiography / Q000592:standards*; D015588:Observer Variation; D011785:Quality Assurance, Health Care / Q000379:methods*; D011871:Radiology / Q000592:standards*; D012015:Reference Standards; D015203:Reproducibility of Results; D012680:Sensitivity and Specificity; D014057:Tomography, X-Ray Computed / Q000592:standards*; D014481:United States
keywords: Reference standard; accuracy; agreement; correlation; interchangeability