title: Design-related bias in estimates of accuracy when comparing imaging tests: examples from breast imaging research.
abstract: This work highlights concepts on the potential for designrelated factors to bias estimates of test accuracy in comparative imaging research. We chose two design factors, selection of eligible subjects and the reference standard, to examine the effect of design limitations on estimates of accuracy. Estimates of sensitivity in a study of the comparative accuracy of mammography and ultrasound differed according to how subjects were selected. Comparison of a new imaging test with an existing test should distinguish whether the new test is to be used as a replacement for, or as an adjunct to, the conventional test, to guide the method for subject selection. Quality of the reference standard, examined in a meta-analysis of preoperative breast MRI, varied across studies and was associated with estimates of incremental accuracy. Potential solutions to deal with the reference standard are outlined where an ideal reference standard may not be available in all subjects. These examples of breast imaging research demonstrate that design-related bias, when comparing a new imaging test with a conventional imaging test, may bias accuracy in a direction that favours the new test by overestimating the accuracy of the new test or by underestimating that of the conventional test.
authors.affiliation: School of Public Health, University of Sydney, Sydney, Australia. nehmath@med.usyd.edu.au
authors.forename: Nehmat
authors.initials: N
authors.lastname: Houssami
authors.forename: Stefano
authors.initials: S
authors.lastname: Ciatto
mesh_terms: D016477:Artifacts; D015982:Bias; D035843:Biomedical Research / Q000379:methods*; D001943:Breast Neoplasms / Q000175:diagnosis* / Q000453:epidemiology*; D005260:Female; D006801:Humans; D007089:Image Enhancement / Q000379:methods*; D007090:Image Interpretation, Computer-Assisted / Q000379:methods*; D015203:Reproducibility of Results; D012680:Sensitivity and Specificity
references.citation: AJR Am J Roentgenol. 2003 Apr;180(4):935-40
references.pmid: 12646432
references.citation: Med Decis Making. 2009 Sep-Oct;29(5):E1-E12
references.pmid: 19773580
references.citation: J Clin Epidemiol. 2009 Aug;62(8):797-806
references.pmid: 19447581
references.citation: Radiology. 2003 Jan;226(1):24-8
references.pmid: 12511664
references.citation: Ann Intern Med. 2006 Jun 6;144(11):850-5
references.pmid: 16754927
references.citation: Health Technol Assess. 2010 Jan;14(1):1-182
references.pmid: 20025837
references.citation: Breast. 2002 Apr;11(2):125-30
references.pmid: 14965658
references.citation: J Clin Oncol. 2008 Jul 1;26(19):3248-58
references.pmid: 18474876
references.citation: BMJ. 2008 May 17;336(7653):1106-10
references.pmid: 18483053
references.citation: AJR Am J Roentgenol. 2004 Oct;183(4):1149-57
references.pmid: 15385322
references.citation: JAMA. 1999 Sep 15;282(11):1061-6
references.pmid: 10493205
references.citation: Breast Cancer Res Treat. 2004 Jun;85(3):223-8
references.pmid: 15111760
references.citation: BMJ. 2006 May 6;332(7549):1089-92
references.pmid: 16675820
references.citation: Breast J. 2005 May-Jun;11(3):167-72
references.pmid: 15871700
references.citation: N Engl J Med. 2007 Mar 29;356(13):1295-303
references.pmid: 17392300
references.citation: Clin Chem. 2007 Feb;53(2):164-72
references.pmid: 17185365
references.citation: J Clin Epidemiol. 2009 Apr;62(4):364-73
references.pmid: 18945590
references.citation: Radiology. 2004 Dec;233(3):830-49
references.pmid: 15486214
references.citation: Eur Radiol. 2010 Jan;20(1):1-15
references.pmid: 20069736
references.citation: BMJ. 2002 Mar 16;324(7338):669-71
references.pmid: 11895830
references.citation: Ann Intern Med. 2004 Feb 3;140(3):189-202
references.pmid: 14757617