Design-related bias in estimates of accuracy when comparing imaging tests: examples from breast imaging research.
This work highlights concepts on the potential for designrelated factors to bias estimates of test accuracy in comparative imaging research. We chose two design factors, selection of eligible subjects and the reference standard, to examine the effect of design limitations on estimates of accuracy. Estimates of sensitivity in a study of the comparative accuracy of mammography and ultrasound differed according to how subjects were selected. Comparison of a new imaging test with an existing test should distinguish whether the new test is to be used as a replacement for, or as an adjunct to, the conventional test, to guide the method for subject selection. Quality of the reference standard, examined in a meta-analysis of preoperative breast MRI, varied across studies and was associated with estimates of incremental accuracy. Potential solutions to deal with the reference standard are outlined where an ideal reference standard may not be available in all subjects. These examples of breast imaging research demonstrate that design-related bias, when comparing a new imaging test with a conventional imaging test, may bias accuracy in a direction that favours the new test by overestimating the accuracy of the new test or by underestimating that of the conventional test.
Houssami
Nehmat
N
School of Public Health, University of Sydney, Sydney, Australia. nehmath@med.usyd.edu.au
Ciatto
Stefano
S
D016477:Artifacts; D015982:Bias; D035843:Biomedical Research / Q000379:methods*; D001943:Breast Neoplasms / Q000175:diagnosis* / Q000453:epidemiology*; D005260:Female; D006801:Humans; D007089:Image Enhancement / Q000379:methods*; D007090:Image Interpretation, Computer-Assisted / Q000379:methods*; D015203:Reproducibility of Results; D012680:Sensitivity and Specificity
AJR Am J Roentgenol. 2003 Apr;180(4):935-40
12646432
Med Decis Making. 2009 Sep-Oct;29(5):E1-E12
19773580
J Clin Epidemiol. 2009 Aug;62(8):797-806
19447581
Radiology. 2003 Jan;226(1):24-8
12511664
Ann Intern Med. 2006 Jun 6;144(11):850-5
16754927
Health Technol Assess. 2010 Jan;14(1):1-182
20025837
Breast. 2002 Apr;11(2):125-30
14965658
J Clin Oncol. 2008 Jul 1;26(19):3248-58
18474876
BMJ. 2008 May 17;336(7653):1106-10
18483053
AJR Am J Roentgenol. 2004 Oct;183(4):1149-57
15385322
JAMA. 1999 Sep 15;282(11):1061-6
10493205
Breast Cancer Res Treat. 2004 Jun;85(3):223-8
15111760
BMJ. 2006 May 6;332(7549):1089-92
16675820
Breast J. 2005 May-Jun;11(3):167-72
15871700
N Engl J Med. 2007 Mar 29;356(13):1295-303
17392300
Clin Chem. 2007 Feb;53(2):164-72
17185365
J Clin Epidemiol. 2009 Apr;62(4):364-73
18945590
Radiology. 2004 Dec;233(3):830-49
15486214
Eur Radiol. 2010 Jan;20(1):1-15
20069736
BMJ. 2002 Mar 16;324(7338):669-71
11895830
Ann Intern Med. 2004 Feb 3;140(3):189-202
14757617