Analysis of diagnostic confidence: application to data from a prospective randomized controlled trial of CT for acute abdominal pain.
BACKGROUND
Changes in diagnostic confidence are used as a measure of a test's efficacy. There are several methods for analyzing such data, but it is unclear which are most robust.


OBJECTIVE
To compare analytical methods for assessing diagnostic confidence, applied to data from a prospective study of computed tomography (CT) for acute abdominal pain.


METHODS
Changes in diagnostic confidence in an illustrative case study were evaluated using five methods: "Basic," "Retained diagnosis," "Omary," "Tsushima," and "Score-based." The case study was a randomized controlled trial of patients admitted to hospital for acute abdominal pain for whom immediate CT was not indicated, comparing "early" CT undertaken within 24 h of admission versus routine standard practice. Admitting surgeons recorded their diagnoses and confidences (5-point scale, 10-90%) both on admission and after 24 h. One- and two-sample t tests, and intention-to-treat and as-treated analyzes, were performed using all five analytic methods.


RESULTS
In the case study, 118 patients were randomized to early CT (n=55) or standard practice (n=63). Mean (SD) diagnostic confidence increased between the two time points by 20.7 (25.2)% and 13.0 (24.9)%, respectively. Early CT showed increases in diagnostic confidence by all one-sample analyzes (all P<0.055). Compared with standard practice, early CT showed significant increases in diagnostic confidence on an a) as-treated basis when using Basic, Omary and Score-based analyzes (P<0.045), but not by the other two analytic methods, and b) intention-to-treat, only by Omary analysis.


CONCLUSIONS
The method of analysis used to evaluate diagnostic confidence can influence conclusions about a test's efficacy. Methods incorporating the soundest analytical principles are recommended.
Ng
Chaan S
CS
Department of Radiology, University of Texas M.D. Anderson Cancer Center, Houston, TX 77030-4009, USA. cng@mdanderson.org
Palmer
Christopher R
CR
D000006:Abdomen, Acute / Q000000981:diagnostic imaging*; D003657:Decision Making; D003951:Diagnostic Errors; D006801:Humans; D011446:Prospective Studies; D016032:Randomized Controlled Trials as Topic; D013223:Statistics as Topic; D014057:Tomography, X-Ray Computed / Q000592:standards*