A modified sign test for comparing paired ROC curves.
We develop a permutation test for assessing a difference in the areas under the curve (AUCs) in a paired setting where both modalities are given to each diseased and nondiseased subject. We propose that permutations be made between subjects specifically by shuffling the diseased/nondiseased labels of the subjects within each modality. As these permutations are made within modality, the permutation test is valid even if both modalities are measured on different scales. We show that our permutation test is a sign test for the symmetry of an underlying discrete distribution whose size remains valid under the assumption of equal AUCs. We demonstrate the operating characteristics of our test via simulation and show that our test is equal in power to a permutation test recently proposed by Bandos and others (2005).
Braun
Thomas M
TM
Department of Biostatistics, University of Michigan, Ann Arbor, MI 48109, USA. tombraun@umich.edu
Alonzo
Todd A
TA
D019540:Area Under Curve; D001699:Biometry / Q000379:methods*; D003131:Combined Modality Therapy / Q000706:statistics & numerical data; D003627:Data Interpretation, Statistical; D003955:Diagnostic Tests, Routine; D006801:Humans; D016555:Matched-Pair Analysis; D012372:ROC Curve; D011897:Random Allocation; D018401:Sample Size; D012680:Sensitivity and Specificity; D013810:Therapeutic Equivalency; D014894:Weights and Measures