Validation of a new reference standard for the diagnosis of vasospasm.
OBJECTIVE
The purpose of our study is to perform an internal validation of a new reference standard for vasospasm diagnosis in aneurysmal subarachnoid hemorrhage (A-SAH) patients.


METHODS
A retrospective study was performed on A-SAH patients between January 2002 and May 2009. All patients were applied to this new reference standard using a multistage hierarchical approach incorporating clinical and imaging criteria. An internal validation method was performed in two phases to compare the new reference standard with digital subtraction angiography (DSA) and to assess accuracy. In Phase I, the diagnostic outcomes from DSA at the primary level were compared with the secondary/tertiary levels in the reference standard. In Phase II, the new reference standard was compared with chart diagnosis. Accuracy test characteristics, agreement rates, kappa values, and bias indices were calculated.


RESULTS
In Phase I (n = 85), there was 87% agreement rate, 0.674 kappa, and 0.12 bias index. However, there was 100% agreement in patients diagnosed with vasospasm by DSA. Sensitivity, specificity, positive predictive value (PPV), and negative predictive value (NPV) were 100%, 61%, 83%, and 100% respectively. In Phase II (n = 137), there was 91% agreement rate, 0.824 kappa, and 0.04 bias index. Sensitivity, specificity, PPV, and NPV were 88%, 95%, 96%, and 87%, respectively.


CONCLUSIONS
Performing validation methods for a new reference standard is an evolving and ongoing process because limitations and bias in the reference standard are identified. Based on the results of this internal validation, a modification in the new reference standard is made at the primary level, resulting in improvement in its accuracy and classification of A-SAH patients.
Reichman
Melissa
M
Department of Radiology, Weill Cornell Medical College/New York Presbyterian Hospital, New York, NY 10065, USA. melissareichman@gmail.com
Gold
Rachel
R
Greenberg
Edward
E
Ivanidze
Jana
J
Elias
Elliot
E
Comunale
Joseph
J
Tsiouris
Apostolos J
AJ
Johnson
Carl
C
Sanelli
Pina C
PC
D000328:Adult; D000368:Aged; D000369:Aged, 80 and over; D015901:Angiography, Digital Subtraction / Q000592:standards* / Q000706:statistics & numerical data*; D005260:Female; D019983:Guideline Adherence / Q000706:statistics & numerical data*; D006801:Humans; D008297:Male; D008875:Middle Aged; D009518:New York / Q000453:epidemiology; D017410:Practice Guidelines as Topic / Q000592:standards*; D011871:Radiology / Q000592:standards*; D012015:Reference Standards; D020301:Vasospasm, Intracranial / Q000000981:diagnostic imaging* / Q000453:epidemiology*