title: Validation of a new reference standard for the diagnosis of vasospasm.
abstract: OBJECTIVE
  The purpose of our study is to perform an internal validation of a new reference standard for vasospasm diagnosis in aneurysmal subarachnoid hemorrhage (A-SAH) patients.
  
  
  METHODS
  A retrospective study was performed on A-SAH patients between January 2002 and May 2009. All patients were applied to this new reference standard using a multistage hierarchical approach incorporating clinical and imaging criteria. An internal validation method was performed in two phases to compare the new reference standard with digital subtraction angiography (DSA) and to assess accuracy. In Phase I, the diagnostic outcomes from DSA at the primary level were compared with the secondary/tertiary levels in the reference standard. In Phase II, the new reference standard was compared with chart diagnosis. Accuracy test characteristics, agreement rates, kappa values, and bias indices were calculated.
  
  
  RESULTS
  In Phase I (n = 85), there was 87% agreement rate, 0.674 kappa, and 0.12 bias index. However, there was 100% agreement in patients diagnosed with vasospasm by DSA. Sensitivity, specificity, positive predictive value (PPV), and negative predictive value (NPV) were 100%, 61%, 83%, and 100% respectively. In Phase II (n = 137), there was 91% agreement rate, 0.824 kappa, and 0.04 bias index. Sensitivity, specificity, PPV, and NPV were 88%, 95%, 96%, and 87%, respectively.
  
  
  CONCLUSIONS
  Performing validation methods for a new reference standard is an evolving and ongoing process because limitations and bias in the reference standard are identified. Based on the results of this internal validation, a modification in the new reference standard is made at the primary level, resulting in improvement in its accuracy and classification of A-SAH patients.
authors.affiliation: Department of Radiology, Weill Cornell Medical College/New York Presbyterian Hospital, New York, NY 10065, USA. melissareichman@gmail.com
authors.forename: Melissa
authors.initials: M
authors.lastname: Reichman
authors.forename: Rachel
authors.initials: R
authors.lastname: Gold
authors.forename: Edward
authors.initials: E
authors.lastname: Greenberg
authors.forename: Jana
authors.initials: J
authors.lastname: Ivanidze
authors.forename: Elliot
authors.initials: E
authors.lastname: Elias
authors.forename: Joseph
authors.initials: J
authors.lastname: Comunale
authors.forename: Apostolos J
authors.initials: AJ
authors.lastname: Tsiouris
authors.forename: Carl
authors.initials: C
authors.lastname: Johnson
authors.forename: Pina C
authors.initials: PC
authors.lastname: Sanelli
mesh_terms: D000328:Adult; D000368:Aged; D000369:Aged, 80 and over; D015901:Angiography, Digital Subtraction / Q000592:standards* / Q000706:statistics & numerical data*; D005260:Female; D019983:Guideline Adherence / Q000706:statistics & numerical data*; D006801:Humans; D008297:Male; D008875:Middle Aged; D009518:New York / Q000453:epidemiology; D017410:Practice Guidelines as Topic / Q000592:standards*; D011871:Radiology / Q000592:standards*; D012015:Reference Standards; D020301:Vasospasm, Intracranial / Q000000981:diagnostic imaging* / Q000453:epidemiology*