title: Assessment of the reliability of data collected for the Department of Veterans Affairs national surgical quality improvement program.
abstract: BACKGROUND
  The Office of the Medical Inspector of the Department of Veterans Affairs (VA) studied the reliability of data collected by the VA's National Surgical Quality Improvement Program (NSQIP). The study focused on case selection bias, accuracy of reports on patients who died, and interrater reliability measurements of patient risk variables and outcomes.
  
  
  METHODS
  Surgical data from a sample of 15 VA medical centers were analyzed. For case selection bias, reviewers applied NSQIP criteria to include or exclude 2,460 patients from the database, comparing their results with those of NSQIP staff. For accurate reporting of patients who died, reviewers compared Social Security numbers of 10,444 NSQIP records with those found in the VA Beneficiary Identification and Records Locator Subsystem, VA Patient Treatment Files, and Social Security Administration death files. For measurement of interrater reliability, reviewers reabstracted 59 variables in each of 550 patient medical records that also were recorded in the NSQIP database.
  
  
  RESULTS
  On case selection bias, the reviewers agreed with NSQIP decisions on 2,418 (98%) of the 2,460 cases. Computer record matching identified 4 more deaths than the NSQIP total of 198, a difference of about 2%. For 52 of the categorical variables, agreement, uncorrected for chance, was 96%. For 48 of 52 categorical variables, kappas ranged from 0.61 to 1.0 (substantial to almost perfect agreement); none of the variables had kappas of less than 0.20 (slight to poor agreement).
  
  
  CONCLUSIONS
  This sample of medical centers shows adherence to criteria in selecting cases for the NSQIP database, for reporting deaths, and for collecting patient risk variables.
authors: Department of Veterans Affairs, Washington, DC 20420, USA.
authors: Chester L
authors: CL
authors: Davis
authors: John R
authors: JR
authors: Pierce
authors: William
authors: W
authors: Henderson
authors: C David
authors: CD
authors: Spencer
authors: Christine
authors: C
authors: Tyler
authors: Robert
authors: R
authors: Langberg
authors: Jennan
authors: J
authors: Swafford
authors: Gladys S
authors: GS
authors: Felan
authors: Martha A
authors: MA
authors: Kearns
authors: Brigitte
authors: B
authors: Booker
mesh_terms: D003625:Data Collection; D005260:Female; D006787:Hospitals, Veterans / Q000592:standards*; D006801:Humans; D008297:Male; D008875:Middle Aged; D015588:Observer Variation; D017063:Outcome Assessment, Health Care; D011785:Quality Assurance, Health Care; D012307:Risk Factors; D013514:Surgical Procedures, Operative / Q000401:mortality / Q000592:standards*; D014481:United States; D014493:United States Department of Veterans Affairs