Validity of selected Patient Safety Indicators: opportunities and concerns.
BACKGROUND
The Agency for Healthcare Research and Quality (AHRQ) recently designed the Patient Safety Indicators (PSIs) to detect potential safety-related adverse events. The National Quality Forum has endorsed several of these ICD-9-CM-based indicators as quality-of-care measures. We examined the positive predictive value (PPV) of 3 surgical PSIs: postoperative pulmonary embolus and deep vein thrombosis (pPE/DVT), iatrogenic pneumothorax (iPTX), and accidental puncture and laceration (APL).


METHODS
We applied the AHRQ PSI software (v.3.1a) to fiscal year 2003 to 2007 Veterans Health Administration (VA) administrative data to identify (flag) patients suspected of having a pPE/DVT, iPTX, or APL. Two trained nurse abstractors reviewed a sample of 336 flagged medical records (112 records per PSI) using a standardized instrument. Inter-rater reliability was assessed.


RESULTS
Of 2,343,088 admissions, 6,080 were flagged for pPE/DVT (0.26%), 1,402 for iPTX (0.06%), and 7,203 for APL (0.31%). For pPE/DVT, the PPV was 43% (95% CI, 34% to 53%); 21% of cases had inaccurate coding (eg, arterial not venous thrombosis); and 36% featured thromboembolism present on admission or preoperatively. For iPTX, the PPV was 73% (95% CI, 64% to 81%); 18% had inaccurate coding (eg, spontaneous pneumothorax), and 9% were pneumothoraces present on admission. For APL, the PPV was 85% (95% CI, 77% to 91%); 10% of cases had coding inaccuracies and 5% indicated injuries present on admission. However, 27% of true APLs were minor injuries requiring no surgical repair (eg, small serosal bowel tear). Inter-rater reliability was >90% for all 3 PSIs.


CONCLUSIONS
Until coding revisions are implemented, these PSIs, especially pPE/DVT, should be used primarily for screening and case-finding. Their utility for public reporting and pay-for-performance needs to be reassessed.
Kaafarani
Haytham M A
HM
Department of Surgery, Tufts Medical Center, Tufts University School of Medicine, Boston, MA, USA.
Borzecki
Ann M
AM
Itani
Kamal M F
KM
Loveland
Susan
S
Mull
Hillary J
HJ
Hickson
Kathleen
K
Macdonald
Sally
S
Shin
Marlena
M
Rosen
Amy K
AK
D000368:Aged; D000369:Aged, 80 and over; D005189:False Positive Reactions; D005260:Female; D006302:Health Services Research; D006787:Hospitals, Veterans / Q000706:statistics & numerical data*; D006801:Humans; D007049:Iatrogenic Disease / Q000453:epidemiology*; D022125:Lacerations / Q000453:epidemiology; D008297:Male; D019300:Medical Errors / Q000517:prevention & control / Q000706:statistics & numerical data*; D016347:Medical Records Systems, Computerized; D008875:Middle Aged; D015588:Observer Variation; D011030:Pneumothorax / Q000453:epidemiology; D011183:Postoperative Complications / Q000453:epidemiology* / Q000517:prevention & control; D011237:Predictive Value of Tests; D011655:Pulmonary Embolism / Q000453:epidemiology; D019984:Quality Indicators, Health Care / Q000592:standards*; D015203:Reproducibility of Results; D012189:Retrospective Studies; D012307:Risk Factors; D017751:Safety Management / Q000592:standards*; D014481:United States / Q000453:epidemiology; D020246:Venous Thrombosis / Q000453:epidemiology; D014947:Wounds and Injuries / Q000453:epidemiology