Using probabilistic corrections to account for abstractor agreement in medical record reviews.
The quality of medical record abstracts is often characterized in a reliability substudy. These results usually indicate agreement, but not the extent to which lack of agreement affects associations observed in the complete data. In this study, medical records were reviewed and abstracted for patients diagnosed with stage I or stage II breast cancer between 1990 and 1994 at one of six US Cancer Research Network sites. For a subsample, interrater reliability data were available. The authors calculated conventional hazard ratios and 95% confidence intervals for the association of demographic, tumor, and treatment characteristics with recurrence rate. These conventional estimates of effect were compared with three sets of estimates and 95% simulation intervals that took account of the uncertainty assessed by lack of agreement in the reliability substudy. The rate of recurrence was associated with increasing cancer stage and with treatment modality but not with demographic characteristics. The hazard ratios and simulation intervals that took account of the reliability data showed that the simulation interval grew wider as the sources of uncertainty taken into account grew more complete, but the associations expected a priori remained readily apparent. While many investigators use reliability data only as a metric for data quality, a more thorough approach can also quantitatively depict the uncertainty in the observed associations.
Lash
Timothy L
TL
Boston University School of Public Health, Boston, MA 02118, USA. tlash@bu.edu
Fox
Matthew P
MP
Thwin
Soe Soe
SS
Geiger
Ann M
AM
Buist
Diana S M
DS
Wei
Feifei
F
Field
Terry S
TS
Yood
Marianne Ulcickas
MU
Frost
Floyd J
FJ
Quinn
Virginia P
VP
Prout
Marianne N
MN
Silliman
Rebecca A
RA
D000043:Abstracting and Indexing; D000368:Aged; D000369:Aged, 80 and over; D001943:Breast Neoplasms / Q000453:epidemiology* / Q000473:pathology / Q000601:surgery; D015897:Comorbidity; D003627:Data Interpretation, Statistical; D005260:Female; D006801:Humans; D008408:Mastectomy / Q000379:methods; D008499:Medical Records; D009364:Neoplasm Recurrence, Local / Q000453:epidemiology; D011336:Probability; D014481:United States / Q000453:epidemiology