title: When to conduct probabilistic linkage vs. deterministic linkage? A simulation study.
abstract: BACKGROUND
  When unique identifiers are unavailable, successful record linkage depends greatly on data quality and types of variables available. While probabilistic linkage theoretically captures more true matches than deterministic linkage by allowing imperfection in identifiers, studies have shown inconclusive results likely due to variations in data quality, implementation of linkage methodology and validation method. The simulation study aimed to understand data characteristics that affect the performance of probabilistic vs. deterministic linkage.
  
  
  METHODS
  We created ninety-six scenarios that represent real-life situations using non-unique identifiers. We systematically introduced a range of discriminative power, rate of missing and error, and file size to increase linkage patterns and difficulties. We assessed the performance difference of linkage methods using standard validity measures and computation time.
  
  
  RESULTS
  Across scenarios, deterministic linkage showed advantage in PPV while probabilistic linkage showed advantage in sensitivity. Probabilistic linkage uniformly outperformed deterministic linkage as the former generated linkages with better trade-off between sensitivity and PPV regardless of data quality. However, with low rate of missing and error in data, deterministic linkage performed not significantly worse. The implementation of deterministic linkage in SAS took less than 1min, and probabilistic linkage took 2min to 2h depending on file size.
  
  
  CONCLUSIONS
  Our simulation study demonstrated that the intrinsic rate of missing and error of linkage variables was key to choosing between linkage methods. In general, probabilistic linkage was a better choice, but for exceptionally good quality data (<5% error), deterministic linkage was a more resource efficient choice.
authors.affiliation: Department of Biostatistics, Graduate School of Medicine, The University of Tokyo, Tokyo, Japan. Electronic address: sophie@epistat.m.u-tokyo.ac.jp.
authors.forename: Ying
authors.initials: Y
authors.lastname: Zhu
authors.affiliation: Department of Biostatistics, Graduate School of Medicine, The University of Tokyo, Tokyo, Japan.
authors.forename: Yutaka
authors.initials: Y
authors.lastname: Matsuyama
authors.affiliation: Department of Biostatistics, Graduate School of Medicine, The University of Tokyo, Tokyo, Japan; Department of Integrated Science and Engineering for Sustainable Society, Chuo University, Tokyo, Japan.
authors.forename: Yasuo
authors.initials: Y
authors.lastname: Ohashi
authors.affiliation: Duke Clinical Research Institute, Duke University School of Medicine, Durham, NC, United States; Department of Pharmacoepidemiology, Graduate School of Medicine, The University of Tokyo, Tokyo, Japan.
authors.forename: Soko
authors.initials: S
authors.lastname: Setoguchi
mesh_terms: D000465:Algorithms; D003198:Computer Simulation; D005189:False Positive Reactions; D008490:Medical Informatics / Q000379:methods*; D008498:Medical Record Linkage / Q000379:methods*; D006278:Medicare; D011237:Predictive Value of Tests; D011336:Probability; D015203:Reproducibility of Results; D012680:Sensitivity and Specificity; D012984:Software; D014481:United States
keywords: Comparative validity; Deterministic linkage; Probabilistic linkage; Record linkage; Simulation study