title: Secure and scalable deduplication of horizontally partitioned health data for privacy-preserving distributed statistical computation.
abstract: Techniques have been developed to compute statistics on distributed datasets without revealing private information except the statistical results. However, duplicate records in a distributed dataset may lead to incorrect statistical results. Therefore, to increase the accuracy of the statistical analysis of a distributed dataset, secure deduplication is an important preprocessing step.
  
  
  
  We designed a secure protocol for the deduplication of horizontally partitioned datasets with deterministic record linkage algorithms. We provided a formal security analysis of the protocol in the presence of semi-honest adversaries. The protocol was implemented and deployed across three microbiology laboratories located in Norway, and we ran experiments on the datasets in which the number of records for each laboratory varied. Experiments were also performed on simulated microbiology datasets and data custodians connected through a local area network.
  
  
  
  The security analysis demonstrated that the protocol protects the privacy of individuals and data custodians under a semi-honest adversarial model. More precisely, the protocol remains secure with the collusion of up to N - 2 corrupt data custodians. The total runtime for the protocol scales linearly with the addition of data custodians and records. One million simulated records distributed across 20 data custodians were deduplicated within 45 s. The experimental results showed that the protocol is more efficient and scalable than previous protocols for the same problem.
  
  
  
  The proposed deduplication protocol is efficient and scalable for practical uses while protecting the privacy of patients and data custodians.
authors.affiliation: Department of Computer Science, UiT The Arctic University of Norway, 9037, Tromsø, Norway. kassaye.y.yigzaw@uit.no.|Norwegian Centre for E-health Research, University Hospital of North Norway, 9019, Tromsø, Norway. kassaye.y.yigzaw@uit.no.
authors.forename: Kassaye Yitbarek
authors.identifier: 0000-0001-7068-117X
authors.initials: KY
authors.lastname: Yigzaw
authors.affiliation: Department of Computer Science, University of Westminster, 115 New Cavendish Street, London, W1W 6UW, UK.
authors.forename: Antonis
authors.initials: A
authors.lastname: Michalas
authors.affiliation: Norwegian Centre for E-health Research, University Hospital of North Norway, 9019, Tromsø, Norway.|Department of Clinical Medicine, UiT The Arctic University of Norway, 9037, Tromsø, Norway.
authors.forename: Johan Gustav
authors.initials: JG
authors.lastname: Bellika
mesh_terms: D003219:Confidentiality / Q000592:standards*; D057286:Electronic Health Records / Q000592:standards*; D006801:Humans; D008498:Medical Record Linkage / Q000592:standards*; D013223:Statistics as Topic / Q000592:standards*
keywords: Bloom Filter; Data Linkage; Data Reuse; Deduplication; Distributed Statistical Computation; Duplicate Record; Electronic Health Record; Privacy; Record Linkage; Set Intersection