title: Development of a clinician reputation metric to identify appropriate problem-medication pairs in a crowdsourced knowledge base.
abstract: BACKGROUND
  Correlation of data within electronic health records is necessary for implementation of various clinical decision support functions, including patient summarization. A key type of correlation is linking medications to clinical problems; while some databases of problem-medication links are available, they are not robust and depend on problems and medications being encoded in particular terminologies. Crowdsourcing represents one approach to generating robust knowledge bases across a variety of terminologies, but more sophisticated approaches are necessary to improve accuracy and reduce manual data review requirements.
  
  
  OBJECTIVE
  We sought to develop and evaluate a clinician reputation metric to facilitate the identification of appropriate problem-medication pairs through crowdsourcing without requiring extensive manual review.
  
  
  METHODS
  We retrieved medications from our clinical data warehouse that had been prescribed and manually linked to one or more problems by clinicians during e-prescribing between June 1, 2010 and May 31, 2011. We identified measures likely to be associated with the percentage of accurate problem-medication links made by clinicians. Using logistic regression, we created a metric for identifying clinicians who had made greater than or equal to 95% appropriate links. We evaluated the accuracy of the approach by comparing links made by those physicians identified as having appropriate links to a previously manually validated subset of problem-medication pairs.
  
  
  RESULTS
  Of 867 clinicians who asserted a total of 237,748 problem-medication links during the study period, 125 had a reputation metric that predicted the percentage of appropriate links greater than or equal to 95%. These clinicians asserted a total of 2464 linked problem-medication pairs (983 distinct pairs). Compared to a previously validated set of problem-medication pairs, the reputation metric achieved a specificity of 99.5% and marginally improved the sensitivity of previously described knowledge bases.
  
  
  CONCLUSIONS
  A reputation metric may be a valuable measure for identifying high quality clinician-entered, crowdsourced data.
authors: The University of Texas School of Biomedical Informatics at Houston, 7000 Fannin St., Ste. 600, Houston, TX 70030, USA. Electronic address: amccoy1@tulane.edu.
authors: Allison B
authors: AB
authors: McCoy
authors: Brigham and Women's Hospital, Harvard Medical School, 1620 Tremont St., Boston, MA 02115, USA. Electronic address: awright5@partners.org.
authors: Adam
authors: A
authors: Wright
authors: The University of Texas School of Biomedical Informatics at Houston, 7000 Fannin St., Ste. 600, Houston, TX 70030, USA. Electronic address: deevakar.rogith@uth.tmc.edu.
authors: Deevakar
authors: D
authors: Rogith
authors: The University of Texas School of Biomedical Informatics at Houston, 7000 Fannin St., Ste. 600, Houston, TX 70030, USA. Electronic address: safa.fathiamini@uth.tmc.edu.
authors: Safa
authors: S
authors: Fathiamini
authors: The University of Texas Medical School at Houston, 6410 Fannin St., Ste. 1100, Houston, TX 77030, USA. Electronic address: allison.ottenbacher@nih.gov.
authors: Allison J
authors: AJ
authors: Ottenbacher
authors: The University of Texas School of Biomedical Informatics at Houston, 7000 Fannin St., Ste. 600, Houston, TX 70030, USA. Electronic address: dean.f.sittig@uth.tmc.edu.
authors: Dean F
authors: DF
authors: Sittig
mesh_terms: D063045:Crowdsourcing; D057286:Electronic Health Records; D006801:Humans; D020407:Internet; D051188:Knowledge Bases; D016015:Logistic Models; D008490:Medical Informatics / Q000379:methods*; D016347:Medical Records Systems, Computerized; D004364:Pharmaceutical Preparations; D010820:Physicians; D015203:Reproducibility of Results; D012984:Software; D014584:User-Computer Interface
chemical_list: D004364:Pharmaceutical Preparations
keywords: Crowdsourcing; Electronic health records; Knowledge bases; Medical records; Problem-oriented