title: Assessing the impact of clinical information-retrieval technology in a family practice residency.
abstract: OBJECTIVE
  Evidence-based sources of information do not integrate self-assessment tools to assess the impact of a users' search for clinical information. We present a method to evaluate evidence-based sources of information, by systematically assessing the impact of searches for clinical information in everyday practice.
  
  
  METHODS
  We integrated an information management tool (InfoRetriever 2003) with an educational intervention in a cohort of 26 family medicine residents. An electronic impact assessment scale was used by these doctors to report the perceived impact of each item of information (each hit) retrieved on hand-held computer. We compared the types of impact associated with hits in two distinct categories: clinical decision support systems (CDSS) vs. clinical information-retrieval technology (CIRT). Information hits in CDSS were defined as any hit in the following InfoRetriever databases: Clinical Prediction Rules, History and Physical Exam diagnostic calculator and Diagnostic Test calculator. CIRT information hits were defined as any hit in: Abstracts of Cochrane Reviews, InfoPOEMs, evidence-based practice guideline summaries and the Griffith's 5 Minute Clinical Consult.
  
  
  RESULTS
  The impact assessment questionnaire was linked to 5160 information hits. 4946 impact assessment questionnaires were answered (95.9%), and 2495 contained reports of impact (48.4%). Reports of positive impact on doctors were most frequently in the areas of learning and practice improvement. In comparison to CDSS, CIRT hits were more frequently associated with learning and recall. CDSS hits were more frequently associated with reports of practice improvement.
  
  
  CONCLUSIONS
  Our new method permits systematic and comparative assessment of impact associated with distinct categories of information.
authors.affiliation: Herzl Family Practice Centre, Sir Mortimer B. Davis Jewish General Hospital, Montreal, QC, Canada. Roland.Grad@mcgill.ca
authors.forename: Roland M
authors.initials: RM
authors.lastname: Grad
authors.forename: Pierre
authors.initials: P
authors.lastname: Pluye
authors.forename: Yuejing
authors.initials: Y
authors.lastname: Meng
authors.forename: Bernard
authors.initials: B
authors.lastname: Segal
authors.forename: Robyn
authors.initials: R
authors.lastname: Tamblyn
mesh_terms: D015331:Cohort Studies; D005069:Evaluation Studies as Topic; D005194:Family Practice; D006801:Humans; D016247:Information Storage and Retrieval / Q000706:statistics & numerical data*; D007396:Internship and Residency; D008490:Medical Informatics; D011446:Prospective Studies