title: Reliability testing of a case-leveling framework for assigning level of difficulty of pharmacist's initial patient medication assessments.
abstract: OBJECTIVE
  To develop and test the reliability of a case-leveling framework for assigning level of difficulty of the pharmacist's task for initial medication assessments conducted by pharmacists integrated into family physician offices.
  
  
  METHODS
  Descriptive study.
  
  
  METHODS
  Seven family practice sites in Ontario from June 2004 to July 2006.
  
  
  METHODS
  Patients referred by their family physician for pharmacist assessment.
  
  
  METHODS
  Individual medication assessments, monitoring, and follow-up by pharmacists. A case-leveling framework was developed with three levels of complexity (graded as I, II, or III) including specific descriptors and practice-based examples. Reliability was assessed between two standardized assessors and between one assessor and project pharmacists. Project pharmacist feedback was elicited through an e-mail survey. Reliability is reported using the kappa statistic.
  
  
  METHODS
  Reliability of a case-leveling framework and helpfulness of the framework as reported by pharmacists.
  
  
  RESULTS
  53 patient cases were evaluated for interrater reliability between standardized assessors. The mean (+/- SD) case level assigned was 1.8 +/- 0.68, and the kappa was 0.62 (95% CI 0.44-0.79), indicating a substantial strength of agreement between raters. For the second reliability test, 52 cases were rated, with a level of agreement between project pharmacists and the external assessor of 0.46 (95% CI 0.27-0.65), indicating moderate agreement. Feedback resulted in slight revisions to the original framework.
  
  
  CONCLUSIONS
  The case-leveling framework was a reliable method and can be used to determine the level of difficulty of patient cases in primary care.
authors.affiliation: Department of Family and Community Medicine, St. Michael's Hospital, Toronto, Canada.
authors.forename: Natalie
authors.initials: N
authors.lastname: Kennie
authors.forename: Lisa
authors.initials: L
authors.lastname: Dolovich
mesh_terms: D003625:Data Collection; D003627:Data Interpretation, Statistical; D019033:Delivery of Health Care, Integrated / Q000458:organization & administration; D005194:Family Practice / Q000458:organization & administration; D006801:Humans; D054539:Medication Therapy Management / Q000458:organization & administration*; D015588:Observer Variation; D010593:Pharmaceutical Services / Q000458:organization & administration*; D010595:Pharmacists / Q000458:organization & administration*; D011320:Primary Health Care / Q000458:organization & administration; D024382:Professional Role; D012017:Referral and Consultation; D015203:Reproducibility of Results