Reliability testing of a case-leveling framework for assigning level of difficulty of pharmacist's initial patient medication assessments.
OBJECTIVE
To develop and test the reliability of a case-leveling framework for assigning level of difficulty of the pharmacist's task for initial medication assessments conducted by pharmacists integrated into family physician offices.


METHODS
Descriptive study.


METHODS
Seven family practice sites in Ontario from June 2004 to July 2006.


METHODS
Patients referred by their family physician for pharmacist assessment.


METHODS
Individual medication assessments, monitoring, and follow-up by pharmacists. A case-leveling framework was developed with three levels of complexity (graded as I, II, or III) including specific descriptors and practice-based examples. Reliability was assessed between two standardized assessors and between one assessor and project pharmacists. Project pharmacist feedback was elicited through an e-mail survey. Reliability is reported using the kappa statistic.


METHODS
Reliability of a case-leveling framework and helpfulness of the framework as reported by pharmacists.


RESULTS
53 patient cases were evaluated for interrater reliability between standardized assessors. The mean (+/- SD) case level assigned was 1.8 +/- 0.68, and the kappa was 0.62 (95% CI 0.44-0.79), indicating a substantial strength of agreement between raters. For the second reliability test, 52 cases were rated, with a level of agreement between project pharmacists and the external assessor of 0.46 (95% CI 0.27-0.65), indicating moderate agreement. Feedback resulted in slight revisions to the original framework.


CONCLUSIONS
The case-leveling framework was a reliable method and can be used to determine the level of difficulty of patient cases in primary care.
Kennie
Natalie
N
Department of Family and Community Medicine, St. Michael's Hospital, Toronto, Canada.
Dolovich
Lisa
L
D003625:Data Collection; D003627:Data Interpretation, Statistical; D019033:Delivery of Health Care, Integrated / Q000458:organization & administration; D005194:Family Practice / Q000458:organization & administration; D006801:Humans; D054539:Medication Therapy Management / Q000458:organization & administration*; D015588:Observer Variation; D010593:Pharmaceutical Services / Q000458:organization & administration*; D010595:Pharmacists / Q000458:organization & administration*; D011320:Primary Health Care / Q000458:organization & administration; D024382:Professional Role; D012017:Referral and Consultation; D015203:Reproducibility of Results