title: Towards an Operational Definition of Clinical Competency in Pharmacy.
abstract: OBJECTIVE
  To estimate the inter-rater reliability and accuracy of ratings of competence in student pharmacist/patient clinical interactions as depicted in videotaped simulations and to compare expert panelist and typical preceptor ratings of those interactions.
  
  
  METHODS
  This study used a multifactorial experimental design to estimate inter-rater reliability and accuracy of preceptors' assessment of student performance in clinical simulations. The study protocol used nine 5-10 minute video vignettes portraying different levels of competency in student performance in simulated clinical interactions. Intra-Class Correlation (ICC) was used to calculate inter-rater reliability and Fisher exact test was used to compare differences in distribution of scores between expert and nonexpert assessments.
  
  
  RESULTS
  Preceptors (n=42) across 5 states assessed the simulated performances. Intra-Class Correlation estimates were higher for 3 nonrandomized video simulations compared to the 6 randomized simulations. Preceptors more readily identified high and low student performances compared to satisfactory performances. In nearly two-thirds of the rating opportunities, a higher proportion of expert panelists than preceptors rated the student performance correctly (18 of 27 scenarios).
  
  
  CONCLUSIONS
  Valid and reliable assessments are critically important because they affect student grades and formative student feedback. Study results indicate the need for pharmacy preceptor training in performance assessment. The process demonstrated in this study can be used to establish minimum preceptor benchmarks for future national training programs.
authors.affiliation: University of Texas at Tyler, Tyler, Texas.
authors.forename: L Douglas
authors.initials: LD
authors.lastname: Ried
authors.affiliation: Texas A&M University Irma Lerma Pangel College of Pharmacy, Kingsville, Texas.
authors.forename: Charles A
authors.initials: CA
authors.lastname: Douglas
mesh_terms: D000328:Adult; D000368:Aged; D002983:Clinical Competence / Q000592:standards*; D003376:Counseling; D003920:Diabetes Mellitus / Q000188:drug therapy; D004512:Education, Pharmacy / Q000379:methods / Q000592:standards; D005246:Feedback; D005260:Female; D006801:Humans; D008297:Male; D008875:Middle Aged; D015588:Observer Variation; D010604:Pharmacy / Q000592:standards*; D011231:Preceptorship; D013339:Students, Pharmacy; D009626:Terminology as Topic; D014741:Video Recording
keywords: advanced pharmacy practice experience; competency assessment; continuing education; faculty development; preceptors; reliability; simulation