title: Norming a Written Communication Rubric in a Graduate Health Science Course.
abstract: BACKGROUND
  There is growing interest in the use of rubrics to assess written work. This study aimed to determine whether or not the norming of a written communication rubric improved scoring consistency among clinical faculty in a critical thinking course. The benefits of a formalized norming process are described.
  
  
  METHODS
  Faculty-raters were trained to apply the rubric to a signature assignment while participating in calibration workshops. For each rubric criterion, faculty examined whether or not heightened congruence in scoring resulted from the training. Inter-rater reliability was determined after raters independently scored de-identified essays.
  
  
  RESULTS
  Pre-workshop intra-class correlations (ICCs) were acceptable (i.e., >0.7) for three of five rubric criteria. Post-workshop ICCs for only two criteria were acceptable: disciplinary conventions, and sources and evidence. Rater attrition and lag-time between calibration and post-workshop activities likely contributed to reduced consistency.
  
  
  CONCLUSIONS
  The rubric was useful for discriminating writing proficiency. Norming led to revision of the signature assignment, the rubric design, and a need for writing workshops. These changes will result in better student preparation for composing evidence-informed essays. Less-rigid approaches are worthy of future exploration.
authors: University of St. Augustine for Health Sciences, 700 Windy Point Dr., San Marcos, CA 92069, USA. Tel 760-591-3012, fax 7605913053. dturbow@usa.edu.
authors: David J
authors: DJ
authors: Turbow
authors: Thomas P
authors: TP
authors: Werner
authors: Ellen
authors: E
authors: Lowe
authors: Huy Q
authors: HQ
authors: Vu
mesh_terms: D003142:Communication; D004500:Education, Graduate; D004521:Educational Measurement / Q000379:methods; D017408:Guidelines as Topic; D006801:Humans; D058980:Information Literacy; D015203:Reproducibility of Results; D014956:Writing