title: Development and initial validation of a project-based rubric to assess the systems-based practice competency of residents in the clinical chemistry rotation of a pathology residency.
abstract: BACKGROUND
  Systems-based practice (SBP) is 1 of 6 core competencies required in all resident training programs accredited by the Accreditation Council for Graduate Medical Education. Reliable methods of assessing resident competency in SBP have not been described in the medical literature.
  
  
  OBJECTIVE
  To develop and validate an analytic grading rubric to assess pathology residents' analyses of SBP problems in clinical chemistry.
  
  
  METHODS
  Residents were assigned an SBP project based upon unmet clinical needs in the clinical chemistry laboratories. Using an iterative method, we created an analytic grading rubric based on critical thinking principles. Four faculty raters used the SBP project evaluation rubric to independently grade 11 residents' projects during their clinical chemistry rotations. Interrater reliability and Cronbach α were calculated to determine the reliability and validity of the rubric. Project mean scores and range were also assessed to determine whether the rubric differentiated resident critical thinking skills related to the SBP projects.
  
  
  RESULTS
  Overall project scores ranged from 6.56 to 16.50 out of a possible 20 points. Cronbach α ranged from 0.91 to 0.96, indicating that the 4 rubric categories were internally consistent without significant overlap. Intraclass correlation coefficients ranged from 0.63 to 0.81, indicating moderate to strong interrater reliability.
  
  
  CONCLUSIONS
  We report development and statistical analysis of a novel SBP project evaluation rubric. The results indicate the rubric can be used to reliably assess pathology residents' critical thinking skills in SBP.
authors.affiliation: From the Center for Individualized Medicine (Ms Vitek), Emeritus Faculty (Drs Dale and Homburger), the Division of Biostatistics and Informatics (Ms Bryant), and the Department of Laboratory Medicine and Pathology (Drs Saenger and Karon), Mayo Clinic, Rochester, Minnesota.
authors.forename: Carolyn R
authors.initials: CR
authors.lastname: Vitek
authors.forename: Jane C
authors.initials: JC
authors.lastname: Dale
authors.forename: Henry A
authors.initials: HA
authors.lastname: Homburger
authors.forename: Sandra C
authors.initials: SC
authors.lastname: Bryant
authors.forename: Amy K
authors.initials: AK
authors.lastname: Saenger
authors.forename: Brad S
authors.initials: BS
authors.lastname: Karon
mesh_terms: D000068:Accreditation / Q000592:standards; D002624:Chemistry, Clinical / Q000193:education* / Q000592:standards; D002983:Clinical Competence / Q000592:standards; D003479:Curriculum / Q000592:standards; D004521:Educational Measurement / Q000592:standards; D006801:Humans; D007396:Internship and Residency / Q000592:standards; D010338:Pathology, Clinical / Q000193:education* / Q000592:standards; D012955:Societies, Medical; D018511:Systems Integration; D014481:United States