title: A generalizability study of student ratings in logbooks to assess the educational process of clinical learning.
abstract: OBJECTIVE
  To investigate the number of student raters necessary for a reliable estimate of how educators approach the process of clinical learning (i.e., levels of student participation and educator feedback) in clinical settings.
  
  
  METHODS
  Third-year medical students documented clinical experiences across seven core clerkships by using an online system. The authors asked students to identify their clinical encounters and to respond to questions about the educational process. The authors' hypothesis was that educators would have different preferences with regard to students' participation in the clinical experience, as well as different preferences with regard to offering their own feedback to students. The authors conducted a generalizability study to ascertain the variance components for the responses to the educator feedback and the student participation log questions. This generalizability study design was a rater-nested-with-person design, noted as r:p. The authors also conducted a decision study to calculate projected G (generalizability) coefficients across different numbers of raters. The results can help determine the minimum number of raters required to reach a reliable estimate of whether a faculty member offers feedback and invites students to participate in the clinical encounter.
  
  
  RESULTS
  The generalizability study found that most of the variance components were attributed to educators. The decision study found that the G coefficients reached acceptable levels of reliability when at least five raters completed clinical encounters with an educator.
  
  
  CONCLUSIONS
  The results suggest that students' responses reliably distinguished the levels of educator feedback and encouragement that were needed for student participation in clinical encounters.
authors.affiliation: West Virginia University School of Medicine, Martinsburg, USA. scottrell@hsc.wvu.edu <scottrell@hsc.wvu.edu>
authors.forename: Scott
authors.initials: S
authors.lastname: Cottrell
authors.forename: Templeton
authors.initials: T
authors.lastname: Smith
authors.forename: Norman
authors.initials: N
authors.lastname: Ferrari
authors.forename: Rosemarie
authors.initials: R
authors.lastname: Cannarella Lorenzetti
authors.forename: Mitch
authors.initials: M
authors.lastname: Jacques
authors.forename: Jamie
authors.initials: J
authors.lastname: Shumway
mesh_terms: D002982:Clinical Clerkship / Q000592:standards*; D002983:Clinical Competence / Q000592:standards; D003162:Competency-Based Education / Q000592:standards; D030141:Feedback, Psychological; D006801:Humans; D020407:Internet; D007858:Learning; D015397:Program Evaluation / Q000379:methods; D013337:Students, Medical; D011795:Surveys and Questionnaires; D014903:West Virginia