A generalizability study of student ratings in logbooks to assess the educational process of clinical learning.
OBJECTIVE
To investigate the number of student raters necessary for a reliable estimate of how educators approach the process of clinical learning (i.e., levels of student participation and educator feedback) in clinical settings.


METHODS
Third-year medical students documented clinical experiences across seven core clerkships by using an online system. The authors asked students to identify their clinical encounters and to respond to questions about the educational process. The authors' hypothesis was that educators would have different preferences with regard to students' participation in the clinical experience, as well as different preferences with regard to offering their own feedback to students. The authors conducted a generalizability study to ascertain the variance components for the responses to the educator feedback and the student participation log questions. This generalizability study design was a rater-nested-with-person design, noted as r:p. The authors also conducted a decision study to calculate projected G (generalizability) coefficients across different numbers of raters. The results can help determine the minimum number of raters required to reach a reliable estimate of whether a faculty member offers feedback and invites students to participate in the clinical encounter.


RESULTS
The generalizability study found that most of the variance components were attributed to educators. The decision study found that the G coefficients reached acceptable levels of reliability when at least five raters completed clinical encounters with an educator.


CONCLUSIONS
The results suggest that students' responses reliably distinguished the levels of educator feedback and encouragement that were needed for student participation in clinical encounters.
Cottrell
Scott
S
West Virginia University School of Medicine, Martinsburg, USA. scottrell@hsc.wvu.edu <scottrell@hsc.wvu.edu>
Smith
Templeton
T
Ferrari
Norman
N
Cannarella Lorenzetti
Rosemarie
R
Jacques
Mitch
M
Shumway
Jamie
J
D002982:Clinical Clerkship / Q000592:standards*; D002983:Clinical Competence / Q000592:standards; D003162:Competency-Based Education / Q000592:standards; D030141:Feedback, Psychological; D006801:Humans; D020407:Internet; D007858:Learning; D015397:Program Evaluation / Q000379:methods; D013337:Students, Medical; D011795:Surveys and Questionnaires; D014903:West Virginia