title: Validation of an observation instrument for measuring student engagement in health professions settings.
abstract: Documenting student engagement has received increased emphasis in medical schools, as teaching strategies are changing to include more student-to-student interactions. The purpose of this study was to develop and evaluate a measure of student engagement completed by independent observers that would not interfere with student learning time. Data from 3,182 observations completed by nine observers in 32 educational classroom settings with 23 different instructors were used to evaluate the interobserver reliability and gather validity evidence for our observational instrument, named the STROBE. Results indicated that interobserver agreement was good to excellent when observations were conducted simultaneously on randomly selected students in the same classroom (84% average agreement and 0.79 average kappa coefficient) and when observations were conducted on different randomly selected students (79% average agreement). Results also provided strong evidence for validity. Overall, findings indicate that the STROBE demonstrates promise for educational research and evaluation by documenting student engagement in medical education settings.
authors.affiliation: Baylor College of Medicine, USA.
authors.forename: Kimberly J
authors.initials: KJ
authors.lastname: O'Malley
authors.forename: Betty Jeanne
authors.initials: BJ
authors.lastname: Moran
authors.forename: Paul
authors.initials: P
authors.lastname: Haidet
authors.forename: Charles L
authors.initials: CL
authors.lastname: Seidel
authors.forename: Virginia
authors.initials: V
authors.lastname: Schneider
authors.forename: Robert O
authors.initials: RO
authors.lastname: Morgan
authors.forename: P Adam
authors.initials: PA
authors.lastname: Kelly
authors.forename: Boyd
authors.initials: B
authors.lastname: Richards
mesh_terms: D006281:Health Occupations / Q000193:education*; D006801:Humans; D015588:Observer Variation; D015203:Reproducibility of Results; D013336:Students, Health Occupations