title: Using video-cases to assess student reflection: development and validation of an instrument.
abstract: BACKGROUND
  Reflection is a meta-cognitive process, characterized by: 1. Awareness of self and the situation; 2. Critical analysis and understanding of both self and the situation; 3. Development of new perspectives to inform future actions. Assessors can only access reflections indirectly through learners' verbal and/or written expressions. Being privy to the situation that triggered reflection could place reflective materials into context. Video-cases make that possible and, coupled with a scoring rubric, offer a reliable way of assessing reflection.
  
  
  METHODS
  Fourth and fifth year undergraduate medical students were shown two interactive video-cases and asked to reflect on this experience, guided by six standard questions. The quality of students' reflections were scored using a specially developed Student Assessment of Reflection Scoring rubric (StARS®). Reflection scores were analyzed concerning interrater reliability and ability to discriminate between students. Further, the intra-rater reliability and case specificity were estimated by means of a generalizability study with rating and case scenario as facets.
  
  
  RESULTS
  Reflection scores of 270 students ranged widely and interrater reliability was acceptable (Krippendorff's alpha = 0.88). The generalizability study suggested 3 or 4 cases were needed to obtain reliable ratings from 4th year students and ≥ 6 cases from 5th year students.
  
  
  CONCLUSIONS
  Use of StARS® to assess student reflections triggered by standardized video-cases had acceptable discriminative ability and reliability. We offer this practical method for assessing reflection summatively, and providing formative feedback in training situations.
authors.affiliation: Centre for Educational Development, Faculty of Medicine and Health Sciences, Ghent University, Ghent, Belgium. sebastiaan.koole@ugent.be
authors.forename: Sebastiaan
authors.initials: S
authors.lastname: Koole
authors.forename: Tim
authors.initials: T
authors.lastname: Dornan
authors.forename: Leen
authors.initials: L
authors.lastname: Aper
authors.forename: Bram
authors.initials: B
authors.lastname: De Wever
authors.forename: Albert
authors.initials: A
authors.lastname: Scherpbier
authors.forename: Martin
authors.initials: M
authors.lastname: Valcke
authors.forename: Janke
authors.initials: J
authors.lastname: Cohen-Schotanus
authors.forename: Anselme
authors.initials: A
authors.lastname: Derese
mesh_terms: D001364:Awareness; D032882:Comprehension; D004501:Education, Medical / Q000592:standards; D004521:Educational Measurement / Q000379:methods; D006801:Humans; D015588:Observer Variation; D011581:Psychological Tests; D015203:Reproducibility of Results; D013337:Students, Medical / Q000523:psychology*; D014741:Video Recording / Q000379:methods