title: The Undergraduate Research Student Self-Assessment (URSSA): Validation for Use in Program Evaluation.
abstract: This article examines the validity of the Undergraduate Research Student Self-Assessment (URSSA), a survey used to evaluate undergraduate research (UR) programs. The underlying structure of the survey was assessed with confirmatory factor analysis; also examined were correlations between different average scores, score reliability, and matches between numerical and textual item responses. The study found that four components of the survey represent separate but related constructs for cognitive skills and affective learning gains derived from the UR experience. Average scores from item blocks formed reliable but moderate to highly correlated composite measures. Additionally, some questions about student learning gains (meant to assess individual learning) correlated to ratings of satisfaction with external aspects of the research experience. The pattern of correlation among individual items suggests that items asking students to rate external aspects of their environment were more like satisfaction ratings than items that directly ask about student skills attainment. Finally, survey items asking about student aspirations to attend graduate school in science reflected inflated estimates of the proportions of students who had actually decided on graduate education after their UR experiences. Recommendations for revisions to the survey include clarified item wording and increasing discrimination between item blocks through reorganization.
authors.affiliation: *Alliance for Technology, Learning and Society, University of Colorado Boulder, Boulder, CO 80309 westont@colorado.edu.
authors.forename: Timothy J
authors.initials: TJ
authors.lastname: Weston
authors.affiliation: Ethnography & Evaluation Research, University of Colorado Boulder, Boulder, CO 80309.
authors.forename: Sandra L
authors.initials: SL
authors.lastname: Laursen
mesh_terms: D002170:Canada; D004493:Education / Q000379:methods*; D004521:Educational Measurement; D006801:Humans; D020407:Internet; D007858:Learning; D015397:Program Evaluation; D015203:Reproducibility of Results; D012106:Research; D012108:Research Personnel; D012586:Science / Q000193:education*; D012647:Self-Assessment; D013334:Students / Q000523:psychology; D014481:United States