Using postgraduate students' evaluations of research experience to benchmark departments and faculties: issues and challenges.
BACKGROUND
The introduction of the Australian Research Training Scheme has been a strong reason for assuring the quality of the research higher degree (RHD) experience; if students experience poor supervision, an unsupportive climate, and inadequate infrastructure, prior research suggests RHD students will be less likely to complete their degree, with negative consequences for the student, the university, and society at large.


OBJECTIVE
The present study examines the psychometric properties of a survey instrument, the Student Research Experience Questionnaire (SREQ), for measuring the RHD experience of currently enrolled students. The core scales of the SREQ focus on student experiences of Supervision; Infrastructure; Intellectual and Social Climate; and Generic Skills Development.


METHODS
Participants were 2,213 postgraduate research students of a large, research-intensive Australian university.


RESULTS
Preliminary factor analyses conducted at the student level supported the a priori four factors that the SREQ was designed to measure. However, multi-level analyses indicated that there was almost no differentiation between faculties or departments nested with faculties, suggesting that the SREQ responses are not appropriate for benchmarking faculties or departments.


CONCLUSIONS
Consistent with earlier research based on comparisons across universities, the SREQ is shown to be almost completely unreliable in terms of benchmarking faculties or departments within a university.
Ginns
Paul
P
Faculty of Education and Social Work, University of Sydney, Sydney, New South Wales, Australia. p.ginns@usyd.edu.au
Marsh
Herbert W
HW
Behnia
Masud
M
Cheng
Jacqueline H S
JH
Scalas
L Francesca
LF
D000328:Adult; D001290:Attitude; D001315:Australia; D019985:Benchmarking; D003479:Curriculum; D004500:Education, Graduate; D005178:Faculty; D005260:Female; D006801:Humans; D008297:Male; D011594:Psychometrics; D012106:Research / Q000193:education*; D011795:Surveys and Questionnaires