title: Using postgraduate students' evaluations of research experience to benchmark departments and faculties: issues and challenges.
abstract: BACKGROUND
  The introduction of the Australian Research Training Scheme has been a strong reason for assuring the quality of the research higher degree (RHD) experience; if students experience poor supervision, an unsupportive climate, and inadequate infrastructure, prior research suggests RHD students will be less likely to complete their degree, with negative consequences for the student, the university, and society at large.
  
  
  OBJECTIVE
  The present study examines the psychometric properties of a survey instrument, the Student Research Experience Questionnaire (SREQ), for measuring the RHD experience of currently enrolled students. The core scales of the SREQ focus on student experiences of Supervision; Infrastructure; Intellectual and Social Climate; and Generic Skills Development.
  
  
  METHODS
  Participants were 2,213 postgraduate research students of a large, research-intensive Australian university.
  
  
  RESULTS
  Preliminary factor analyses conducted at the student level supported the a priori four factors that the SREQ was designed to measure. However, multi-level analyses indicated that there was almost no differentiation between faculties or departments nested with faculties, suggesting that the SREQ responses are not appropriate for benchmarking faculties or departments.
  
  
  CONCLUSIONS
  Consistent with earlier research based on comparisons across universities, the SREQ is shown to be almost completely unreliable in terms of benchmarking faculties or departments within a university.
authors: Faculty of Education and Social Work, University of Sydney, Sydney, New South Wales, Australia. p.ginns@usyd.edu.au
authors: Paul
authors: P
authors: Ginns
authors: Herbert W
authors: HW
authors: Marsh
authors: Masud
authors: M
authors: Behnia
authors: Jacqueline H S
authors: JH
authors: Cheng
authors: L Francesca
authors: LF
authors: Scalas
mesh_terms: D000328:Adult; D001290:Attitude; D001315:Australia; D019985:Benchmarking; D003479:Curriculum; D004500:Education, Graduate; D005178:Faculty; D005260:Female; D006801:Humans; D008297:Male; D011594:Psychometrics; D012106:Research / Q000193:education*; D011795:Surveys and Questionnaires