title: Assessing suitability for a problem-based learning curriculum: evaluating a new student selection instrument.
abstract: BACKGROUND
  A new student selection instrument has been designed to assess candidate suitability for a problem-based learning, small group curriculum.
  
  
  OBJECTIVE
  To evaluate the performance of the new teamwork selection instrument in terms of its discriminatory power, fairness, validity, reliability and acceptability among candidates.
  
  
  METHODS
  A sample of 69 volunteer candidates attending for interview formed 13 teams of 5 or 6 candidates each. Each candidate was assessed independently by 2 assessors. Candidate performance in the exercise was used for instrument evaluation purposes only.
  
  
  RESULTS
  The instrument demonstrated good item discrimination (item-total correlations r = between 0.75 and 0.83, P <0.01); the potential for good agreement between raters (63% agreement, weighted kappa = 0.38, P <0.01); strong internal consistency reliability (Cronbach's alpha = 0.93), and good acceptability among candidates. No sources of assessment bias were identified on the basis of candidates' age (univariate anova F = 0.43, P >0.05), gender (unrelated samples t-test F = 1.2, P >0.05) or socioeconomic background (univariate anova F = 0.85, P >0.05). There was no statistically significant relationship between the candidates' performance in the new exercise and their performance in the standardised formal interview (r = - 0.37, P >0.05); the instrument had limited predictive validity, and some of the measured attributes require conceptual clarification. Discussion Statistical and conceptual analysis highlights the scope for development in the teamwork exercise. The exercise appears to be well suited to assessing candidate suitability for a problem-based learning curriculum.
authors.affiliation: Institute of Clinical Education, Peninsula Medical School, Exeter, UK. suzanne.chamberlain@pms.ac.uk
authors.forename: Suzanne E
authors.initials: SE
authors.lastname: Chamberlain
authors.forename: Judy
authors.initials: J
authors.lastname: Searle
mesh_terms: D000293:Adolescent; D000328:Adult; D001290:Attitude; D004504:Education, Medical, Undergraduate; D006801:Humans; D015588:Observer Variation; D011237:Predictive Value of Tests; D018794:Problem-Based Learning; D012570:School Admission Criteria; D012577:Schools, Medical; D012680:Sensitivity and Specificity; D006113:United Kingdom