title: MRCGP CSA: are the examiners biased, favouring their own by sex, ethnicity, and degree source?
abstract: BACKGROUND
  Concern exists regarding differential performance of candidates in postgraduate clinical assessments by ethnicity, sex, and country of primary qualification. Could examiner bias be responsible?
  
  
  OBJECTIVE
  To explore whether candidate demographics affect examiners' judgements, by investigating candidates' case performances by candidates' and examiners' demographics.
  
  
  METHODS
  Data on 4000 candidates (52,000 cases) sitting the MRCGP clinical skills assessment in 2011-2012.
  
  
  METHODS
  Univariate analyses were undertaken of subgroup performance (male/female, white/black and minority ethnic (BME), UK/non-UK graduates) by parallel examiner demographics. Due to confounding of variables, these were complemented by multivariate ANOVA and multiple regression analyses.
  
  
  RESULTS
  Univariate analysis showed some differences between outcomes between the same-group and other-group examiners: these were contradictory regarding examiners 'favouring their own', for example, males received higher marks from female examiners than from males: maximum effect size was 3.6%. A six-way ANOVA confirmed all three candidate and examiner variables as having significant effects individually, identifying one significant interaction (examiner sex by examiner ethnicity). Stepwise regression showed candidate variables predicting 12% of score variance, parallel examiner demographics adding little (approximately 0.2% of variance). One 'transactional' variable proved significant, explaining 0.06% of score variance.
  
  
  CONCLUSIONS
  Examiners show no general tendency to 'favour their own kind'. With confounding between variables, as far as the impact on candidates' case scores, substantial effects relate to candidate and not examiner characteristics. Candidate-examiner interaction effects were inconsistent in their direction and slight in their calculated impact.
authors.affiliation: MRCGP, Royal College of General Practitioners, London, UK.
authors.forename: Mei Ling
authors.initials: ML
authors.lastname: Denney
authors.forename: Adrian
authors.initials: A
authors.lastname: Freeman
authors.forename: Richard
authors.initials: R
authors.lastname: Wakeford
mesh_terms: D000704:Analysis of Variance; D002983:Clinical Competence / Q000592:standards; D004503:Education, Medical, Graduate; D004521:Educational Measurement / Q000592:standards*; D004522:Educational Status; D005006:Ethnicity / Q000706:statistics & numerical data; D005260:Female; D058006:General Practice / Q000193:education*; D006801:Humans; D007600:Judgment; D008297:Male; D011287:Prejudice; D012044:Regression Analysis; D006113:United Kingdom