title: Republished paper: Assuring validity of multisource feedback in a national programme.
abstract: OBJECTIVE
  To report the evidence for and challenges to the validity of Sheffield Peer Review Assessment Tool (SPRAT) with paediatric Specialist Registrars (SpRs) across the UK as part of Royal College of Paediatrics and Child Health workplace based assessment programme.
  
  
  METHODS
  Quality assurance analysis, including generalisability, of a multisource feedback questionnaire study.
  
  
  METHODS
  All UK Deaneries between August 2005 and May 2006.
  
  
  METHODS
  577 year 2 and 4 Paediatric SpRs.
  
  
  METHODS
  Trainees were evaluated using SPRAT sent to clinical colleagues of their choosing. Data were analysed reporting totals, means and SD, and year groups were compared using independent t tests. A factor analysis was undertaken. Reliability was estimated using generalisability theory. Trainee and assessor demographic details were explored to try to explain variability in scores.
  
  
  METHODS
  4770 SPRAT assessments were provided about 577 paediatric SpRs. The mean scores between years were significantly different (Year 2 mean=5.08, SD=0.34, Year 4 mean=5.18, SD=0.34). A factor analysis returned a two-factor solution, clinical care and psychosocial skills. The 95% CI showed that trainees scoring ≥4.3 with nine assessors can be seen as achieving satisfactory performance with statistical confidence. Consultants marked trainees significantly lower (t=-4.52) whereas Senior House Officers and Foundation doctors scored their SpRs significantly higher (SHO t=2.06, Foundation t=2.77).
  
  
  CONCLUSIONS
  There is increasing evidence that multisource feedback (MSF) assesses two generic traits, clinical care and psychosocial skills. The validity of MSF is threatened by systematic bias, namely leniency bias and the seniority of assessors. Unregulated self-selection of assessors needs to end.
authors.affiliation: Peninsula College of Medicine and Dentistry, University of Plymouth, UK. julian.archer@pms.ac.uk
authors.forename: Julian
authors.initials: J
authors.lastname: Archer
authors.forename: Mary
authors.initials: M
authors.lastname: McGraw
authors.forename: Helena
authors.initials: H
authors.lastname: Davies
mesh_terms: D002983:Clinical Competence; D004503:Education, Medical, Graduate / Q000379:methods*; D004521:Educational Measurement / Q000379:methods*; D004650:Employee Performance Appraisal / Q000379:methods; D030141:Feedback, Psychological; D005260:Female; D006801:Humans; D008297:Male; D008505:Medical Staff, Hospital / Q000193:education / Q000592:standards; D010372:Pediatrics / Q000193:education* / Q000592:standards; D010380:Peer Review / Q000379:methods; D015203:Reproducibility of Results; D006113:United Kingdom