title: Assessing doctors' competencies using multisource feedback: validating a Japanese version of the Sheffield Peer Review Assessment Tool (SPRAT).
abstract: OBJECTIVE
  To assess the validity and reliability of the Sheffield Peer Review Assessment Tool (SPRAT) Japanese version for evaluating doctors' competencies using multisource feedback.
  
  
  METHODS
  SPRAT, originally developed in the UK, was translated and validated in three phases: (1) an existing Japanese version of SPRAT was back-translated into English; (2) two expert panel meetings were held to develop and assure content validity in a Japanese setting; (3) the newly devised Japanese SPRAT instrument was tested by a multisource feedback survey, validity was tested using principal component factor analysis, and reliability was assessed using generalisability and decision studies based on generalisability theory.
  
  
  RESULTS
  86 doctors who had been practising for between 2 and 33 years participated as assessees and were evaluated with the SPRAT tool. First, the doctors identified 1019 potential assessors who were each sent SPRAT forms (response rate, 81%). The mean number of assessors per doctor was 9.7 (SD=2.5). The decision study showed that 95% CIs of ± 0.5 were achieved with only 5 assessors. 85 of the 86 doctors achieved scores that could be placed with 95% CI above the 4 expected standard. Doctors received lower scores from more senior assessors (p<0.001) and higher scores from those they had known longer (p<0.001). Scores also varied with the job role (p<0.05).
  
  
  CONCLUSIONS
  Following translation and content validation, the Japanese instrument behaved similarly to the UK tool. Assessor selection remains a primary concern, as the assessment scores are affected by the seniority of the assessor, the length of the assessor-assessee working relationship, and the assessor's job role. Users of the SPRAT tool need to be aware of these limitations when administering the instrument.
authors.affiliation: Department of Health Informatics, School of Public Health, Kyoto University, Kyoto, Japan Department of Health Policy, National Center for Child Health and Development, Tokyo, Japan.
authors.forename: Hatoko
authors.initials: H
authors.lastname: Sasaki
authors.affiliation: The Collaboration for the Advancement of Medical Education Research & Assessment (CAMERA), Plymouth University Peninsula Schools of Medicine & Dentistry, Plymouth University, Plymouth, UK.
authors.forename: Julian
authors.initials: J
authors.lastname: Archer
authors.affiliation: Department of Neuropsychopharmacology, National Center of Neurology and Psychiatry, Kodaira, Japan.
authors.forename: Naohiro
authors.initials: N
authors.lastname: Yonemoto
authors.affiliation: Department of Health Policy, National Center for Child Health and Development, Tokyo, Japan.
authors.forename: Rintaro
authors.initials: R
authors.lastname: Mori
authors.affiliation: Department of Neonatology, Tokyo Women's Medical University, Maternal and Perinatal Center, Tokyo, Japan.
authors.forename: Toshihiko
authors.initials: T
authors.lastname: Nishida
authors.affiliation: Department of Neonatology, Tokyo Women's Medical University, Maternal and Perinatal Center, Tokyo, Japan.
authors.forename: Satoshi
authors.initials: S
authors.lastname: Kusuda
authors.affiliation: Department of Health Informatics, School of Public Health, Kyoto University, Kyoto, Japan.
authors.forename: Takeo
authors.initials: T
authors.lastname: Nakayama
mesh_terms: D002983:Clinical Competence / Q000592:standards*; D005260:Female; D000068156:Formative Feedback; D006801:Humans; D007564:Japan; D008297:Male; D018024:Peer Review, Health Care / Q000379:methods*; D010865:Pilot Projects; D015203:Reproducibility of Results
keywords: EDUCATION & TRAINING (see Medical Education & Training); MEDICAL EDUCATION & TRAINING; PAEDIATRICS