title: Validation of the behavior and concept based assessment of professionalism competence in postgraduate first-year residents.
abstract: BACKGROUND
  The evaluation of professional behaviors and concepts of postgraduate first-year (PGY1) residents has been identified as an area for development. This study examined the efficiency of the professionalism-assessing objective structured clinical examination (OSCE), 360° evaluation, and mini-Clinical Examination Exercise scores (mini-CEX; p-OSCE, p-360° evaluation, and p-mini-CEX scores).
  
  
  METHODS
  Between January 2009 and January 2012, 189 PGY1 residents were evaluated for behavior- and concept-based professionalism competence based on the above three methods using two checklists unique to each case. Data were analyzed for reliability, inter-rater agreement, interval changes, and gender-related difference for each method.
  
  
  RESULTS
  The test reliabilities of p-OSCE, p-360° evaluation, and p-mini-CEX were acceptable. Further, the reliability of concept and combined p-OSCE was higher than that of behavior p-OSCE. In addition, the concept OSCE p-scores and behavior 360° evaluation p-scores were significantly improved after 6 months of training. The inter-rater agreements were relatively good in p-OSCE and p-360° evaluation. Interestingly, male PGY1 residents had higher behavior 360° evaluation p-scores from nurses than those of females, whereas female PGY1 residents had higher behavior 360° evaluation p-scores from patients than those of males. Behavior and concept OSCE p-scores were positively correlated with behavior 360° evaluation p-scores. In comparison with p-360° evaluation, the combination of p-360° evaluation + OSCE + mini-CEX significantly increases their reliabilities.
  
  
  CONCLUSIONS
  The current study suggests that the p-OSCE, p-360° evaluation, and p-mini-CEX are feasible methods for evaluating professionalism in clinical training of PGY1 residents. Combination of the above three evaluations, participation, and support from multiple constituencies and multiple representatives provides good reliability and adds credibility in the assessment of professionalism competence.
authors.affiliation: Division of General Medicine, Department of Medicine, Taipei Veterans General Hospital, Taipei, Taiwan, ROC.
authors.forename: Ying-Ying
authors.initials: YY
authors.lastname: Yang
authors.forename: Fa-Yauh
authors.initials: FY
authors.lastname: Lee
authors.forename: Hui-Chi
authors.initials: HC
authors.lastname: Hsu
authors.forename: Wen-Shin
authors.initials: WS
authors.lastname: Lee
authors.forename: Chiao-Lin
authors.initials: CL
authors.lastname: Chuang
authors.forename: Ching-Chih
authors.initials: CC
authors.lastname: Chang
authors.forename: Teh-Ia
authors.initials: TI
authors.lastname: Huo
authors.forename: Yi-Hsiang
authors.initials: YH
authors.lastname: Huang
authors.forename: Chia-Chang
authors.initials: CC
authors.lastname: Huang
authors.forename: Chin-Chou
authors.initials: CC
authors.lastname: Huang
mesh_terms: D002983:Clinical Competence; D004521:Educational Measurement / Q000379:methods*; D005260:Female; D006801:Humans; D007396:Internship and Residency; D008297:Male; D015203:Reproducibility of Results; D012727:Sex Characteristics