title: A competency based selection procedure for Dutch postgraduate GP training: a pilot study on validity and reliability.
abstract: Abstract Background: Historically, semi-structured interviews (SSI) have been the core of the Dutch selection for postgraduate general practice (GP) training. This paper describes a pilot study on a newly designed competency-based selection procedure that assesses whether candidates have the competencies that are required to complete GP training.
  
  
  OBJECTIVE
  The objective was to explore reliability and validity aspects of the instruments developed.
  
  
  METHODS
  The new selection procedure comprising the National GP Knowledge Test (LHK), a situational judgement tests (SJT), a patterned behaviour descriptive interview (PBDI) and a simulated encounter (SIM) was piloted alongside the current procedure. Forty-seven candidates volunteered in both procedures. Admission decision was based on the results of the current procedure.
  
  
  RESULTS
  Study participants did hardly differ from the other candidates. The mean scores of the candidates on the LHK and SJT were 21.9 % (SD 8.7) and 83.8% (SD 3.1), respectively. The mean self-reported competency scores (PBDI) were higher than the observed competencies (SIM): 3.7(SD 0.5) and 2.9(SD 0.6), respectively. Content-related competencies showed low correlations with one another when measured with different instruments, whereas more diverse competencies measured by a single instrument showed strong to moderate correlations. Moreover, a moderate correlation between LHK and SJT was found. The internal consistencies (intraclass correlation, ICC) of LHK and SJT were poor while the ICC of PBDI and SIM showed acceptable levels of reliability.
  
  
  CONCLUSIONS
  Findings on content validity and reliability of these new instruments are promising to realize a competency based procedure. Further development of the instruments and research on predictive validity should be pursued.
authors.affiliation: Julius Centre for Health Sciences and Primary Care, University Medical Centre Utrecht , the Netherlands.
authors.forename: Margit I
authors.initials: MI
authors.lastname: Vermeulen
authors.forename: Fred
authors.initials: F
authors.lastname: Tromp
authors.forename: Nicolaas P A
authors.initials: NP
authors.lastname: Zuithoff
authors.forename: Ron H M
authors.initials: RH
authors.lastname: Pieters
authors.forename: Roger A M J
authors.initials: RA
authors.lastname: Damoiseaux
authors.forename: Marijke M
authors.initials: MM
authors.lastname: Kuyvenhoven
mesh_terms: D000328:Adult; D002983:Clinical Competence; D004503:Education, Medical, Graduate / Q000379:methods*; D005194:Family Practice / Q000193:education*; D005260:Female; D006801:Humans; D008297:Male; D009426:Netherlands; D010865:Pilot Projects; D015203:Reproducibility of Results; D012189:Retrospective Studies
keywords: Medical education; general practice; post graduate training; selection