title: Development and initial validation of a program director's evaluation form for third-year residents.
abstract: BACKGROUND
  Using a previously developed postgraduate year (PGY)-1 program director's evaluation survey, we developed a parallel form to assess more senior residents (PGY-3). The PGY-3 survey, which aligns with the core competencies established by the Accreditation Council for Graduate Medical Education, also includes items that reflect our institution's military-unique context.
  
  
  OBJECTIVE
  To collect feasibility, reliability, and validity evidence for the new PGY-3 evaluation.
  
  
  METHODS
  We collected PGY-3 data from program directors who oversee the education of military residents. The current study's cohort consisted of Uniformed Services University of the Health Sciences students graduating in 2008, 2009, and 2010. We performed exploratory factor analysis (EFA) to examine the internal structure of the survey and subjected each of the factors identified in the EFA to an internal consistency reliability analysis. We then performed correlation analysis to examine the relationships between PGY-3 ratings and several
  
  
  RESULTS
  PGY-1 ratings, cumulative medical school grade point average (GPA), and performance on U.S. Medical Licensing Examinations (USMLE) Step 1, Step 2 Clinical Knowledge, and Step 3.
  
  
  RESULTS
  Of the 510 surveys we distributed, 388 (76%) were returned. Results from the EFA suggested four factors: "Medical Expertise," "Professionalism," "Military-unique Practice," and "Systems-based Practice." Scores on these four factors showed good internal consistency reliability, as measured by Cronbach's α (α ranged from 0.92 to 0.98). Further, as expected, "Medical Expertise" and "Professionalism" had small to moderate correlations with cumulative medical school GPA and performance on the USMLE Step examinations.
  
  
  CONCLUSIONS
  The new program director's evaluation survey instrument developed in this study appears to be feasible, and the scores that emerged have reasonable evidence of reliability and validity in a sample of third-year residents.
authors: Department of Medicine, Uniformed Services University of the Health Sciences, 4301 Jones Bridge Road, Bethesda, MD 20814.
authors: Anthony R
authors: AR
authors: Artino
authors: Department of Medicine, Uniformed Services University of the Health Sciences, 4301 Jones Bridge Road, Bethesda, MD 20814.
authors: Ting
authors: T
authors: Dong
authors: Department of Preventive Medicine and Biometrics, Uniformed Services University of the Health Sciences, 4301 Jones Bridge Road, Bethesda, MD 20814.
authors: David F
authors: DF
authors: Cruess
authors: F. Edward Hébert School of Medicine, Uniformed Services University of the Health Sciences, 4301 Jones Bridge Road, Bethesda, MD 20814.
authors: William R
authors: WR
authors: Gilliland
authors: Department of Medicine, Uniformed Services University of the Health Sciences, 4301 Jones Bridge Road, Bethesda, MD 20814.
authors: Steven J
authors: SJ
authors: Durning
mesh_terms: D000328:Adult; D002983:Clinical Competence; D004503:Education, Medical, Graduate / Q000379:methods / Q000592:standards*; D004521:Educational Measurement / Q000379:methods*; D005163:Factor Analysis, Statistical; D005180:Faculty, Medical; D005240:Feasibility Studies; D005260:Female; D006801:Humans; D007396:Internship and Residency / Q000706:statistics & numerical data*; D008007:Licensure, Medical / Q000706:statistics & numerical data; D008297:Male; D008887:Military Medicine / Q000193:education; D015203:Reproducibility of Results; D012577:Schools, Medical; D011795:Surveys and Questionnaires / Q000592:standards*; D014481:United States