title: Using Precept-Assist® to predict performance on the American Board of Family Medicine In-Training Examination.
abstract: OBJECTIVE
  Precept-Assist® (PA) is a computer-based program developed by the Virtua Family Medicine Residency where residents receive a score on a Likert-type scale from an attending for each precept based on their knowledge base. The purpose of this study was to attempt to validate this program for precepting family medicine residents.
  
  
  METHODS
  This was a validation study. PA and American Board of Family Medicine (ABFM) In-Training Exam (ITE) scores for all residents from a community-based family medicine residency between the years 2002 and 2011 were included (n=216). Pearson correlation coefficients were calculated between PA scores for the second quarter of the academic year (October 1 to December 31) and scores on the ITE. An ROC curve was also created to determine sensitivity and specificity for various PA scores in predicting residents scoring 500 or above on the ITE.
  
  
  RESULTS
  The PA mean (SD) score was 5.18 (0.84) and the ITE mean (SD) score was 425.1 (87.6). The Pearson correlation coefficient between PA and ITE scores was 0.55, which is a moderately positive correlation. The AUC of the ROC curve was 0.783 (95% CI 0.704-0.859). A PA score of 5.5 (between the level of a PGY-2 and PGY-3) was 72% sensitive and 77% specific for scoring 500 or above on the ITE with a positive LR of 3.12.
  
  
  CONCLUSIONS
  There is a significant correlation between PA scores and ABFM In-Training Exam scores. PA is a valid screening tool that can be used as a predictor for future performance in Family Medicine In-Training exams.
authors.affiliation: Virtua Family Medicine Residency, Voorhees, NJ.
authors.forename: Robert E
authors.initials: RE
authors.lastname: Post
authors.forename: Gemma P
authors.initials: GP
authors.lastname: Jamena
authors.forename: James D
authors.initials: JD
authors.lastname: Gamble
mesh_terms: D002983:Clinical Competence; D004521:Educational Measurement / Q000379:methods*; D005180:Faculty, Medical; D005194:Family Practice / Q000193:education*; D006801:Humans; D007396:Internship and Residency; D015203:Reproducibility of Results; D012189:Retrospective Studies; D012984:Software; D013044:Specialty Boards; D014481:United States