Residency directors' predictions of candidates' performances on a licensing examination.
OBJECTIVE
To look at how well residency directors in family medicine predict the performances of their candidates on the Quebec Licensing Examination (QLEx) for family physicians.


METHODS
The four family medicine program directors in Quebec were asked in the spring of 1992 to identify the ten residents from their own programs who would get the highest marks on the QLEx and the ten who would get the lowest marks. From the results of these candidates, and those of the intermediate groups defined by default, the prediction abilities of the program directors were assessed.


RESULTS
Descriptive statistics showed that the program directors had difficulties discriminating among the different groups and tended to collapse the predicted scores toward the total mean. Analysis-of-variance studies confirmed the absence of difference between the predicted weak and intermediate groups as well as between the predicted intermediate and strong groups for each program and for each QLEx component. Regression analysis as well as eta 2 studies showed that the program directors' prediction abilities were low for all components and represented less than 25% of the explained variance of the QLEx scores.


CONCLUSIONS
The residency directors did not accurately categorize their residents' performances on the QLEx. Both the evaluations of program directors and terminal examination results are complementary approaches to clinical competence assessment and should be used for licensure.
Brailovsky
C A
CA
Centre d'évaluation des sciences de la santé, Laval University Faculty of Medicine, Quebec, Canada.
Grand'Maison
P
P
Lescop
J
J
D004521:Educational Measurement; D005194:Family Practice / Q000193:education*; D007396:Internship and Residency; D008007:Licensure, Medical; D008486:Physician Executives; D011792:Quebec