Composition of the panel of reference for concordance tests: do teaching functions have an impact on examinees' ranks and absolute scores?
BACKGROUND
Concordance tests are designed to assess the component of uncertainty of clinical reasoning. Scoring is based on a comparison of examinees' answers with those of a panel of reference, including their variability. This allows construction of tests that are close to real clinical life, with all its complexity and ambiguity.


OBJECTIVE
This study was carried out to determine the effect of teaching functions of members composing the reference panels on students' scores and ranking.


METHODS
A group of 80 residents in family medicine from a French University (Bobigny) completed a 72-item concordance test. The answers of two panels, each made up of 29 family physicians (teaching function versus non-teaching function), were used to generate the correction keys.


RESULTS
Correlation between the sets of data obtained with the two panels is high (ICC = 0.98). Concordance scores obtained from the teaching-function panel are higher than scores obtained from the non-teaching-function panel (72.0 versus 76.3; p < 0.001). Ranking provided by the two panels was very similar.


CONCLUSIONS
This legitimizes the use of non-teaching physicians on panels. Panel composition influenced absolute score values: Residents showed more concordance with their academic trainers than with community-based physicians.
Charlin
Bernard
B
University of Montreal, Québec, H3C 3J7, Canada. bernard.charlin@umontreal.ca
Gagnon
Robert
R
Sauvé
Evelyne
E
Coletti
Michel
M
D000328:Adult; D002983:Clinical Competence / Q000592:standards*; D004503:Education, Medical, Graduate / Q000592:standards*; D004521:Educational Measurement / Q000379:methods*; D005194:Family Practice / Q000193:education*; D005260:Female; D005602:France; D006801:Humans; D008297:Male; D013663:Teaching / Q000592:standards*; D035501:Uncertainty