title: Composition of the panel of reference for concordance tests: do teaching functions have an impact on examinees' ranks and absolute scores?
abstract: BACKGROUND
  Concordance tests are designed to assess the component of uncertainty of clinical reasoning. Scoring is based on a comparison of examinees' answers with those of a panel of reference, including their variability. This allows construction of tests that are close to real clinical life, with all its complexity and ambiguity.
  
  
  OBJECTIVE
  This study was carried out to determine the effect of teaching functions of members composing the reference panels on students' scores and ranking.
  
  
  METHODS
  A group of 80 residents in family medicine from a French University (Bobigny) completed a 72-item concordance test. The answers of two panels, each made up of 29 family physicians (teaching function versus non-teaching function), were used to generate the correction keys.
  
  
  RESULTS
  Correlation between the sets of data obtained with the two panels is high (ICC = 0.98). Concordance scores obtained from the teaching-function panel are higher than scores obtained from the non-teaching-function panel (72.0 versus 76.3; p < 0.001). Ranking provided by the two panels was very similar.
  
  
  CONCLUSIONS
  This legitimizes the use of non-teaching physicians on panels. Panel composition influenced absolute score values: Residents showed more concordance with their academic trainers than with community-based physicians.
authors.affiliation: University of Montreal, Québec, H3C 3J7, Canada. bernard.charlin@umontreal.ca
authors.forename: Bernard
authors.initials: B
authors.lastname: Charlin
authors.forename: Robert
authors.initials: R
authors.lastname: Gagnon
authors.forename: Evelyne
authors.initials: E
authors.lastname: Sauvé
authors.forename: Michel
authors.initials: M
authors.lastname: Coletti
mesh_terms: D000328:Adult; D002983:Clinical Competence / Q000592:standards*; D004503:Education, Medical, Graduate / Q000592:standards*; D004521:Educational Measurement / Q000379:methods*; D005194:Family Practice / Q000193:education*; D005260:Female; D005602:France; D006801:Humans; D008297:Male; D013663:Teaching / Q000592:standards*; D035501:Uncertainty