A study of a multi-source feedback system for international medical graduates holding defined licences.
OBJECTIVE
To develop and assess the feasibility and psychometric properties of multi-source feedback questionnaires to monitor international medical graduates practising in Canada under 'defined' licences.


METHODS
Four questionnaires (patient, co-worker, colleague and self) were developed and administered in 2 phases through paper-based and telephone or Internet formats. Reliability was assessed with Cronbach's alpha and generalisability coefficient analyses. Validity was established through mean ratings, 'unable to respond' rates and factor analyses.


RESULTS
A total of 37 doctors participated in the 2 phases. Overall response rates were 70% for patients, 86% for co-workers, 72% for medical colleagues and 92% for self, with response rates higher for the paper-based format than the Internet and phone formats. The instruments had high internal consistency reliability, with Cronbach's alphas of 0.83 for self-assessment and > 0.90 for the other instruments. The generalisability coefficients were Ep(2) = 0.71 for 25 patients on a 13-item survey, Ep(2) = 0.59 for 8 co-workers on a 13-item survey, and Ep(2) = 0.67 for 8 colleagues on a 21-item questionnaire. The range of mean scores was narrow (between 4 and 5) for all items and all surveys. The factor analyses identified that 2 factors accounted for 70% or more of the variance for the patient and colleague surveys and 60% of the variance for the co-worker survey.


CONCLUSIONS
These data suggest that the instruments have reasonable psychometric properties. Traditional survey methods (i.e. paper-based) yielded better results than Internet or phone methods for this group of doctors.
Department of Continuing Medical Education and Professional Development, University of Calgary, Calgary, Alberta, Canada. lockyer@ucalgary.ca
Jocelyn
J
Lockyer
David
D
Blackmore
Herta
H
Fidler
Rod
R
Crutcher
Brian
B
Salte
Karen
K
Shaw
Bryan
B
Ward
Norman
N
Wolfish
D000068:Accreditation; D002170:Canada; D002983:Clinical Competence / Q000592:standards*; D005240:Feasibility Studies; D005246:Feedback; D005550:Foreign Medical Graduates / Q000592:standards*; D006801:Humans; D008004:Licensure; D010865:Pilot Projects; D011594:Psychometrics; D013663:Teaching / Q000379:methods*