Assessing validity in written tests of general practice - exploration by factor analysis of candidate response patterns to paper 1 of the MRCGP examination.
OBJECTIVE
To investigate the content validity of Paper 1 of the MRCGP examination.


METHODS
Exploratory factor analysis was carried out on candidate responses to Paper 1 of the May and October MRCGP examination in 1998. Contribution of each test question across factors was assessed using a pattern matrix of the oblique rotation. Common dimensions and variations between factor sets were identified. Key testing areas were then matched against the 'domains of competence' intended to be assessed by Paper 1 (as defined within the examination blueprint matrix).


RESULTS
Whilst critical appraisal, disease prevention/evidence-based medicine and clinical management emerged as areas tested consistently, content variation was observed between factor sets extracted from both sittings. In addition, some overlap, in terms of domains tested, was seen among other assessment instruments used within the examination.


CONCLUSIONS
Paper 1 conforms to the majority of its stated intentions. However, further development of techniques for investigating validity will be required in order to minimize content variation between both sittings of the examination as well as to help more closely define areas of competence to be tested by Paper 1 of the MRCGP examination.
Munro
N
N
Consultant to the MRCGP examination, Members of the Panel of Examiners, Royal College of General Practitioners, 14 Princes Gate, Hyde Park, London, UK.
Rughani
A
A
Foulkes
J
J
Wilson
A
A
Neighbour
R
R
D004504:Education, Medical, Undergraduate; D004521:Educational Measurement; D005194:Family Practice / Q000193:education*; D006801:Humans; D008131:London; D014956:Writing