title: Clinical audit project in undergraduate medical education curriculum: an assessment validation study.
abstract: OBJECTIVE
  To evaluate the merit of the Clinical Audit Project (CAP) in an assessment program for undergraduate medical education using a systematic assessment validation framework.
  
  
  METHODS
  A cross-sectional assessment validation study at one medical school in Western Australia, with retrospective qualitative analysis of the design, development, implementation and outcomes of the CAP, and quantitative analysis of assessment data from four cohorts of medical students (2011- 2014).
  
  
  RESULTS
  The CAP is fit for purpose with clear external and internal alignment to expected medical graduate outcomes.  Substantive validity in students' and examiners' response processes is ensured through relevant methodological and cognitive processes. Multiple validity features are built-in to the design, planning and implementation process of the CAP.  There is evidence of high internal consistency reliability of CAP scores (Cronbach's alpha > 0.8) and inter-examiner consistency reliability (intra-class correlation>0.7). Aggregation of CAP scores is psychometrically sound, with high internal consistency indicating one common underlying construct.  Significant but moderate correlations between CAP scores and scores from other assessment modalities indicate validity of extrapolation and alignment between the CAP and the overall target outcomes of medical graduates.  Standard setting, score equating and fair decision rules justify consequential validity of CAP scores interpretation and use.
  
  
  CONCLUSIONS
  This study provides evidence demonstrating that the CAP is a meaningful and valid component in the assessment program. This systematic framework of validation can be adopted for all levels of assessment in medical education, from individual assessment modality, to the validation of an assessment program as a whole.
authors.affiliation: School of Medicine Fremantle, The University of Notre Dame Australia, Fremantle, Western Australia, Australia.
authors.forename: Elina
authors.initials: E
authors.lastname: Tor
authors.affiliation: Learning and Teaching Office, The University of Notre Dame Australia, Fremantle, Western Australia, Australia.
authors.forename: Carole
authors.initials: C
authors.lastname: Steketee
authors.affiliation: School of Medicine Fremantle, The University of Notre Dame Australia, Fremantle, Western Australia, Australia.
authors.forename: Donna
authors.initials: D
authors.lastname: Mak
mesh_terms: D054869:Clinical Audit; D002983:Clinical Competence; D003430:Cross-Sectional Studies; D003479:Curriculum / Q000592:standards*; D004504:Education, Medical, Undergraduate / Q000592:standards*; D004521:Educational Measurement / Q000379:methods* / Q000592:standards; D006801:Humans; D015397:Program Evaluation; D015203:Reproducibility of Results; D012189:Retrospective Studies; D012577:Schools, Medical / Q000592:standards; D013337:Students, Medical; D011795:Surveys and Questionnaires; D014904:Western Australia
keywords: assessment in medical education; assessment validation; population and preventive health curriculum; quality and safety curriculum; validity