title: Feasibility and reliability of an in-training assessment programme in an undergraduate clerkship.
abstract: BACKGROUND
  Structured assessment, embedded in a training programme, with systematic observation, feedback and appropriate documentation may improve the reliability of clinical assessment. This type of assessment format is referred to as in-training assessment (ITA). The feasibility and reliability of an ITA programme in an internal medicine clerkship were evaluated. The programme comprised 4 ward-based test formats and 1 outpatient clinic-based test format. Of the 4 ward-based test formats, 3 were single-sample tests, consisting of 1 student-patient encounter, 1 critical appraisal session and 1 case presentation. The other ward-based test and the outpatient-based test were multiple sample tests, consisting of 12 ward-based case write-ups and 4 long cases in the outpatient clinic. In all the ITA programme consisted of 19 assessments.
  
  
  METHODS
  During 41 months, data were collected from 119 clerks. Feasibility was defined as over two thirds of the students obtaining 19 assessments. Reliability was estimated by performing generalisability analyses with 19 assessments as items and 5 test formats as items.
  
  
  RESULTS
  A total of 73 students (69%) completed 19 assessments. Reliability expressed by the generalisability coefficients was 0.81 for 19 assessments and 0.55 for 5 test formats.
  
  
  CONCLUSIONS
  The ITA programme proved to be feasible. Feasibility may be improved by scheduling protected time for assessment for both students and staff. Reliability may be improved by more frequent use of some of the test formats.
authors.affiliation: Skills Training Department, Vrije Universiteit Medical Centre, Amsterdam, The Netherlands. hem.daelmans@vumc.nl
authors.forename: H E M
authors.initials: HE
authors.lastname: Daelmans
authors.forename: H H
authors.initials: HH
authors.lastname: van der Hem-Stokroos
authors.forename: R J I
authors.initials: RJ
authors.lastname: Hoogenboom
authors.forename: A J J A
authors.initials: AJ
authors.lastname: Scherpbier
authors.forename: C D A
authors.initials: CD
authors.lastname: Stehouwer
authors.forename: C P M
authors.initials: CP
authors.lastname: van der Vleuten
mesh_terms: D002982:Clinical Clerkship / Q000592:standards; D002983:Clinical Competence / Q000592:standards; D003479:Curriculum; D003625:Data Collection; D003718:Denmark; D004504:Education, Medical, Undergraduate / Q000592:standards*; D004521:Educational Measurement / Q000592:standards*; D005240:Feasibility Studies; D006801:Humans; D007318:Inservice Training / Q000379:methods; D015203:Reproducibility of Results