title: Programmatic assessment of competency-based workplace learning: when theory meets practice.
abstract: BACKGROUND
  In competency-based medical education emphasis has shifted towards outcomes, capabilities, and learner-centeredness. Together with a focus on sustained evidence of professional competence this calls for new methods of teaching and assessment. Recently, medical educators advocated the use of a holistic, programmatic approach towards assessment. Besides maximum facilitation of learning it should improve the validity and reliability of measurements and documentation of competence development. We explored how, in a competency-based curriculum, current theories on programmatic assessment interacted with educational practice.
  
  
  METHODS
  In a development study including evaluation, we investigated the implementation of a theory-based programme of assessment. Between April 2011 and May 2012 quantitative evaluation data were collected and used to guide group interviews that explored the experiences of students and clinical supervisors with the assessment programme. We coded the transcripts and emerging topics were organised into a list of lessons learned.
  
  
  RESULTS
  The programme mainly focuses on the integration of learning and assessment by motivating and supporting students to seek and accumulate feedback. The assessment instruments were aligned to cover predefined competencies to enable aggregation of information in a structured and meaningful way. Assessments that were designed as formative learning experiences were increasingly perceived as summative by students. Peer feedback was experienced as a valuable method for formative feedback. Social interaction and external guidance seemed to be of crucial importance to scaffold self-directed learning. Aggregating data from individual assessments into a holistic portfolio judgement required expertise and extensive training and supervision of judges.
  
  
  CONCLUSIONS
  A programme of assessment with low-stakes assessments providing simultaneously formative feedback and input for summative decisions proved not easy to implement. Careful preparation and guidance of the implementation process was crucial. Assessment for learning requires meaningful feedback with each assessment. Special attention should be paid to the quality of feedback at individual assessment moments. Comprehensive attention for faculty development and training for students is essential for the successful implementation of an assessment programme.
authors.affiliation: Quality Improvement in Veterinary Education, Faculty of Veterinary Medicine, Utrecht University, Utrecht, The Netherlands. G.J.Bok@uu.nl.
authors.forename: Harold G J
authors.initials: HG
authors.lastname: Bok
authors.forename: Pim W
authors.initials: PW
authors.lastname: Teunissen
authors.forename: Robert P
authors.initials: RP
authors.lastname: Favier
authors.forename: Nancy J
authors.initials: NJ
authors.lastname: Rietbroek
authors.forename: Lars F H
authors.initials: LF
authors.lastname: Theyse
authors.forename: Harold
authors.initials: H
authors.lastname: Brommer
authors.forename: Jan C M
authors.initials: JC
authors.lastname: Haarhuis
authors.forename: Peter
authors.initials: P
authors.lastname: van Beukelen
authors.forename: Cees P M
authors.initials: CP
authors.lastname: van der Vleuten
authors.forename: Debbie A D C
authors.initials: DA
authors.lastname: Jaarsma
mesh_terms: D003162:Competency-Based Education / Q000379:methods* / Q000592:standards; D003479:Curriculum; D004501:Education, Medical / Q000379:methods* / Q000592:standards; D004520:Education, Veterinary / Q000379:methods* / Q000592:standards; D004521:Educational Measurement / Q000379:methods / Q000592:standards; D006801:Humans; D009426:Netherlands; D012582:Schools, Veterinary / Q000458:organization & administration / Q000592:standards; D011795:Surveys and Questionnaires
references.citation: J Vet Med Educ. 2011 Fall;38(3):262-9
references.pmid: 22023978
references.citation: Med Educ. 2005 Mar;39(3):309-17
references.pmid: 15733167
references.citation: Med Educ. 2002 Jan;36(1):92-7
references.pmid: 11849528
references.citation: Med Educ. 2008 Nov;42(11):1071-9
references.pmid: 19141009
references.citation: Qual Saf Health Care. 2002 Jun;11(2):148-52
references.pmid: 12448807
references.citation: Med Educ. 2012 Jun;46(6):604-12
references.pmid: 22626052
references.citation: Med Educ. 2007 Dec;41(12):1224-33
references.pmid: 18045373
references.citation: Med Teach. 2007 Nov;29(9):855-71
references.pmid: 18158655
references.citation: Med Teach. 2012;34(3):205-14
references.pmid: 22364452
references.citation: Med Educ. 2002 Oct;36(10):925-30
references.pmid: 12390459
references.citation: JAMA. 2009 Sep 23;302(12):1316-26
references.pmid: 19773567
references.citation: Med Teach. 2007 Sep;29(6):527-45
references.pmid: 17978966
references.citation: Adv Health Sci Educ Theory Pract. 1996 Jan;1(1):41-67
references.pmid: 24178994
references.citation: Acad Med. 2002 May;77(5):361-7
references.pmid: 12010689
references.citation: Adv Health Sci Educ Theory Pract. 2010 Aug;15(3):379-93
references.pmid: 19821042
references.citation: Med Educ. 2005 Feb;39(2):214-20
references.pmid: 15679689
references.citation: Med Teach. 2011;33(6):478-85
references.pmid: 21609177
references.citation: Med Teach. 2003 Jan;25(1):54-62
references.pmid: 14741860
references.citation: Med Educ. 2004 Jul;38(7):767-77
references.pmid: 15200401
references.citation: Med Educ. 1996 Jan;30(1):44-9
references.pmid: 8736188
references.citation: Med Teach. 2010;32(8):638-45
references.pmid: 20662574
references.citation: Med Teach. 2006 Feb;28(1):70-6
references.pmid: 16627328
references.citation: Med Educ. 2013 Mar;47(3):282-91
references.pmid: 23398014
references.citation: Acad Med. 2007 May;82(5):493-502
references.pmid: 17457074
references.citation: Med Teach. 2007 Sep;29(6):591-9
references.pmid: 17922354
references.citation: JAMA. 2002 Jan 9;287(2):226-35
references.pmid: 11779266