title: A BEME (Best Evidence in Medical Education) systematic review of the use of workplace-based assessment in identifying and remediating poor performance among postgraduate medical trainees.
abstract: BACKGROUND
  Workplace-based assessments were designed to facilitate observation and structure feedback on the performance of trainees in real-time clinical settings and scenarios. Research in workplace-based assessments has primarily centred on understanding psychometric qualities and performance improvement impacts of trainees generally. An area that is far less understood is the use of workplace-based assessments for trainees who may not be performing at expected or desired standards, referred to within the literature as trainees 'in difficulty' or 'underperforming'. In healthcare systems that increasingly depend on service provided by junior doctors, early detection (and remediation) of poor performance is essential. However, barriers to successful implementation of workplace-based assessments (WBAs) in this context include a misunderstanding of the use and purpose of these formative assessment tools. This review aims to explore the impact - or effectiveness - of workplace-based assessment on the identification of poor performance and to determine those conditions that support and enable detection, i.e. whether by routine or targeted use where poor performance is suspected. The review also aims to explore what effect (if any) the use of WBA may have on remediation or on changing clinical practice. The personal impact of the detection of poor performance on trainees and/or trainers may also be explored.
  
  
  METHODS
  Using BEME (Best Evidence in Medical Education) Collaboration review guidelines, nine databases will be searched for English-language records. Studies examining interventions for workplace-based assessment either routinely or in relation to poor performance will be included. Independent agreement (kappa .80) will be achieved using a randomly selected set of records prior to commencement of screening and data extraction using a BEME coding sheet modified as applicable (Buckley et al., Med Teach 31:282-98, 2009) as this has been used in previous WBA systematic reviews (Miller and Archer, BMJ doi:10.1136/bmj.c5064, 2010) allowing for more rigorous comparisons with the published literature. Educational outcomes will be evaluated using Kirkpatrick's framework of educational outcomes using Barr's adaptations (Barr et al., Evaluations of interprofessional education; a United Kingdom review of health and social care, 2000) for medical education research.
  
  
  CONCLUSIONS
  Our study will contribute to an ongoing international debate regarding the applicability of workplace-based assessments as a meaningful formative assessment approach within the context of postgraduate medical education.
  
  
  BACKGROUND
  The review has been registered by the BEME Collaboration www.bemecollaboration.org .
authors.affiliation: Education and Professional Development Unit, Royal College of Physicians of Ireland, Frederick House, 19 South Frederick St, Dublin 2, Ireland. aileenbarrett@ymail.com.|School of Medicine, College of Medicine and Health Sciences, Brookfield Health Sciences Complex, University College Cork, Corcaigh, Ireland. aileenbarrett@ymail.com.
authors.forename: Aileen
authors.initials: A
authors.lastname: Barrett
authors.affiliation: Discipline of Physiotherapy, Department of Clinical Therapies, Faculty of Education and Health Sciences, University of Limerick, Limerick, Ireland. Rose.Galvin@ul.ie.
authors.forename: Rose
authors.initials: R
authors.lastname: Galvin
authors.affiliation: Center for Medical Education, Faculty of Medicine, McGill University, Lady Meredith House, 1110 Pine Avenue West, Montreal, Quebec, H3A 1A3, Canada. yvonne.steinert@mcgill.ca.
authors.forename: Yvonne
authors.initials: Y
authors.lastname: Steinert
authors.affiliation: Faculty of Health, Medicine and Life Sciences, University of Maastricht, Universiteitssingel 60, Maastricht, 6229, ER, Netherlands. a.scherpbier@maastrichtuniversity.nl.
authors.forename: Albert
authors.initials: A
authors.lastname: Scherpbier
authors.affiliation: Education and Professional Development Unit, Royal College of Physicians of Ireland, Frederick House, 19 South Frederick St, Dublin 2, Ireland. annoshaughnessy@rcpi.ie.
authors.forename: Ann
authors.initials: A
authors.lastname: O'Shaughnessy
authors.affiliation: School of Medicine, College of Medicine and Health Sciences, Brookfield Health Sciences Complex, University College Cork, Corcaigh, Ireland. m.horgan@ucc.ie.
authors.forename: Mary
authors.initials: M
authors.lastname: Horgan
authors.affiliation: Research Unit, Royal College of Physicians and Surgeons of Canada, 774 Echo Drive, Ottawa, ON, K1S 5N8, Canada. thorsley@royalcollege.ca.|Faculty of Medicine, University of Ottawa, 451 Smyth Road, Ottawa, Ontario, K1H 8M5, Canada. thorsley@royalcollege.ca.
authors.forename: Tanya
authors.initials: T
authors.lastname: Horsley
mesh_terms: D002983:Clinical Competence; D003299:Cooperative Behavior; D004521:Educational Measurement / Q000379:methods*; D000068156:Formative Feedback; D006801:Humans; D007396:Internship and Residency; D017132:Workplace