title: The impact of large scale licensing examinations in highly developed countries: a systematic review.
abstract: BACKGROUND
  To investigate the existing evidence base for the validity of large-scale licensing examinations including their impact.
  
  
  METHODS
  Systematic review against a validity framework exploring: Embase (Ovid Medline); Medline (EBSCO); PubMed; Wiley Online; ScienceDirect; and PsychINFO from 2005 to April 2015. All papers were included when they discussed national or large regional (State level) examinations for clinical professionals, linked to examinations in early careers or near the point of graduation, and where success was required to subsequently be able to practice. Using a standardized data extraction form, two independent reviewers extracted study characteristics, with the rest of the team resolving any disagreement. A validity framework was used as developed by the American Educational Research Association, American Psychological Association, and National Council on Measurement in Education to evaluate each paper's evidence to support or refute the validity of national licensing examinations.
  
  
  RESULTS
  24 published articles provided evidence of validity across the five domains of the validity framework. Most papers (n = 22) provided evidence of national licensing examinations relationships to other variables and their consequential validity. Overall there was evidence that those who do well on earlier or on subsequent examinations also do well on national testing. There is a correlation between NLE performance and some patient outcomes and rates of complaints, but no causal evidence has been established.
  
  
  CONCLUSIONS
  The debate around licensure examinations is strong on opinion but weak on validity evidence. This is especially true of the wider claims that licensure examinations improve patient safety and practitioner competence.
authors.affiliation: Collaboration for the Advancement of Medical Education Research and Assessment, Plymouth University Peninsula Schools of Medicine & Dentistry, Plymouth, Devon, UK. julian.archer@plymouth.ac.uk.
authors.forename: Julian
authors.identifier: http://orcid.org/0000-0001-9983-9655
authors.initials: J
authors.lastname: Archer
authors.affiliation: Collaboration for the Advancement of Medical Education Research and Assessment, Plymouth University Peninsula Schools of Medicine & Dentistry, Plymouth, Devon, UK.
authors.forename: Nick
authors.initials: N
authors.lastname: Lynn
authors.affiliation: Centre for Medical Education, Cardiff University School of Medicine, Heath Park, Cardiff, UK.
authors.forename: Lee
authors.initials: L
authors.lastname: Coombes
authors.affiliation: Collaboration for the Advancement of Medical Education Research and Assessment, Plymouth University Peninsula Schools of Medicine & Dentistry, Plymouth, Devon, UK.
authors.forename: Martin
authors.initials: M
authors.lastname: Roberts
authors.affiliation: Collaboration for the Advancement of Medical Education Research and Assessment, Plymouth University Peninsula Schools of Medicine & Dentistry, Plymouth, Devon, UK.
authors.forename: Tom
authors.initials: T
authors.lastname: Gale
authors.affiliation: Collaboration for the Advancement of Medical Education Research and Assessment, Plymouth University Peninsula Schools of Medicine & Dentistry, Plymouth, Devon, UK.
authors.forename: Tristan
authors.initials: T
authors.lastname: Price
authors.affiliation: Collaboration for the Advancement of Medical Education Research and Assessment, Plymouth University Peninsula Schools of Medicine & Dentistry, Plymouth, Devon, UK.
authors.forename: Sam
authors.initials: S
authors.lastname: Regan de Bere
mesh_terms: D002983:Clinical Competence / Q000592:standards; D003695:Delivery of Health Care / Q000592:standards; D019049:Developed Countries; D004503:Education, Medical, Graduate / Q000592:standards*; D004521:Educational Measurement; D019317:Evidence-Based Medicine; D006801:Humans; D007396:Internship and Residency / Q000592:standards*; D008007:Licensure, Medical / Q000592:standards / Q000639:trends
keywords: Assessment; Impact; National licensing examination; Systematic review; Validity