title: Script concordance testing: a review of published validity evidence.
abstract: BACKGROUND
  Script concordance test (SCT) scores are intended to reflect respondents' competence in interpreting clinical data under conditions of uncertainty. The validity of inferences based on SCT scores has not been rigorously established.
  
  
  OBJECTIVE
  This study was conducted in order to develop a structured validity argument for the interpretation of test scores derived through use of the script concordance method.
  
  
  METHODS
  We searched the PubMed, EMBASE and PsycINFO databases for articles pertaining to script concordance testing. We then reviewed these articles to evaluate the construct validity of the script concordance method, following an established approach for analysing validity data from five categories: content; response process; internal structure; relations to other variables, and consequences.
  
  
  RESULTS
  Content evidence derives from clear guidelines for the creation of authentic, ill-defined scenarios. High internal consistency reliability supports the internal structure of SCT scores. As might be expected, SCT scores correlate poorly with assessments of pure factual knowledge, in which correlations for more advanced learners are lower. The validity of SCT scores is weakly supported by evidence pertaining to examinee response processes and educational consequences.
  
  
  CONCLUSIONS
  Published research generally supports the use of SCT to assess the interpretation of clinical data under conditions of uncertainty, although specifics of the validity argument vary and require verification in different contexts and for particular SCTs. Our review identifies potential areas of further validity inquiry in all five categories of evidence. In particular, future SCT research might explore the impact of the script concordance method on teaching and learning, and examine how SCTs integrate with other assessment methods within comprehensive assessment programmes.
authors.affiliation: Department of Neurology and Neurosurgery, McGill University, Montreal, Quebec, Canada. stuart.lubarsky@mcgill.ca
authors.forename: Stuart
authors.initials: S
authors.lastname: Lubarsky
authors.forename: Bernard
authors.initials: B
authors.lastname: Charlin
authors.forename: David A
authors.initials: DA
authors.lastname: Cook
authors.forename: Colin
authors.initials: C
authors.lastname: Chalk
authors.forename: Cees P M
authors.initials: CP
authors.lastname: van der Vleuten
mesh_terms: D002983:Clinical Competence; D004501:Education, Medical / Q000379:methods* / Q000592:standards; D004521:Educational Measurement / Q000379:methods* / Q000592:standards; D006801:Humans; D011340:Problem Solving; D015203:Reproducibility of Results; D013337:Students, Medical / Q000523:psychology; D035501:Uncertainty