title: Decision-making bias in assessment: the effect of aggregating objective information and anecdote.
abstract: BACKGROUND
  Assessment decisions increasingly rely on synthesis of information from a variety of sources. It is known that aggregation of information to make decisions is open to a number of biases. The aim of this research was to investigate bias, accuracy and confidence of assessment decision making.
  
  
  METHODS
  The participants were consultation skills assessors. A model for incremental information was developed with participants being shown results from purposefully selected, but authentic, data from the University's final summative 10-station Objective Structured Clinical Examination (OSCE). After each piece of information, participants gave a pass-fail decision and their confidence in that choice. Following the information from 10 OSCE stations the participants were given a discordant fictional anecdote and again participants gave a pass-fail decision and their confidence.
  
  
  RESULTS
  When there is overwhelming evidence to support a pass or fail, participants were not as confident as the data would support. Participants were less confident to make a fail decision than a pass. Despite considerable evidence from multiple results some participants altered decisions based on isolated contradictory information from an anecdote.
  
  
  CONCLUSIONS
  These findings are significant in understanding decision-making. Given equivalent levels of evidence, decision makers are less confident to fail than pass and less robust information can undermine more robust information.
authors.affiliation: University of Otago , New Zealand.
authors.forename: Mike J
authors.initials: MJ
authors.lastname: Tweed
authors.forename: Mark
authors.initials: M
authors.lastname: Thompson-Fawcett
authors.forename: Tim J
authors.initials: TJ
authors.lastname: Wilkinson
mesh_terms: D002983:Clinical Competence; D003142:Communication; D003657:Decision Making; D006801:Humans; D012649:Self Concept; D013337:Students, Medical / Q000523:psychology*