title: Limitations of subjective cognitive load measures in simulation-based procedural training.
abstract: BACKGROUND
  The effective implementation of cognitive load theory (CLT) to optimise the instructional design of simulation-based training requires sensitive and reliable measures of cognitive load. This mixed-methods study assessed relationships between commonly used measures of total cognitive load and the extent to which these measures reflected participants' experiences of cognitive load in simulation-based procedural skills training.
  
  
  METHODS
  Two groups of medical residents (n = 38) completed three questionnaires after participating in simulation-based procedural skills training sessions: the Paas Cognitive Load Scale; the NASA Task Load Index (TLX), and a cognitive load component (CLC) questionnaire we developed to assess total cognitive load as the sum of intrinsic load (how complex the task is), extraneous load (how the task is presented) and germane load (how the learner processes the task for learning). We calculated Pearson's correlation coefficients to assess agreement among these instruments. Group interviews explored residents' perceptions about how the simulation sessions contributed to their total cognitive load. Interviews were audio-recorded, transcribed and subjected to qualitative content analysis.
  
  
  RESULTS
  Total cognitive load scores differed significantly according to the instrument used to assess them. In particular, there was poor agreement between the Paas Scale and the TLX. Quantitative and qualitative findings supported intrinsic cognitive load as synonymous with mental effort (Paas Scale), mental demand (TLX) and task difficulty and complexity (CLC questionnaire). Additional qualitative themes relating to extraneous and germane cognitive loads were not reflected in any of the questionnaires.
  
  
  CONCLUSIONS
  The Paas Scale, TLX and CLC questionnaire appear to be interchangeable as measures of intrinsic cognitive load, but not of total cognitive load. A more complete understanding of the sources of extraneous and germane cognitive loads in simulation-based training contexts is necessary to determine how best to measure and assess their effects on learning and performance outcomes.
authors.affiliation: Wilson Centre, University Health Network and University of Toronto, Toronto, Ontario, Canada.|HoPingKong Centre for Excellence in Education and Practice, University Health Network, Toronto, Ontario, Canada.
authors.forename: Laura M
authors.identifier: http://orcid.org/0000-0003-3294-7090
authors.initials: LM
authors.lastname: Naismith
authors.affiliation: Wilson Centre, University Health Network and University of Toronto, Toronto, Ontario, Canada.
authors.forename: Jeffrey J H
authors.initials: JJ
authors.lastname: Cheung
authors.affiliation: Wilson Centre, University Health Network and University of Toronto, Toronto, Ontario, Canada.|Department of Anesthesia, University of Toronto, Toronto, Ontario, Canada.
authors.forename: Charlotte
authors.initials: C
authors.lastname: Ringsted
authors.affiliation: HoPingKong Centre for Excellence in Education and Practice, University Health Network, Toronto, Ontario, Canada.|Department of Medicine, University of Toronto, Toronto, Ontario, Canada.
authors.forename: Rodrigo B
authors.initials: RB
authors.lastname: Cavalcanti
mesh_terms: D003071:Cognition; D004521:Educational Measurement; D006801:Humans; D007396:Internship and Residency; D007858:Learning; D009864:Ontario; D011582:Psychological Theory; D000066908:Simulation Training / Q000379:methods; D011795:Surveys and Questionnaires