title: A systematic review of behavioural marker systems in healthcare: what do we know about their attributes, validity and application?
abstract: OBJECTIVE
  Behavioural marker systems are advocated as a method for providing accurate assessments, directing feedback and determining the impact of teamwork improvement initiatives. The present article reports on the state of quality surrounding their use in healthcare and discusses the implications of these findings for future research, development and application. In doing so, this article provides a practical resource where marker systems can be selected and evaluated based on their strengths and limitations.
  
  
  METHODS
  Four research questions framed this review: what are the attributes of behavioural marker systems? What evidence of reliability and validity exists? What skills and expertise are required for their use? How have they been applied to investigate the relationship between teamwork and other constructs?
  
  
  RESULTS
  Behavioural markers systems are generally designed for specific work domains or tasks. They often cover similar content with inconsistent terminology, which complicates the comparison of research findings across clinical domains. Although several approaches were used to establish the reliability and validity of marker systems, the marker system literature, as a whole, requires more robust reliability and validity evidence. The impact of rater training on rater proficiency was mixed, but evidence suggests that improvements can be made over time.
  
  
  CONCLUSIONS
  A consensus of definitions for teamwork constructs must be reached to ensure that the meaning behind behavioural measurement is understood across disciplines, work domains and task types. Future development efforts should focus on the cost effectiveness and feasibility of measurement tools including time spent training raters. Further, standards for the testing and reporting of psychometric evidence must be established. Last, a library of tools should be generated around whether the instrument measures general or domain-specific behaviours.
authors.affiliation: The Armstrong Institute for Patient Safety and Quality, The Johns Hopkins University School of Medicine, Baltimore, Maryland, USA.
authors.forename: Aaron S
authors.initials: AS
authors.lastname: Dietz
authors.affiliation: The Armstrong Institute for Patient Safety and Quality, The Johns Hopkins University School of Medicine, Baltimore, Maryland, USA Department of Anesthesiology and Critical Care Medicine, The Johns Hopkins University School of Medicine, Baltimore, Maryland, USA.
authors.forename: Peter J
authors.initials: PJ
authors.lastname: Pronovost
authors.affiliation: The Armstrong Institute for Patient Safety and Quality, The Johns Hopkins University School of Medicine, Baltimore, Maryland, USA.
authors.forename: Kari N
authors.initials: KN
authors.lastname: Benson
authors.affiliation: Department of Anesthesiology and Critical Care Medicine, The Johns Hopkins University School of Medicine, Baltimore, Maryland, USA.
authors.forename: Pedro Alejandro
authors.initials: PA
authors.lastname: Mendez-Tellez
authors.affiliation: Surginal Intensive Care Unit, Johns Hopkins Hospital, Baltimore, Maryland, USA.
authors.forename: Cynthia
authors.initials: C
authors.lastname: Dwyer
authors.affiliation: The Armstrong Institute for Patient Safety and Quality, The Johns Hopkins University School of Medicine, Baltimore, Maryland, USA.
authors.forename: Rhonda
authors.initials: R
authors.lastname: Wyskiel
authors.affiliation: The Armstrong Institute for Patient Safety and Quality, The Johns Hopkins University School of Medicine, Baltimore, Maryland, USA Department of Anesthesiology and Critical Care Medicine, The Johns Hopkins University School of Medicine, Baltimore, Maryland, USA.
authors.forename: Michael A
authors.initials: MA
authors.lastname: Rosen
mesh_terms: D005246:Feedback; D006302:Health Services Research / Q000379:methods*; D006801:Humans; D010043:Outcome and Process Assessment, Health Care; D010348:Patient Care Team / Q000592:standards*; D011594:Psychometrics; D058996:Quality Improvement; D019984:Quality Indicators, Health Care
keywords: Performance measures; Qualitative research; Teamwork