A multicentre observational study to evaluate a new tool to assess emergency physicians' non-technical skills.
OBJECTIVE
To evaluate a new tool to assess emergency physicians' non-technical skills.


METHODS
This was a multicentre observational study using data collected at four emergency departments in England. A proportion of observations used paired observers to obtain data for inter-rater reliability. Data were also collected for test-retest reliability, observability of skills, mean ratings and dispersion of ratings for each skill, as well as a comparison of skill level between hospitals. Qualitative data described the range of non-technical skills exhibited by trainees and identified sources of rater error.


RESULTS
96 assessments of 43 senior trainees were completed. At a scale level, intra-class coefficients were 0.575, 0.532 and 0.419 and using mean scores were 0.824, 0.702 and 0.519. Spearman's ρ for calculating test-retest reliability was 0.70 using mean scores. All skills were observed more than 60% of the time. The skill Maintenance of Standards received the lowest mean rating (4.8 on a nine-point scale) and the highest mean was calculated for Team Building (6.0). Two skills, Supervision & Feedback and Situational Awareness-Gathering Information, had significantly different distributions of ratings across the four hospitals (p<0.04 and 0.007, respectively), and this appeared to be related to the leadership roles of trainees.


CONCLUSIONS
This study shows the performance of the assessment tool is acceptable and provides valuable information to structure the assessment and training of non-technical skills, especially in relation to leadership. The framework of skills may be used to identify areas for development in individual trainees, as well as guide other patient safety interventions.
Centre for Patient Safety and Service Quality, Imperial College London, London, UK. l.flowerdew@imperial.ac.uk
Lynsey
L
Flowerdew
Arran
A
Gaunt
Jessica
J
Spedding
Ajay
A
Bhargava
Ruth
R
Brown
Charles
C
Vincent
Maria
M
Woloshynowych
D002983:Clinical Competence / Q000592:standards; D003657:Decision Making; D017598:Efficiency, Organizational; D004635:Emergency Medicine / Q000592:standards*; D004739:England; D005246:Feedback; D006801:Humans; D033183:Interdisciplinary Communication; D007857:Leadership; D010348:Patient Care Team / Q000458:organization & administration* / Q000592:standards; D010820:Physicians / Q000523:psychology* / Q000592:standards*; D011594:Psychometrics / Q000592:standards*; D036301:Qualitative Research; D011787:Quality of Health Care / Q000592:standards; D015203:Reproducibility of Results; D012919:Social Behavior; D011795:Surveys and Questionnaires