title: Psychometric evaluation of a direct observation of procedural skills assessment tool for ultrasound-guided regional anaesthesia.
abstract: Assessment tools must be investigated for reliability, validity and feasibility before being implemented. In 2013, the Australian and New Zealand College of Anaesthetists introduced workplace-based assessments, including a direct observation of a procedural skills assessment tool. The objective of this study was to evaluate the psychometric properties of this assessment tool for ultrasound-guided regional anaesthesia. Six experts assessed 30 video-recorded trainee performances of ultrasound-guided regional anaesthesia. Inter-rater reliability, assessed using absolute agreement intraclass correlation coefficients, varied from 0.10 to 0.49 for the nine individual nine-point scale items, and was 0.25 for a 'total score' of all items. Internal consistency was measured by correlation between 'total score' and 'overall performance' scale item (r = 0.68, p < 0.001). Construct validity was demonstrated by the 'total score' correlating with trainee experience (r = 0.51, p = 0.004). The mean time taken to complete assessments was 6 min 35 s.
authors.affiliation: University of Newcastle, Newcastle, New South Wales, Australia.
authors.forename: M J
authors.initials: MJ
authors.lastname: Watson
authors.forename: D M
authors.initials: DM
authors.lastname: Wong
authors.forename: R
authors.initials: R
authors.lastname: Kluger
authors.forename: A
authors.initials: A
authors.lastname: Chuan
authors.forename: M D
authors.initials: MD
authors.lastname: Herrick
authors.forename: I
authors.initials: I
authors.lastname: Ng
authors.forename: D J
authors.initials: DJ
authors.lastname: Castanelli
authors.forename: L
authors.initials: L
authors.lastname: Lin
authors.forename: A K
authors.initials: AK
authors.lastname: Lansdown
authors.forename: M J
authors.initials: MJ
authors.lastname: Barrington
mesh_terms: D000328:Adult; D000368:Aged; D000369:Aged, 80 and over; D000765:Anesthesia, Conduction; D002983:Clinical Competence; D005260:Female; D006801:Humans; D008297:Male; D008875:Middle Aged; D011594:Psychometrics; D018084:Ultrasonography, Interventional