title: Observational tools for assessment of procedural skills: a systematic review.
abstract: BACKGROUND
  Assessment by direct observation of procedural skills is an important source of constructive feedback. The aim of this study was to identify observational tools for technical skill assessment, to assess characteristics of these tools, and to assess their usefulness for assessment.
  
  
  METHODS
  Included studies reported tools for observational assessment of technical skills. A total of 106 articles were included.
  
  
  RESULTS
  Three main categories included global assessment scales evaluating generic skills (n = 29), task-specific methods assessing procedure-specific skills (n = 30), and combinations of tools evaluating both generic and task-specific skills (n = 47). In most studies, content validity was not evaluated using an accepted scientific method. All tools were assessed for inter-rater reliability and construct validity. Data on feasibility, acceptability, and educational impact were sparse.
  
  
  CONCLUSIONS
  There is evidence of validity and reliability for observational assessment tools at the trainee level. In most studies a comprehensive analysis of the tools was not achieved. Evaluation of technical skill using current observational assessment tools is not reliable and valid at the specialist level. Future research needs to focus on further systematic tool development and analysis, especially at the specialist level.
authors.affiliation: Department of Surgery and Cancer, Imperial College London, St. Mary's Hospital Campus, UK.
authors.forename: Kamran
authors.initials: K
authors.lastname: Ahmed
authors.forename: Danilo
authors.initials: D
authors.lastname: Miskovic
authors.forename: Ara
authors.initials: A
authors.lastname: Darzi
authors.forename: Thanos
authors.initials: T
authors.lastname: Athanasiou
authors.forename: George B
authors.initials: GB
authors.lastname: Hanna
mesh_terms: D002983:Clinical Competence; D004521:Educational Measurement; D005240:Feasibility Studies; D013502:General Surgery / Q000193:education*; D006801:Humans; D015203:Reproducibility of Results; D013514:Surgical Procedures, Operative / Q000193:education*; D013647:Task Performance and Analysis