title: Reliability of results produced through objectively structured assessment of technical skills (OSATS) for endotracheal intubation (ETI).
abstract: OBJECTIVE
  To estimate the reliability of an objectively structured assessment of technical skills (OSATS) in patients for Endotracheal Intubation (ETI), using task-specific checklists during administration of general anaesthesia.
  
  
  METHODS
  Co-relational reliability study.
  
  
  METHODS
  Department of Anaesthesia and Medical Education, Pakistan Institute of Medical Sciences, Islamabad, from September to November 2010.
  
  
  METHODS
  Ten first year residents of the department of anaesthesia who had successfully completed a one-day course of ETI on mannequins and had performed 25 ETI in actual patients under supervision, were asked to perform ETI on anaesthetized patients in the operation theater after consent. The procedure was directly observed by two expert observers (ER1 and ER2) and one inexperienced observer (novice, NR) using a validated task oriented checklist. Each trainee repeated the procedure on another patient and was reassessed using the same checklist. Inter-rater for test and retest, and expert to novice reliabilities along with internal consistency were calculated.
  
  
  RESULTS
  The inter-rater reliability was 0.92 in test and was 0.98 in retest (Ebel's calculation). Spearman Rank Correlation Coefficient (SRCC) for ER 1 and NR was 0.76 in test and 0.89 in retest. Cronbach's alpha for internal consistency was 0.93 for first test and 0.97 for retest.
  
  
  CONCLUSIONS
  The OSATS for ETI in real patients demonstrated excellent inter-rater test reliability, inter-rater retest reliability, expert-novice reliability and internal consistency. This instrument, therefore, proves promising as a tool for competency-based evaluations.
authors.affiliation: Department of Medical Education, Pakistan Institute of Medical Sciences, Islamabad, Pakistan. ktanwir@hotmail.com
authors.forename: Tanwir
authors.initials: T
authors.lastname: Khaliq
mesh_terms: D000328:Adult; D000758:Anesthesia / Q000379:methods; D000776:Anesthesiology / Q000193:education*; D057189:Checklist; D002983:Clinical Competence; D004503:Education, Medical, Graduate / Q000379:methods; D004521:Educational Measurement / Q000379:methods; D005260:Female; D006801:Humans; D007396:Internship and Residency / Q000379:methods*; D007442:Intubation, Intratracheal / Q000379:methods* / Q000592:standards; D008297:Male; D015588:Observer Variation; D010154:Pakistan; D015203:Reproducibility of Results; D013647:Task Performance and Analysis