title: Observer variation in drug-induced sleep endoscopy: experienced versus nonexperienced ear, nose, and throat surgeons.
abstract: OBJECTIVE
  To determine variations in interobserver and intraobserver agreement of drug-induced sleep endoscopy (DISE) in a cohort of experienced versus nonexperienced ear, nose, and throat (ENT) surgeons.
  
  
  METHODS
  Prospective, blinded agreement study.
  
  
  METHODS
  Ninety-seven ENT surgeons (90 nonexperienced with DISE; seven experienced) observed six different DISE videos and were asked to score the upper airway (UA) level (palate, oropharynx, tongue base, hypopharynx, epiglottis), direction (anteroposterior, concentric, lateral), and degree of collapse (none; partial or complete collapse). Findings were collected and analyzed, determining interobserver and intraobserver agreement [overall agreement (OA), specific agreement (SA)] and kappa values per UA level.
  
  
  RESULTS
  In the nonexperienced group, overall interobserver agreement on presence of tongue base collapse (OA = 0.63; kappa = 0.33) was followed by the agreement on epiglottis (OA = 0.57; kappa = 0.23) and oropharynx collapse (OA = 0.45; kappa = 0.09). Low overall interobserver agreement in this group was found for hypopharyngeal collapse (OA = 0.33; kappa = 0.08). A similar ranking was found for degree of collapse. For direction of collapse, high interobserver agreement was found for the palate (OA = 0.57; kappa = 0.16). Among the experienced observers, overall interobserver agreement was highest for presence of tongue base collapse (OA = 0.93; kappa = 0.71), followed by collapse of the palate (OA = 0.80; kappa = 0.51). In this group, lowest agreement was also found for hypopharyngeal collapse (OA = 0.47; kappa = 0.03). Interob-server agreement on direction of collapse was highest for epiglottis collapse (OA = 0.97; kappa = 0.97). Concerning the degree of collapse, highest agreement was found for degree of oropharyngeal collapse (OA = 0.82; kappa = 0.66). Among the experienced observers a statistically significant higher interobserver agreement was obtained for presence, direction, and degree of oropharyngeal collapse, as well as for presence of tongue base collapse and degree of epiglottis collapse. Among the nonexperienced observers, high intraobserver agreement was found in particular for tongue base and epiglottis collapse. Among the experienced observers, high agreement was found for all levels but to a lesser extent for hypopharyngeal collapse. Intraobserver agreement was statistically significantly higher in the experienced group, for all UA levels expect for the hypopharynx.
  
  
  CONCLUSIONS
  This study indicates that both interobserver and intraobserver agreement was higher in experienced versus nonexperienced ENT surgeons. Agreement ranged from poor to excellent in both groups. The current results suggest that experience in performing DISE is necessary to obtain reliable observations.
authors.affiliation: ENT, Head and Neck Surgery, Antwerp University Hospital, Edegem, Antwerp, Belgium. anneclaire.vroegop@uza.be
authors.forename: Anneclaire V M T
authors.initials: AV
authors.lastname: Vroegop
authors.forename: Olivier M
authors.initials: OM
authors.lastname: Vanderveken
authors.forename: Kristien
authors.initials: K
authors.lastname: Wouters
authors.forename: Evert
authors.initials: E
authors.lastname: Hamans
authors.forename: Marijke
authors.initials: M
authors.lastname: Dieltjens
authors.forename: Nele R
authors.initials: NR
authors.lastname: Michels
authors.forename: Winfried
authors.initials: W
authors.lastname: Hohenhorst
authors.forename: Eric J
authors.initials: EJ
authors.lastname: Kezirian
authors.forename: Bhik T
authors.initials: BT
authors.lastname: Kotecha
authors.forename: Nico
authors.initials: N
authors.lastname: de Vries
authors.forename: Marc J
authors.initials: MJ
authors.lastname: Braem
authors.forename: Paul H
authors.initials: PH
authors.lastname: Van de Heyning
mesh_terms: D002983:Clinical Competence / Q000706:statistics & numerical data*; D004724:Endoscopy / Q000379:methods*; D004825:Epiglottis / Q000503:physiopathology; D006801:Humans; D008297:Male; D008875:Middle Aged; D015588:Observer Variation; D009960:Oropharynx / Q000503:physiopathology; D010036:Otolaryngology / Q000379:methods / Q000592:standards / Q000706:statistics & numerical data*; D010159:Palate / Q000503:physiopathology; D011446:Prospective Studies; D020181:Sleep Apnea, Obstructive / Q000175:diagnosis / Q000503:physiopathology; D014059:Tongue / Q000503:physiopathology
keywords: Drug-induced sleep endoscopy; interobserver agreement; intraobserver agreement; obstructive sleep apnea; sleep disordered breathing; sleep related breathing disorders