title: Rater methodology for stroboscopy: a systematic review.
abstract: OBJECTIVE
  Laryngeal endoscopy with stroboscopy (LES) remains the clinical gold standard for assessing vocal fold function. LES is used to evaluate the efficacy of voice treatments in research studies and clinical practice. LES as a voice treatment outcome tool is only as good as the clinician interpreting the recordings. Research using LES as a treatment outcome measure should be evaluated based on rater methodology and reliability. The purpose of this literature review was to evaluate the rater-related methodology from studies that use stroboscopic findings as voice treatment outcome measures.
  
  
  METHODS
  Systematic literature review.
  
  
  METHODS
  Computerized journal databases were searched for relevant articles using terms: stroboscopy and treatment. Eligible articles were categorized and evaluated for the use of rater-related methodology, reporting of number of raters, types of raters, blinding, and rater reliability.
  
  
  RESULTS
  Of the 738 articles reviewed, 80 articles met inclusion criteria. More than one-third of the studies included in the review did not report the number of raters who participated in the study. Eleven studies reported results of rater reliability analysis with only two studies reporting good inter- and intrarater reliability.
  
  
  CONCLUSIONS
  The comparability and use of results from treatment studies that use LES are limited by a lack of rigor in rater methodology and variable, mostly poor, inter- and intrarater reliability. To improve our ability to evaluate and use the findings from voice treatment studies that use LES features as outcome measures, greater consistency of reporting rater methodology characteristics across studies and improved rater reliability is needed.
authors.affiliation: Department of Health Science and Research, Medical University of South Carolina, Charleston, South Carolina; Department of Otolaryngology-Head and Neck Surgery, Medical University of South Carolina, Charleston, South Carolina. Electronic address: bonilhah@musc.edu.
authors.forename: Heather Shaw
authors.initials: HS
authors.lastname: Bonilha
authors.affiliation: Department of Health Science and Research, Medical University of South Carolina, Charleston, South Carolina.
authors.forename: Kendrea L
authors.initials: KL
authors.lastname: Focht
authors.affiliation: Department of Health Science and Research, Medical University of South Carolina, Charleston, South Carolina; Department of Otolaryngology-Head and Neck Surgery, Medical University of South Carolina, Charleston, South Carolina.
authors.forename: Bonnie
authors.initials: B
authors.lastname: Martin-Harris
mesh_terms: D006801:Humans; D015203:Reproducibility of Results; D049489:Stroboscopy / Q000379:methods*; D014832:Voice Disorders / Q000175:diagnosis*
keywords: Rater; Reliability; Stroboscopy; Voice