title: Comparing two methods for reducing variability in voice quality measurements.
abstract: OBJECTIVE
  Interrater disagreements in ratings of quality plague the study of voice. This study compared 2 methods for handling this variability.
  
  
  METHODS
  Listeners provided multiple breathiness ratings for 2 sets of pathological voices, one including 20 male and 20 female voices unselected for quality and one including 20 breathy female voices. Ratings for each listener were averaged together, mean ratings were z transformed, and the likelihood that 2 listeners would agree exactly in their ratings was calculated as a function of averaging and standardizing condition. Data were also multidimensionally scaled to examine similarities among listeners in perceptual strategy. Results were compared with parallel analyses of existing breathiness ratings of the same voices gathered using a method-of-adjustment task.
  
  
  RESULTS
  Three-way interactions between the mean rating for a voice, standardization condition, and the number of voices averaged together were observed, but no main effect of averaging condition emerged. Multidimensional scaling revealed significant residual differences in perceptual strategy across listeners after averaging and standardizing. Ratings from the method-of-adjustment task showed both high agreement levels and consistent perceptual strategies across listeners, as theoretically predicted.
  
  
  CONCLUSIONS
  Averaging multiple ratings and standardizing the mean are inadequate in addressing variations in voice quality perception.
authors.affiliation: University of California, Los Angeles, USA. jkreiman@ucla.edu
authors.forename: Jody
authors.initials: J
authors.lastname: Kreiman
authors.forename: Bruce R
authors.initials: BR
authors.lastname: Gerratt
mesh_terms: D000293:Adolescent; D000328:Adult; D055154:Dysphonia / Q000175:diagnosis*; D005260:Female; D006801:Humans; D008297:Male; D015588:Observer Variation; D011594:Psychometrics / Q000379:methods / Q000592:standards / Q000706:statistics & numerical data; D015656:Respiratory Mechanics; D013061:Speech Acoustics; D013063:Speech Discrimination Tests / Q000379:methods / Q000592:standards / Q000706:statistics & numerical data; D013067:Speech Perception; D014833:Voice Quality; D055815:Young Adult