title: The intrajudge reliability of the perceptual rating of cleft palate speech before and after pharyngeal flap surgery: the effect of judges and speech samples.
abstract: OBJECTIVE
  In this pilot study, the reliabilities of the perceptual ratings of four types of speech samples by six judges, with and without expertise in evaluating cleft palate speech, were studied.
  
  
  METHODS
  Pre- and postoperative tape recordings of 15 patients with cleft lip and palate who had undergone a superiorly based pharyngeal flap operation were selected. Five speech-language pathologists and one oral and maxillofacial surgeon perceptually rated the following variables on separate 100-mm visual analog scales: hypernasality, audible nasal emission, intelligibility, misarticulations associated with velopharyngeal insufficiency, voice quality, and the presence or absence of hyponasality. These six variables were rated in four types of speech samples: reading of three sentences, repeating after the speech pathologist of three sentences, 10 sentences containing the aforementioned material, and the same 10 sentences in paired comparison. All speech samples were rerated after 3 months by the same judges.
  
  
  RESULTS
  Judges differed largely in the range they used in their rating. Intrajudge reliability of .56 to .78 was found for ratings of hypernasality. No significant differences in intrajudge reliability were found for the ratings with the different types of speech samples. The intrajudge reliability of a judge with expertise was not necessarily higher than of a judge without this expertise.
  
  
  CONCLUSIONS
  The improvement in speech is most reliably assessed with speech samples in paired comparison. A speech-language pathologist with expertise in evaluating cleft palate speech does not guarantee a high intrajudge reliability of the rating.
authors.affiliation: Department of Oral and Maxillofacial Surgery, Utrecht University Hospital, The Netherlands. K.Keuning@kmb.azu.nl
authors.forename: K H
authors.initials: KH
authors.lastname: Keuning
authors.forename: G H
authors.initials: GH
authors.lastname: Wieneke
authors.forename: P H
authors.initials: PH
authors.lastname: Dejonckere
mesh_terms: D000293:Adolescent; D016009:Chi-Square Distribution; D002648:Child; D002972:Cleft Palate / Q000503:physiopathology* / Q000601:surgery; D005260:Female; D006801:Humans; D008297:Male; D015588:Observer Variation; D010614:Pharynx / Q000601:surgery*; D011184:Postoperative Period; D015203:Reproducibility of Results; D013065:Speech Intelligibility; D013068:Speech Production Measurement / Q000379:methods / Q000706:statistics & numerical data; D013524:Surgical Flaps; D013637:Tape Recording; D014681:Velopharyngeal Insufficiency / Q000503:physiopathology / Q000601:surgery