Developing auditory-perceptual judgment reliability in otolaryngology residents.
OBJECTIVE
To determine how standard residency training and educational background affect otolaryngology resident auditory-perceptual judgments compared with inexperienced listeners. A secondary aim was to assess the impact of a brief training session on the reliability of judgments among otolaryngology residents.


METHODS
Mixed cross-sectional and cohort study.


METHODS
Twenty adult dysphonic and four normal speaker samples were selected from a database. Fifteen listeners at various stages of residency in otolaryngology judged all samples for breathiness and roughness using visual analog scales. Fifteen inexperienced listeners judged the same samples. Subsequently, 12 otolaryngology residents underwent a brief training session. During training, listeners were provided definitions of rating dimensions and accuracy feedback while rating 20 novel (training) stimuli. The feedback included averaged responses from experienced clinicians who had previously evaluated the same speech samples. The residents then completed posttraining evaluations.


RESULTS
As a group, otolaryngology resident listeners had significantly better interrater agreement for judgments of roughness compared with inexperienced listeners (P<0.05) and trended toward better interrater agreement for judgments of breathiness (P=0.058). Posttraining, no significant improvement in interrater agreement for judgments of roughness was observed; however, a significant improvement was found in interrater agreement of judgments of breathiness (P<0.05).


CONCLUSIONS
Compared with inexperienced (untrained) listeners, otolaryngology resident listeners had better interrater agreement overall for judgments of dysphonia. A short training module was associated with improved reliability in resident evaluation of breathiness. Results have implications for improving and standardizing resident evaluation of dysphonia.
Vanderbilt Voice Center, Departments of Otolaryngology and Hearing and Speech Sciences, Vanderbilt University, Nashville, TN, USA.
S
S
Misono
A L
AL
Merati
T L
TL
Eadie
D000293:Adolescent; D000328:Adult; D000368:Aged; D000369:Aged, 80 and over; D001307:Auditory Perception; D002983:Clinical Competence / Q000592:standards; D003430:Cross-Sectional Studies; D055154:Dysphonia / Q000175:diagnosis* / Q000503:physiopathology / Q000523:psychology; D004503:Education, Medical, Graduate / Q000592:standards; D030141:Feedback, Psychological; D005260:Female; D006801:Humans; D007396:Internship and Residency / Q000592:standards; D007600:Judgment; D008297:Male; D008875:Middle Aged; D015588:Observer Variation; D010036:Otolaryngology / Q000193:education* / Q000592:standards; D015203:Reproducibility of Results; D013068:Speech Production Measurement / Q000592:standards; D014833:Voice Quality; D014861:Washington; D055815:Young Adult