title: Effects of congruent and incongruent visual cues on speech perception and brain activity in cochlear implant users.
abstract: While deafness-induced plasticity has been investigated in the visual and auditory domains, not much is known about language processing in audiovisual multimodal environments for patients with restored hearing via cochlear implant (CI) devices. Here, we examined the effect of agreeing or conflicting visual inputs on auditory processing in deaf patients equipped with degraded artificial hearing. Ten post-lingually deafened CI users with good performance, along with matched control subjects, underwent H 2 (15) O-positron emission tomography scans while carrying out a behavioral task requiring the extraction of speech information from unimodal auditory stimuli, bimodal audiovisual congruent stimuli, and incongruent stimuli. Regardless of congruency, the control subjects demonstrated activation of the auditory and visual sensory cortices, as well as the superior temporal sulcus, the classical multisensory integration area, indicating a bottom-up multisensory processing strategy. Compared to CI users, the control subjects exhibited activation of the right ventral premotor-supramarginal pathway. In contrast, CI users activated primarily the visual cortices more in the congruent audiovisual condition than in the null condition. In addition, compared to controls, CI users displayed an activation focus in the right amygdala for congruent audiovisual stimuli. The most notable difference between the two groups was an activation focus in the left inferior frontal gyrus in CI users confronted with incongruent audiovisual stimuli, suggesting top-down cognitive modulation for audiovisual conflict. Correlation analysis revealed that good speech performance was positively correlated with right amygdala activity for the congruent condition, but negatively correlated with bilateral visual cortices regardless of congruency. Taken together these results suggest that for multimodal inputs, cochlear implant users are more vision-reliant when processing congruent stimuli and are disturbed more by visual distractors when confronted with incongruent audiovisual stimuli. To cope with this multimodal conflict, CI users activate the left inferior frontal gyrus to adopt a top-down cognitive modulation pathway, whereas normal hearing individuals primarily adopt a bottom-up strategy.
authors.affiliation: Department of Otorhinolaryngology-Head and Neck Surgery, Seoul National University Bundang Hospital, Seongnam, Korea, jjsong96@gmail.com.
authors.forename: Jae-Jin
authors.initials: JJ
authors.lastname: Song
authors.forename: Hyo-Jeong
authors.initials: HJ
authors.lastname: Lee
authors.forename: Hyejin
authors.initials: H
authors.lastname: Kang
authors.forename: Dong Soo
authors.initials: DS
authors.lastname: Lee
authors.forename: Sun O
authors.initials: SO
authors.lastname: Chang
authors.forename: Seung Ha
authors.initials: SH
authors.lastname: Oh
mesh_terms: D000161:Acoustic Stimulation; D000328:Adult; D001302:Audiometry, Speech; D001303:Auditory Cortex / Q000000981:diagnostic imaging / Q000503:physiopathology*; D001931:Brain Mapping / Q000379:methods; D016022:Case-Control Studies; D019929:Cochlear Implantation / Q000295:instrumentation*; D003054:Cochlear Implants; D003071:Cognition; D003463:Cues; D005260:Female; D006801:Humans; D008297:Male; D008875:Middle Aged; D009473:Neuronal Plasticity; D046709:Pattern Recognition, Physiological; D010470:Perceptual Masking; D019986:Persons with Hearing Disabilities / Q000523:psychology / Q000534:rehabilitation*; D010775:Photic Stimulation; D049268:Positron-Emission Tomography; D011571:Psychoacoustics; D013067:Speech Perception; D014793:Visual Cortex / Q000000981:diagnostic imaging / Q000503:physiopathology*; D014796:Visual Perception; D055815:Young Adult