Neural Mechanisms of Multisensory Attention: Impacts on Speech Perception
Location
Science Center: Bent Corridor
Document Type
Poster - Open Access
Start Date
4-28-2023 12:00 PM
End Date
4-28-2023 2:00 PM
Abstract
Visual information plays a crucial role in audiovisual speech perception as demonstrated by the McGurk illusion. Previous research has shown that audiovisual speech integration is lessened under high attentional demands. Additionally, secondary visual versus auditory tasks have disparate effects on the likelihood of perceiving the McGurk illusion and may impact speech integration through separate mechanisms. However, differences in gaze patterns or the type of attention engaged may explain these differential effects. The current study seeks to determine whether the differential impact of auditory and visual distractors can be replicated for featural attention, specifically, and whether differences in gaze patterns are evident across secondary tasks. Participants completed a psychophysical task modeled after the McGurk effect in one of three conditions: 1) distractor-free, 2) secondary auditory task, 3) and secondary visual task. For the secondary task, participants were presented with a series of gray boxes (visual) or white noise bursts (auditory) and were asked to determine whether one of the gray boxes/white noise bursts was a higher or lower intensity. Thresholds for intensity judgments were determined for each participant using a psychophysical staircase procedure. Participants’ gaze was measured during distractor-free and high-load blocks. Preliminary results suggest that differences in gaze patterns are minimal between auditory and visual secondary tasks suggesting that modality effects on audiovisual speech integration are independent of looking behavior. This line of research has implications for understanding the differences in multisensory processing in developmental disorders which are commonly characterized by disruptions in attentional regulation.
Keywords:
Multisensory integration, Attention, McGurk Effect, Eye gaze, Speech perception
Recommended Citation
Goldberg, Jessie; Kwakye, Leslie; Orozco, Andrea; Barana, Ankit; Liberatore, Dov; Ade, Emi; Stenberg, Luke; Kirkland, James; Gutierrez, Z; Rogers, Destiny-Ranae; and Rosa-Palermo, Nicolas, "Neural Mechanisms of Multisensory Attention: Impacts on Speech Perception" (2023). Research Symposium. 13.
https://digitalcommons.oberlin.edu/researchsymp/2023/posters/13
Major
Neuroscience; Anthropology
Project Mentor(s)
Leslie Kwakye, Neuroscience
2023
Neural Mechanisms of Multisensory Attention: Impacts on Speech Perception
Science Center: Bent Corridor
Visual information plays a crucial role in audiovisual speech perception as demonstrated by the McGurk illusion. Previous research has shown that audiovisual speech integration is lessened under high attentional demands. Additionally, secondary visual versus auditory tasks have disparate effects on the likelihood of perceiving the McGurk illusion and may impact speech integration through separate mechanisms. However, differences in gaze patterns or the type of attention engaged may explain these differential effects. The current study seeks to determine whether the differential impact of auditory and visual distractors can be replicated for featural attention, specifically, and whether differences in gaze patterns are evident across secondary tasks. Participants completed a psychophysical task modeled after the McGurk effect in one of three conditions: 1) distractor-free, 2) secondary auditory task, 3) and secondary visual task. For the secondary task, participants were presented with a series of gray boxes (visual) or white noise bursts (auditory) and were asked to determine whether one of the gray boxes/white noise bursts was a higher or lower intensity. Thresholds for intensity judgments were determined for each participant using a psychophysical staircase procedure. Participants’ gaze was measured during distractor-free and high-load blocks. Preliminary results suggest that differences in gaze patterns are minimal between auditory and visual secondary tasks suggesting that modality effects on audiovisual speech integration are independent of looking behavior. This line of research has implications for understanding the differences in multisensory processing in developmental disorders which are commonly characterized by disruptions in attentional regulation.
