Cross-Modal Attentional Cueing in An Immersive 3D Virtual Environment

Science Center, Bent Corridor

When sights and sounds in our environment have common origin, they are processed in concert in the brain as part of a process called multisensory integration (MSI). Research in the field of MSI has shown that a stimulus in one modality can direct the spatial attention of another modality. For example, an auditory cue presented in the left hemisphere will decrease the time to detect a visual target presented on the left, whereas detection of a visual target on the right will be relatively slowed. Previous studies investigating attentional cueing have used white noise and circular flashes on a computer screen. Research in the field of virtual reality has shown that increased realism can enhance behavioral responses, suggesting that experiments conducted on computer screens may not validly reflect how attentional cueing operates in a real-world context. In this study, we developed an orthogonal cross-modal attentional cueing paradigm with exogenous visual-cueing-audio and audio-cueing-visual conditions. Participants identified locations of targets (animate jellyfish; bubble sounds) after being presented with cues (realistic flashes; noise burst) in an immersive 3D underwater virtual environment. The time between cues and targets (stimulus onset asynchronies) varied in duration. Participants’ response times to valid, invalid, and neutral cues were measured. Preliminary results indicate that invalid cues produce the largest response times as compared to valid and neutral cues, in both the auditorycueing- visual and visual-cueing-audio conditions. These results are in line with previous findings suggesting that traditional psychophysics methods may accurately reflect cross-modal attentional cueing in realistic perception.