A joint project of the Graduate School, Peabody College, and the Jean & Alexander Heard Library

Title page for ETD etd-08162018-094951


Type of Document Dissertation
Author Nidiffer, Aaron Ross
URN etd-08162018-094951
Title Temporal Correlation and Its Role in Multisensory Feature Integration and Binding
Degree PhD
Department Hearing and Speech Sciences
Advisory Committee
Advisor Name Title
Ramnarayan Ramachandran Committee Chair
Adele Deiderich Committee Member
G. Christopher Stecker Committee Member
Mark T. Wallace Committee Member
Keywords
  • binding
  • behavior
  • audiovisual
  • integration
  • proximity
  • similarity
Date of Defense 2018-07-17
Availability restrictone
Abstract
Our successful interaction with the environment requires the brain to appropriately combine and segregate sensory information coming from various events. These events frequently produce energy and signals that are processed by different sensory systems. These signals, such as the voice and the mouth movements of a speaker, are often temporally correlated (i.e., they change together over time). Human observers can use these correlation cues to facilitate feature binding—which results in the formation of a multisensory perceptual object. This binding of signals across sensory modalities often confers a constellation of behavioral and perceptual benefits. For example, the intelligibility of speech in noise is improved when we can see the face of the person speaking. Because complex sensory environments can generate weak—but significant—correlations between sensory signals from separate events, evaluating the strength of correlation would enable the selection of the appropriate signals to be bound. We investigated how observers utilized different levels of temporal correlation during multisensory behaviors. We found that detection of amplitude modulation (AM) in auditory and visual stimuli improved in a linear manner with the temporal correlation of those AM features. The same benefit conferred by AM correlation was also observed in the detection of an orthogonal frequency modulation feature. We also discovered a novel phenomenon where participant behaviors were uniquely phase-shifted relative to stimulus correlation. We hypothesized that this transformation reflected individual differences in the temporal processing abilities of participants’ auditory and visual systems. Phase shifts were accounted for by simulating a lag in either the auditory or visual AM and separate experiments show that phase shifts across participants were related to their differences in auditory and visual reaction time. These experiments indicate that the brain’s computation of correlation is more sophisticated than previously shown and suggest a potential mechanism for the binding of cues in complex sensory landscapes
Files
  Filename       Size       Approximate Download Time (Hours:Minutes:Seconds) 
 
 28.8 Modem   56K Modem   ISDN (64 Kb)   ISDN (128 Kb)   Higher-speed Access 
[campus] NidifferA.pdf 34.91 Mb 02:41:38 01:23:07 01:12:44 00:36:22 00:03:06
[campus] indicates that a file or directory is accessible from the campus network only.

Browse All Available ETDs by ( Author | Department )

If you have more questions or technical problems, please Contact LITS.