Abstract
According to the predictive coding theory, top-down predictions are conveyed by backward connections and prediction errors are propagated forward across the cortical hierarchy. Using MEG in humans, we show that violating multisensory predictions causes a fundamental and qualitative change in both the frequency and spatial distribution of cortical activity. When visual speech input correctly predicted auditory speech signals, a slow delta regime (3–4 Hz) developed in higher-order speech areas. In contrast, when auditory signals invalidated predictions inferred from vision, a low-beta (14–15 Hz) / high-gamma (60–80 Hz) coupling regime appeared locally in a multisensory area (area STS). This frequency shift in oscillatory responses scaled with the degree of audio-visual congruence and was accompanied by increased gamma activity in lower sensory regions. These findings are consistent with the notion that bottom-up prediction errors are communicated in predominantly high (gamma) frequency ranges, whereas top-down predictions are mediated by slower (beta) frequencies.
This is a preview of subscription content, access via your institution
Access options
Subscribe to this journal
Receive 12 print issues and online access
$209.00 per year
only $17.42 per issue
Buy this article
- Purchase on Springer Link
- Instant access to full article PDF
Prices may be subject to local taxes which are calculated during checkout
Similar content being viewed by others
Change history
15 May 2011
In the HTML version of this article initially published online, the International Phonetic Alphabet character “ʒ” was missing in the first sentence of the Results. The error has been corrected in the HTML.
References
Friston, K. A theory of cortical responses. Phil. Trans. R. Soc. Lond. B 360, 815–836 (2005).
Mumford, D. On the computational architecture of the neocortex. II. The role of cortico-cortical loops. Biol. Cybern. 66, 241–251 (1992).
Rao, R.P. & Ballard, D.H. Predictive coding in the visual cortex: a functional interpretation of some extra-classical receptive-field effects. Nat. Neurosci. 2, 79–87 (1999).
Buzsáki, G. & Draguhn, A. Neuronal oscillations in cortical networks. Science 304, 1926–1929 (2004).
Canolty, R.T. et al. High gamma power is phase-locked to theta oscillations in human neocortex. Science 313, 1626–1628 (2006).
Lakatos, P. et al. An oscillatory hierarchy controlling neuronal excitability and stimulus processing in the auditory cortex. J. Neurophysiol. 94, 1904–1911 (2005).
Wang, X.J. Neurophysiological and computational principles of cortical rhythms in cognition. Physiol. Rev. 90, 1195–1268 (2010).
van Wassenhove, V., Grant, K.W. & Poeppel, D. Visual speech speeds up the neural processing of auditory speech. Proc. Natl. Acad. Sci. USA 102, 1181–1186 (2005).
Chandrasekaran, C., Trubanova, A., Stillittano, S., Caplier, A. & Ghazanfar, A.A. The natural statistics of audiovisual speech. PLOS Comput. Biol. 5, e1000436 (2009).
Arnal, L.H., Morillon, B., Kell, C.A. & Giraud, A.-L. Dual neural routing of visual facilitation in speech processing. J. Neurosci. 29, 13445–13453 (2009).
McGurk, H. & MacDonald, J. Hearing lips and seeing voices. Nature 264, 746–748 (1976).
Shah, A.S. et al. Neural dynamics and the fundamental mechanisms of event-related brain potentials. Cereb. Cortex 14, 476–483 (2004).
Garrido, M.I., Kilner, J.M., Kiebel, S.J., Stephan, K.E. & Friston, K.J. Dynamic causal modelling of evoked potentials: a reproducibility study. Neuroimage 36, 571–580 (2007).
Hickok, G. & Poeppel, D. The cortical organization of speech processing. Nat. Rev. Neurosci. 8, 393–402 (2007).
Beauchamp, M.S., Lee, K.E., Argall, B.D. & Martin, A. Integration of auditory and visual information about objects in superior temporal sulcus. Neuron 41, 809–823 (2004).
Ghazanfar, A.A., Chandrasekaran, C. & Logothetis, N.K. Interactions between the superior temporal sulcus and auditory cortex mediate dynamic face/voice integration in rhesus monkeys. J. Neurosci. 28, 4457–4469 (2008).
Kayser, C. & Logothetis, N.K. Directed interactions between auditory and superior temporal cortices and their role in sensory integration. Front. Integr. Neurosci. 3, 7 (2009).
Koch, S.P., Werner, P., Steinbrink, J., Fries, P. & Obrig, H. Stimulus-induced and state-dependent sustained gamma activity is tightly coupled to the hemodynamic response in humans. J. Neurosci. 29, 13962–13970 (2009).
Mukamel, R. et al. Coupling between neuronal firing, field potentials, and FMRI in human auditory cortex. Science 309, 951–954 (2005).
Niessing, J. et al. Hemodynamic signals correlate tightly with synchronized gamma oscillations. Science 309, 948–951 (2005).
Engel, A.K. & Fries, P. Beta-band oscillations—signalling the status quo? Curr. Opin. Neurobiol. 20, 156–165 (2010).
von Stein, A., Chiang, C. & Konig, P. Top-down processing mediated by interareal synchronization. Proc. Natl. Acad. Sci. USA 97, 14748–14753 (2000).
Womelsdorf, T. et al. Modulation of neuronal interactions through neuronal synchronization. Science 316, 1609–1612 (2007).
Sejnowski, T.J. & Paulsen, O. Network oscillations: emerging computational principles. J. Neurosci. 26, 1673–1676 (2006).
Wróbel, A. Beta activity: a carrier for visual attention. Acta Neurobiol. Exp. (Warsz.) 60, 247–260 (2000).
Neisser, U. Cognitive Psychology (Appleton-Century-Crofts, New York, 1967).
Poeppel, D., Idsardi, W.J. & van Wassenhove, V. Speech perception at the interface of neurobiology and linguistics. Phil. Trans. R. Soc. Lond. B 363, 1071–1086 (2008).
Stevens, K.N. & Halle, M. Remarks on analysis by synthesis and distinctive features. in Models for the Perception of Speech and Visual Form: Proceedings of a Symposium (ed. Wathen-Dunn, W.) 88–102 (MIT Press, Cambridge, Massachusetts, USA, 1967).
Wolpert, D.M., Ghahramani, Z. & Jordan, M.I. An internal model for sensorimotor integration. Science 269, 1880–1882 (1995).
Roopun, A.K. et al. Period concatenation underlies interactions between gamma and beta rhythms in neocortex. Front. Cell. Neurosci. 2, 1 (2008).
Spratling, M.W. Reconciling predictive coding and biased competition models of cortical function. Front. Comput. Neurosci. 2, 4 (2008).
Buschman, T.J. & Miller, E.K. Top-down versus bottom-up control of attention in the prefrontal and posterior parietal cortices. Science 315, 1860–1862 (2007).
George, D. & Hawkins, J. Towards a mathematical theory of cortical micro-circuits. PLOS Comput. Biol. 5, e1000532 (2009).
Kopell, N., Kramer, M.A., Malerba, P. & Whittington, M.A. Are different rhythms good for different functions? Front. Hum. Neurosci. 4, 187 (2010).
Kramer, M.A. et al. Rhythm generation through period concatenation in rat somatosensory cortex. PLOS Comput. Biol. 4, e1000169 (2008).
Hagoort, P., Hald, L., Bastiaansen, M. & Petersson, K.M. Integration of word meaning and world knowledge in language comprehension. Science 304, 438–441 (2004).
Hald, L.A., Bastiaansen, M.C. & Hagoort, P. EEG theta and gamma responses to semantic violations in online sentence processing. Brain Lang. 96, 90–105 (2006).
Kayser, C., Montemurro, M.A., Logothetis, N.K. & Panzeri, S. Spike-phase coding boosts and stabilizes information carried by spatial and temporal spike patterns. Neuron 61, 597–608 (2009).
Schroeder, C.E. & Lakatos, P. The gamma oscillation: master or slave? Brain Topogr. 22, 24–26 (2009).
Engel, A.K., Fries, P. & Singer, W. Dynamic predictions: oscillations and synchrony in top-down processing. Nat. Rev. Neurosci. 2, 704–716 (2001).
Fries, P., Roelfsema, P.R., Engel, A.K., Konig, P. & Singer, W. Synchronization of oscillatory responses in visual cortex correlates with perception in interocular rivalry. Proc. Natl. Acad. Sci. USA 94, 12699–12704 (1997).
Wyart, V. & Tallon-Baudry, C. Neural dissociation between visual awareness and spatial attention. J. Neurosci. 28, 2667–2679 (2008).
Schroeder, C.E. & Lakatos, P. Low-frequency neuronal oscillations as instruments of sensory selection. Trends Neurosci. 32, 9–18 (2009).
Rockland, K.S. & Pandya, D.N. Laminar origins and terminations of cortical connections of the occipital lobe in the rhesus monkey. Brain Res. 179, 3–20 (1979).
Akam, T. & Kullmann, D.M. Oscillations and filtering networks support flexible routing of information. Neuron 67, 308–320 (2010).
Colgin, L.L. et al. Frequency of gamma oscillations routes flow of information in the hippocampus. Nature 462, 353–357 (2009).
Panzeri, S., Brunel, N., Logothetis, N.K. & Kayser, C. Sensory neural codes using multiplexed temporal scales. Trends Neurosci. 33, 111–120 (2010).
Grill-Spector, K., Henson, R. & Martin, A. Repetition and the brain: neural models of stimulus-specific effects. Trends Cogn. Sci. 10, 14–23 (2006).
Tallon-Baudry, C., Bertrand, O., Delpuech, C. & Pernier, J. Stimulus specificity of phase-locked and non-phase-locked 40 Hz visual responses in human. J. Neurosci. 16, 4240–4249 (1996).
Maris, E. & Oostenveld, R. Nonparametric statistical testing of EEG- and MEG-data. J. Neurosci. Methods 164, 177–190 (2007).
Acknowledgements
We thank the staff of the Centre de Neuroimagerie de Recherche and the Magnetoencephalography Center (Hôpital de la Pitié-Salpêtrière), in particular J.-D. Lemarechal, A. Ducorps and D. Schwarz, and the colleagues who commented on this work: V. van Wassenhove, B. Morillon, A. Hyafil, S. Denève, L. Melloni, F. Griffon, A. Kleinschmidt, E. Koechlin, B. Fischer, K. Friston and C. Tallon-Baudry. This work was supported by the Centre National de la Recherche Scientifique (A.-L.G.) and the Fondation Fyssen (V.W.).
Author information
Authors and Affiliations
Contributions
Conceived and designed the experiments: L.H.A., A.-L.G.; performed the experiments: L.H.A.; analyzed the data: L.H.A., V.W., A.-L.G.; contributed to analysis tools: L.H.A., V.W.; wrote the paper: L.H.A., V.W., A.-L.G.
Corresponding author
Ethics declarations
Competing interests
The authors declare no competing financial interests.
Supplementary information
Supplementary Text and Figures
Supplementary Data and Supplementary Figures 1–5 (PDF 1038 kb)
Rights and permissions
About this article
Cite this article
Arnal, L., Wyart, V. & Giraud, AL. Transitions in neural oscillations reflect prediction errors generated in audiovisual speech. Nat Neurosci 14, 797–801 (2011). https://doi.org/10.1038/nn.2810
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1038/nn.2810
This article is cited by
-
Aberrant neurophysiological signaling associated with speech impairments in Parkinson’s disease
npj Parkinson's Disease (2023)
-
Immediate neural impact and incomplete compensation after semantic hub disconnection
Nature Communications (2023)
-
Preparatory delta phase response is correlated with naturalistic speech comprehension performance
Cognitive Neurodynamics (2022)
-
Coupled oscillations enable rapid temporal recalibration to audiovisual asynchrony
Communications Biology (2021)
-
Visual Influences on Auditory Behavioral, Neural, and Perceptual Processes: A Review
Journal of the Association for Research in Otolaryngology (2021)