Abstract
Recognizing predictive relationships is critical for survival, but an understanding of the underlying neural mechanisms remains elusive. In particular, it is unclear how the brain distinguishes predictive relationships from spurious ones when evidence about a relationship is ambiguous, or how it computes predictions given such uncertainty. To better understand this process, we introduced ambiguity into an associative learning task by presenting aversive outcomes both in the presence and in the absence of a predictive cue. Electrophysiological and optogenetic approaches revealed that amygdala neurons directly regulated and tracked the effects of ambiguity on learning. Contrary to established accounts of associative learning, however, interference from competing associations was not required to assess an ambiguous cue–outcome contingency. Instead, animals' behavior was explained by a normative account that evaluates different models of the environment's statistical structure. These findings suggest an alternative view of amygdala circuits in resolving ambiguity during aversive learning.
This is a preview of subscription content, access via your institution
Access options
Subscribe to this journal
Receive 12 print issues and online access
$209.00 per year
only $17.42 per issue
Buy this article
- Purchase on Springer Link
- Instant access to full article PDF
Prices may be subject to local taxes which are calculated during checkout
Similar content being viewed by others
References
Maren, S. & Quirk, G.J. Neuronal signalling of fear memory. Nat. Rev. Neurosci. 5, 844–852 (2004).
LeDoux, J.E. Emotion circuits in the brain. Annu. Rev. Neurosci. 23, 155–184 (2000).
Herry, C. & Johansen, J.P. Encoding of fear learning and memory in distributed neuronal circuits. Nat. Neurosci. 17, 1644–1654 (2014).
Gründemann, J. & Lüthi, A. Ensemble coding in amygdala circuits for associative learning. Curr. Opin. Neurobiol. 35, 200–206 (2015).
Rescorla, R.A. Probability of shock in the presence and absence of CS in fear conditioning. J. Comp. Physiol. Psychol. 66, 1–5 (1968).
Rescorla, R.A. & Wagner, A.R. A theory of pavlovian conditioning: variations in the effectiveness of reinforcement and nonreinforcement. in Classical Conditioning II Current Research and Theory (eds. Black, A.H. & Prokasy, W.F.) 64–99 (Appleton-Century-Crofts, New York, 1972).
Van Hamme, L.J. & Wasserman, E.A. Cue competition in causality judgments: the role of nonpresentation of compound stimulus elements. Learn. Motiv. 25, 127–151 (1994).
Stout, S.C. & Miller, R.R. Sometimes-competing retrieval (SOCR): a formalization of the comparator hypothesis. Psychol. Rev. 114, 759–783 (2007).
Pearce, J.M. & Bouton, M.E. Theories of associative learning in animals. Annu. Rev. Psychol. 52, 111–139 (2001).
Kamin, L.J. Attention-like processes in classical conditioning. in Miami Symposium On The Production Of Behavior Aversive Stimulation (ed. Jones, M.R.) 9–33 (University of Miami Press, 1968).
Bauer, E.P., LeDoux, J.E. & Nader, K. Fear conditioning and LTP in the lateral amygdala are sensitive to the same stimulus contingencies. Nat. Neurosci. 4, 687–688 (2001).
Holyoak, K.J. & Cheng, P.W. Causal learning and inference as a rational process: the new synthesis. Annu. Rev. Psychol. 62, 135–163 (2011).
Kim, J.J., DeCola, J.P., Landeira-Fernandez, J. & Fanselow, M.S. N-Methyl-D-aspartate receptor antagonist APV blocks acquisition but not expression of fear conditioning. Behav. Neurosci. 105, 126–133 (1991).
McNish, K.A., Gewirtz, J.C. & Davis, M. Disruption of contextual freezing, but not contextual blocking of fear-potentiated startle, after lesions of the dorsal hippocampus. Behav. Neurosci. 114, 64–76 (2000).
Durlach, P.J. Effect of signaling intertrial unconditioned stimuli in autoshaping. J. Exp. Psychol. Anim. Behav. Process. 9, 374–389 (1983).
Gunther, L.M. & Miller, R.R. Prevention of the degraded-contingency effect by signalling training trials. Q. J. Exp. Psychol. B 53, 97–119 (2000).
Balleine, B.W., Killcross, A.S. & Dickinson, A. The effect of lesions of the basolateral amygdala on instrumental conditioning. J. Neurosci. 23, 666–675 (2003).
Bermudez, M.A. & Schultz, W. Responses of amygdala neurons to positive reward-predicting stimuli depend on background reward (contingency) rather than stimulus-reward pairing (contiguity). J. Neurophysiol. 103, 1158–1170 (2010).
Rogan, M.T., Stäubli, U.V. & LeDoux, J.E. Fear conditioning induces associative long-term potentiation in the amygdala. Nature 390, 604–607 (1997).
Chow, B.Y. et al. High-performance genetically targetable optical neural silencing by light-driven proton pumps. Nature 463, 98–102 (2010).
Johansen, J.P. et al. Hebbian and neuromodulatory mechanisms interact to trigger associative memory formation. Proc. Natl. Acad. Sci. USA 111, E5584–E5592 (2014).
Goosens, K.A., Hobin, J.A. & Maren, S. Auditory-evoked spike firing in the lateral amygdala and Pavlovian fear conditioning: mnemonic code or fear bias? Neuron 40, 1013–1022 (2003).
Prévost, C., McNamee, D., Jessup, R.K., Bossaerts, P. & O'Doherty, J.P. Evidence for model-based computations in the human amygdala during Pavlovian conditioning. PLoS Comput. Biol. 9, e1002918 (2013).
Saez, A., Rigotti, M., Ostojic, S., Fusi, S. & Salzman, C.D. Abstract context representations in primate amygdala and prefrontal cortex. Neuron 87, 869–881 (2015).
Pearl, J. Probabilistic Reasoning in Intelligent Systems (Morgan Kauffmann, San Mateo, California, USA, 1988).
Griffiths, T.L. & Tenenbaum, J.B. Structure and strength in causal induction. Cognit. Psychol. 51, 334–384 (2005).
Hall, G. & Symonds, M. Overshadowing and latent inhibition of context aversion conditioning in the rat. Auton. Neurosci. 129, 42–49 (2006).
Witnauer, J.E. & Miller, R.R. Degraded contingency revisited: posttraining extinction of a cover stimulus attenuates a target cue's behavioral control. J. Exp. Psychol. Anim. Behav. Process. 33, 440–450 (2007).
Matzel, L.D., Schachtman, T.R. & Miller, R.R. Recovery of an overshadowed association achieved by extinction of the overshadowing stimulus. Learn. Motiv. 16, 398–412 (1985).
Koller, D. & Friedman, N. Probabilistic Graphical Models: Principles and Techniques, Chs. XVII and XVIII, pp. 726, 784 (MIT Press, 2009).
Quirk, G.J. et al. Erasing fear memories with extinction training. J. Neurosci. 30, 14993–14997 (2010).
Karlsson, M.P., Tervo, D.G.R. & Karpova, A.Y. Network resets in medial prefrontal cortex mark the onset of behavioral uncertainty. Science 338, 135–139 (2012).
Behrens, T.E.J., Woolrich, M.W., Walton, M.E. & Rushworth, M.F.S. Learning the value of information in an uncertain world. Nat. Neurosci. 10, 1214–1221 (2007).
Likhtik, E. & Paz, R. Amygdala-prefrontal interactions in (mal)adaptive learning. Trends Neurosci. 38, 158–166 (2015).
Yu, A.J. & Dayan, P. Uncertainty, neuromodulation, and attention. Neuron 46, 681–692 (2005).
Rosas, J.M., Todd, T.P. & Bouton, M.E. Context change and associative learning. Wiley Interdiscip. Rev. Cogn. Sci. 4, 237–244 (2013).
Courville, A., Daw, N.D., Gordon, G. & Touretzky, D.S. Model uncertainty in classical conditioning. Adv. Neural Inf. Process. Syst. 16, 977–984 (2004).
Gershman, S.J., Blei, D.M. & Niv, Y. Context, learning, and extinction. Psychol. Rev. 117, 197–209 (2010).
Soto, F.A., Gershman, S.J. & Niv, Y. Explaining compound generalization in associative and causal learning through rational principles of dimensional generalization. Psychol. Rev. 121, 526–558 (2014).
Uwano, T., Nishijo, H., Ono, T. & Tamura, R. Neuronal responsiveness to various sensory stimuli, and associative learning in the rat amygdala. Neuroscience 68, 339–361 (1995).
Romanski, L.M., Clugnet, M.C., Bordi, F. & LeDoux, J.E. Somatosensory and auditory convergence in the lateral nucleus of the amygdala. Behav. Neurosci. 107, 444–450 (1993).
Weinberger, N.M. Auditory associative memory and representational plasticity in the primary auditory cortex. Hear. Res. 229, 54–68 (2007).
Pecevski, D., Buesing, L. & Maass, W. Probabilistic inference in general graphical models through sampling in stochastic networks of spiking neurons. PLoS Comput. Biol. 7, e1002294 (2011).
Lochmann, T. & Deneve, S. Neural processing as causal inference. Curr. Opin. Neurobiol. 21, 774–781 (2011).
Nessler, B., Pfeiffer, M., Buesing, L. & Maass, W. Bayesian computation emerges in generic cortical microcircuits through spike-timing-dependent plasticity. PLoS Comput. Biol. 9, e1003037 (2013).
Pouget, A., Beck, J.M., Ma, W.J. & Latham, P.E. Probabilistic brains: knowns and unknowns. Nat. Neurosci. 16, 1170–1178 (2013).
Chistiakova, M., Bannon, N.M., Bazhenov, M. & Volgushev, M. Heterosynaptic plasticity: multiple mechanisms and multiple roles. Neuroscientist 20, 483–498 (2014).
Abraham, W.C. How long will long-term potentiation last? Phil. Trans. R. Soc. Lond. B 358, 735–744 (2003).
Friedman, N. & Koller, D. Being Bayesian about network structure. A Bayesian approach to structure discovery in Bayesian networks. Mach. Learn. 50, 95–125 (2003).
Kappel, D., Habenschuss, S., Legenstein, R. & Maass, W. Network plasticity as Bayesian inference. PLoS Comput. Biol. 11, e1004485 (2015).
Johansen, J.P., Tarpley, J.W., LeDoux, J.E. & Blair, H.T. Neural substrates for expectation-modulated fear learning in the amygdala and periaqueductal gray. Nat. Neurosci. 13, 979–986 (2010).
Acknowledgements
We thank J. Gardner, W.J. Ma and C. Yokoyama for advice and comments on the manuscript and N. Daw for discussions during the course of this work. We thank the UNC vector core and E. Boyden (MIT) for the lentiviral vectors. This study was funded by National Institute of Mental Health (NIMH) grants R01-MH046516 and R01-MH38774, National Institute on Drug Abuse (NIDA) grant DA029053 (J.E.L.), MEXT Strategic Research Program for Brain Sciences (11041047) and Grants-in-Aid for Scientific Research (25710003, 25116531, 15H04264, 16H01291, 15H01301) (J.P.J.), and an NYU Williamson Fellowship (T.J.M.).
Author information
Authors and Affiliations
Contributions
T.J.M. designed the experiments, collected and analyzed data, developed the computational models and wrote the manuscript. J.P.J. designed the experiments, collected data and wrote the manuscript. L.D.-M. collected and analyzed electrophysiology data. O.A. collected data. E.A.Y. collected and analyzed single-cell electrophysiology data. J.E.L. contributed to data interpretation and the final version of the manuscript.
Corresponding authors
Ethics declarations
Competing interests
The authors declare no competing financial interests.
Integrated supplementary information
Supplementary Figure 1 Correlations between tone and context freezing by animal in each of the four groups in experiment 1.
Every animal is represented by a blue circle. Correlation was measured by Spearman’s rank correlation coefficient (ρ).
Supplementary Figure 2 Comparisons of context memory between groups are stable over time and invariant to the salience of the conditioning context.
(a) Minute-by-minute analysis of freezing during the context test for the groups in experiment 1 (and Pairings Last, included for illustration but not in the statistical analysis). A repeated measures ANOVA showed no Time*Contingency*Spacing interaction (n = 16,17,18,20,21,F1,288 = 1.96, P = 0.17). A comparison restricted to the massed condition (between CTL II and Pairings First) also showed no Time*Contingency interaction (F 1, 144) = 0.74, P =0.40). (b) Comparison of CTL II and Pairings First groups with conditioning and context test performed in a more salient context (lit by a visible light and with citrus odor). Reduction in Tone memory matched previous result (ratio between CTLII and Pairings First 0.63 vs. 0.66 originally), whereas Context memory was similar between the groups, as before. Error bars indicate s.e.m.
Supplementary Figure 3 Defecation gives similar results to freezing for experiments described in Figure 2.
(a) Contingency degradation in the Intermixed condition with APV infusion in dorsal hippocampus (DH) prior to conditioning, as measured by defecation during tone test. (n = 9,9, Mann-Whitney U test, U = 17.5, P = 0.04). (b) Impaired contextual aversive memory, as measured by defecation, following APV infusion in DH prior to conditioning (n = 9,7, Mann-Whitney U test, U = 9.5, P = 0.018). Error bars indicate s.e.m.
Supplementary Figure 4 Effect of repeated USs depends on contingencies.
(a) Behavioral data (left) and model simulation (right) for conditioning with 15, and 21 CS-US pairings (n=9, 7). Adding further shocks paired with the same tone CS (21 pairings in total) did not reduce tone memory strength. (b) Behavioral data (left) and model simulation (right) for the cover stimulus effect. Signaling shocks with a second CS (in this case a flashing light), instead of giving unsignaled shocks attenuates contingency degradation (comparison between Pairings First and Cover Stimulus groups, n = 18, 12, unpaired sample t-test, t28 = 2.42, * P = 0.022) Error bars indicate s.e.m.
Supplementary Figure 7 Effects of contingency on amygdala LFP potentiation.
(a) Example traces before, and after conditioning for a representative animal each in the Control II group (left) and Pairings First group (right). Red arrows indicate the peak depolarization. (b) Averaged peak depolarizations in the CTL II and Pairings First groups before (Habituation), and after (LTM) conditioning. There was a marginally significant interaction between time and contingency (n=10, 8, repeated measures ANOVA, F1,16 = 4.30, P=0.055), and a simple effects analysis showed significant potentiation of the LFP response in the CTL II, but not the Pairings First condition (F1,16 = 18.0, P = 0.001 and F1,16 = 1.022, P = 0.33), further indicating that conditioning differentially effects synaptic processing depending on contingency. Error bars indicate s.e.m.
Supplementary Figure 8 Comparison of SLM to behavioral results for experiments described in Figure 1.
Direct comparison of behavioral data (top panel) and SLM (bottom panel) for experiment 1. Error bars indicate s.e.m.
Supplementary Figure 10 SLM’s predictions for further conditioning phenomena.
(a) Cover stimulus effect: Replacing unsignaled USs by USs signaled by a second discrete cue (e.g. a light) reverses the effects of contingency degradation. (b) Overshadowing: Conditioning to a single cue (Tone) is reduced if it is trained in compound with a second cue (Light). (c) Recovery from overshadowing: Unreinforced presentations of the overshadowing second cue (Light) restores the level of responding to the first cue. (d) Blocking: Initial conditioning to a Light reduces subsequent conditioning to the Tone when the Tone is conditioned in compound with the Light.
Supplementary Figure 11 Graphical illustration of the fits of some of the different models compared.
(a) Behavioral Data. (b) Bayesian model that learned both structure and parameters (SPLM). (c) Bayesian model that learned parameters using Graph 6 (from Fig. 5a) and the best Beta priors for edge parameters. (d) Van Hamme and Wasserman’s extension of the Rescorla-Wagner model.
Supplementary information
Supplementary Text and Figures
Supplementary Figures 1–11, Supplementary Tables 1–8 and Supplementary Modeling (PDF 2320 kb)
Rights and permissions
About this article
Cite this article
Madarasz, T., Diaz-Mataix, L., Akhand, O. et al. Evaluation of ambiguous associations in the amygdala by learning the structure of the environment. Nat Neurosci 19, 965–972 (2016). https://doi.org/10.1038/nn.4308
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1038/nn.4308
This article is cited by
-
Long time-scales in primate amygdala neurons support aversive learning
Nature Communications (2018)