Abstract
We have long known that dopamine encodes the predictive relationship between cues and rewards. But what about relief learning? In this issue of Neuropsychopharmacology, Mayer et al. show that the same circuits encoding rewarding events also encode relief from aversive events. And this appears to be in a manner distinct from encoding of the aversive event itself. So does dopamine only contribute to learning about positive events? And are these events encoded in the same way regardless of how that positive experience came about? Not quite. Turns out, the devil is in the details.
Going to Paris for the first time feels good. Particularly if you have managed to swing five-star accommodation by the river Seine for you and your significant other. Severe turbulence on the plane as you travel over the Atlantic Ocean to Paris feels bad. Being trapped in your seat while screaming babies and nervous fliers create an ominous cabin atmosphere is enough to ruin your relaxed holiday mood and put you on the edge. But seeing the seatbelt sign go off and the air hosts get up from their seats to resume service provides a sense of relief. In fact, this feeling of relief from the turbulence ending is thought to be processed cognitively in much the same way as being in Paris itself [1,2,3]. Such a conception reflects the traditional view that we are wired to process information in a binary fashion, good and bad. But does our brain really process information in this manner? In this issue of Neuropsychopharmacology, Mayer et al. [4] provide causal evidence to support a case that it does.
To test relief learning in the lab, Mayer et al. presented rats with a mild footshock followed by a flashing light. Just like the seat belt sign going off signals the end of turbulence, here flashing light becomes associated with the cessation of shock and should provide a sense of relief. Indeed, during a subsequent test the “relief” light reduced the usual startle to a novel loud stimulus, showing that the relief light had acquired a positive valence that could counteract the fear produced by the loud noise. So, would the same neural circuits that are involved in reward learning also be involved in the attribution of positive valence to a cue which elicits relief?
To answer this question, Mayer et al. tested whether the mesolimbic dopamine system—a critical component of the reward circuit [5]—is also necessary for relief learning. Mayer et al. injected a toxin (6-OHDA) to destroy dopamine neurons in the posterior medial ventral tegmental area (pmVTA) in a group of rats and then tested the rats’ ability to show relief learning. In support of their hypothesis, Mayer et al. demonstrated that lesions of dopamine neurons in the pmVTA impaired learning to attribute positive valence to the relief light. That is, the 6-OHDA lesioned rats failed to show the reduction in the startle response when the relief light was presented. Instead, these rats showed equally high levels of startle to the loud noise regardless of whether the cue was present or not.
Next, Mayer et al. set out to investigate the specific pathway that might be involved in producing the effect of the dopamine lesions on relief learning. Mayer et al.’s previous work indicated that the shell of the nucleus accumbens (AcbSh) was important for relief learning [6]. Thus, they reasoned that dopamine neurons projecting from the pmVTA to the AcbSh might be endowing the relief light with positive valence (in much the same way that this pathway regulates learning about cues predicting reward [7]). To test this idea, Mayer et al. chemogenetically inactivated dopamine neurons in the VTA projecting to the AcbSh during relief learning. Sure enough, inactivation of this pathway during learning prevented the reduction in startle to the loud noise when the relief light was presented later, suggesting a failure to attribute positive valence to the relief light.
But could this pathway simply be involved in attributing any kind of valence to sensory cues? That is, perhaps a role of the pmVTA dopamine neurons in both reward and relief learning reflects a general role of these neurons in associative learning, whether the outcome is rewarding or not. The data obtained by Mayer et al. suggest this is not the case. Following relief learning, they tested whether manipulation of the dopamine system would affect the ability of rats to attribute negative valence to another sensory cue. To do this, rats received another session with additional presentations of the mild footshocks. However, this time a tone was presented just prior to the delivery of the footshocks instead of after the shock has ceased. This meant that the tone came to predict the imminent arrival of the footshock, rather than its cessation. In this case, presentation of the “aversive” tone resulted in an increase in the startle to the novel loud stimulus, showing the attribution of negative valence to the aversive tone.
Interestingly, lesions of dopamine neurons in the pmVTA had no effect on the rats’ ability to assign negative valence to the aversive tone. All rats subsequently showed a potentiation of the startle response to the loud noise in the presence of the aversive tone. This is even though the only difference between this procedure and the dopamine-dependent relief procedure was that the aversive tone was presented before and not after the footshock. This clever design demonstrates that the dopamine circuit is important for the attribution of positive valence to cues in a manner quite distinct from the attribution of negative valence in this preparation.
Taken together, the data from Mayer et al.’s elegant study mount a convincing case that the mesolimbic dopamine system contributes to relief learning in much the same way that it contributes to reward learning. In terms of the wider literature, one is tempted to paint a picture of the dopamine system—once again—as a center for all things rewarding. The brain’s answer to Disneyland, if you like. So, perhaps we do process all events that harbor positive valence in the same way, regardless of the specific details of how that positive experience came about? And perhaps there are specific circuits of the brain designed to encode positive valence, and those which encode negative valence?
But there is another aspect of the data presented by Mayer et al. that pose a problem for such a simplistic view. In a final experiment, they tested whether lesioning of dopamine neurons in the pmVTA would affect safety learning. Here, rats were exposed to mild footshocks, but now another tone they had not experienced before was presented in between presentations of the footshocks. In this way, the tone signaled the explicit absence of the footshock as the shock could not be presented on either side of this “safety” tone. Rats learned to attribute positive valence to the safety tone, again indexed by their reduction in startle to the loud noise when the safety tone was presented. Under these circumstances, damage to dopamine neurons in the pmVTA had no effect on the ability of rats to attribute positive valence to the safety tone. Rats in both the lesion and control groups showed the reduction in startle in the presence of the safety tone. These data suggest that dopamine neurons do care about the way in which positive valence is acquired. That is, qualitative changes in our experience alters the way the dopamine system encodes that experience.
The complexity of the role for dopamine in learning is also suggested by the electrophysiological profile of dopamine neurons garnered from the wider literature. Dopamine neurons do not just respond to reward or events which predict something rewarding is about to happen. Rather, changes in the activity of dopamine neurons are seen to neutral and even aversive stimuli [8,9,10]. What is more, it that the responses are far from binary. For example, while some dopamine neurons show decreases in activity to aversive stimuli, others show increases in responding to these events [8, 10]. Not exactly what you would expect from a region encoding positive valence. So how should we interpret these mixed signals? Can something bad be good? Or does this contradiction illustrate that we should move away from focusing on a dichotomy of good and bad and consider instead the associative structure of what is learned and how that learning came about? Far from bringing the debate to a close, the exciting study by Mayer et al. opens up novel directions for causal research investigating how the dopamine system might contribute differently to distinct forms of learning—regardless of whether the experience is good or bad. In the end, the story might not turn out to be as simple as black and white, but more like 50 shades of gray.
References
Dickinson A, Dearing MF. Appetitive-aversive interactions and inhibitory processes. In: Dickson A, Boakes RA, editors. Mechanisms of learning and motivation: a memorial volume to Jerzy Konorski. Hillsdale, NJ: Erlbaum; 1979. p. 203–31.
Seymour B, O’doherty JP, Koltzenburg M, Wiech K, Frackowiak R, Friston K, et al. Opponent appetitive-aversive neural processes underlie predictive learning of pain relief. Nat Neurosci. 2005;8:1234.
Solomon RL, Corbit JD. An opponent-process theory of motivation. Am Econ Rev. 1978; 68:12–24.
Mayer D, Kahl E, Uzuneser TC, Fendt M. Role of the mesolimbic dopamine system in relief learning. Neuropsychopharmacol. 2018; 43:1651–1659
Wise RA. Dopamine, learning and motivation. Nat Rev Neurosci. 2004;5:483.
Mohammadi M, Bergado-Acosta JR, Fendt M. Relief learning is distinguished from safety learning by the requirement of the nucleus accumbens. Behav Brain Res. 2014;272:40–5.
Day JJ, Carelli RM. The nucleus accumbens and Pavlovian reward learning. Neuroscientist. 2007;13:148–59.
Cohen JY, Haesler S, Vong L, Lowell BB, Uchida N. Neuron-type-specific signals for reward and punishment in the ventral tegmental area. Nature. 2012;482:85.
Sadacca BF, Jones JL, Schoenbaum G. Midbrain dopamine neurons compute inferred and cached value prediction errors in a common framework. eLife. 2016;5:e13665.
Ungless MA, Magill PJ, Bolam JP. Uniform inhibition of dopamine neurons in the ventral tegmental area by aversive stimuli. Science. 2004;303:2040–2.
Acknowledgements
This work was supported by an NHMRC CJ Martin awarded to MJS. The opinions expressed in this article are the authors’ own and do not reflect the view of the NIH/DHHS.
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Competing interests
The authors declare no competing interests.
Additional information
Publisher's note: Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Sharpe, M.J. What a relief! A role for dopamine in positive (but not negative) valence. Neuropsychopharmacol 43, 1–2 (2018). https://doi.org/10.1038/s41386-018-0036-6
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1038/s41386-018-0036-6