Abstract
A fundamental goal of motor learning is to establish the neural patterns that produce a desired behavioral outcome. It remains unclear how and when the nervous system solves this 'credit assignment' problem. Using neuroprosthetic learning, in which we could control the causal relationship between neurons and behavior, we found that sleep-dependent processing was required for credit assignment and the establishment of task-related functional connectivity reflecting the casual neuron–behavior relationship. Notably, we observed a strong link between the microstructure of sleep reactivations and credit assignment, with downscaling of non-causal activity. Decoupling of spiking to slow oscillations using optogenetic methods eliminated rescaling. Thus, our results suggest that coordinated firing during sleep is essential for establishing sparse activation patterns that reflect the causal neuron-behavior relationship.
This is a preview of subscription content, access via your institution
Access options
Access Nature and 54 other Nature Portfolio journals
Get Nature+, our best-value online-access subscription
$29.99 / 30 days
cancel any time
Subscribe to this journal
Receive 12 print issues and online access
$209.00 per year
only $17.42 per issue
Buy this article
- Purchase on Springer Link
- Instant access to full article PDF
Prices may be subject to local taxes which are calculated during checkout
Similar content being viewed by others
Change history
18 July 2017
In the version of this article initially published online, the abstract read "casual neuron–behavior relationship" instead of "causal neuron–behavior relationship." The error has been corrected in the print, PDF and HTML versions of this article.
31 July 2017
In the version of this article initially published online, the x-axis label for the righthand column in each graph in Figure 6b read BMI1Early; it should have read BMI2Early. The error has been corrected in the print, PDF and HTML versions of this article.
References
Yin, H.H. et al. Dynamic reorganization of striatal circuits during the acquisition and consolidation of a skill. Nat. Neurosci. 12, 333–341 (2009).
Dayan, E. & Cohen, L.G. Neuroplasticity subserving motor skill learning. Neuron 72, 443–454 (2011).
Tumer, E.C. & Brainard, M.S. Performance variability enables adaptive plasticity of 'crystallized' adult birdsong. Nature 450, 1240–1244 (2007).
Shmuelof, L. & Krakauer, J.W. Are we ready for a natural history of motor learning? Neuron 72, 469–476 (2011).
Peters, A.J., Chen, S.X. & Komiyama, T. Emergence of reproducible spatiotemporal activity during motor learning. Nature 510, 263–267 (2014).
Ganguly, K. & Carmena, J.M. Emergence of a stable cortical map for neuroprosthetic control. PLoS Biol. 7, e1000153 (2009).
Huber, D. et al. Multiple dynamic representations in the motor cortex during sensorimotor learning. Nature 484, 473–478 (2012).
Ganguly, K., Dimitrov, D.F., Wallis, J.D. & Carmena, J.M. Reversible large-scale modification of cortical networks during neuroprosthetic control. Nat. Neurosci. 14, 662–667 (2011).
Abbott, L.F., DePasquale, B. & Memmesheimer, R.M. Building functional networks of spiking model neurons. Nat. Neurosci. 19, 350–355 (2016).
Lee, D., Seo, H. & Jung, M.W. Neural basis of reinforcement learning and decision making. Annu. Rev. Neurosci. 35, 287–308 (2012).
Clancy, K.B., Koralek, A.C., Costa, R.M., Feldman, D.E. & Carmena, J.M. Volitional modulation of optically recorded calcium signals during neuroprosthetic learning. Nat. Neurosci. 17, 807–809 (2014).
Tononi, G. & Cirelli, C. Sleep and the price of plasticity: from synaptic and cellular homeostasis to memory consolidation and integration. Neuron 81, 12–34 (2014).
Diekelmann, S. & Born, J. The memory function of sleep. Nat. Rev. Neurosci. 11, 114–126 (2010).
Genzel, L., Kroes, M.C., Dresler, M. & Battaglia, F.P. Light sleep versus slow wave sleep in memory consolidation: a question of global versus local processes? Trends Neurosci. 37, 10–19 (2014).
Cramer, S.C. et al. Motor cortex activation is preserved in patients with chronic hemiplegic stroke. Ann. Neurol. 52, 607–616 (2002).
Marshall, L. & Born, J. The contribution of sleep to hippocampus-dependent memory consolidation. Trends Cogn. Sci. 11, 442–450 (2007).
Wilson, M.A. & McNaughton, B.L. Reactivation of hippocampal ensemble memories during sleep. Science 265, 676–679 (1994).
Nere, A., Hashmi, A., Cirelli, C. & Tononi, G. Sleep-dependent synaptic down-selection (I): modeling the benefits of sleep on memory consolidation and integration. Front. Neurol. 4, 143 (2013).
Jarosiewicz, B. et al. Functional network reorganization during learning in a brain-computer interface paradigm. Proc. Natl. Acad. Sci. USA 105, 19486–19491 (2008).
Koralek, A.C., Jin, X., Long, J.D. II, Costa, R.M. & Carmena, J.M. Corticostriatal plasticity is necessary for learning intentional neuroprosthetic skills. Nature 483, 331–335 (2012).
Taylor, D.M., Tillery, S.I. & Schwartz, A.B. Direct cortical control of 3D neuroprosthetic devices. Science 296, 1829–1832 (2002).
Moritz, C.T., Perlmutter, S.I. & Fetz, E.E. Direct control of paralyzed muscles by cortical neurons. Nature 456, 639–642 (2008).
Gulati, T., Ramanathan, D.S., Wong, C.C. & Ganguly, K. Reactivation of emergent task-related ensembles during slow-wave sleep after neuroprosthetic learning. Nat. Neurosci. 17, 1107–1113 (2014).
Gulati, T. et al. Robust neuroprosthetic control from the stroke perilesional cortex. J. Neurosci. 35, 8653–8661 (2015).
Fetz, E.E. Volitional control of neural activity: implications for brain-computer interfaces. J. Physiol. (Lond.) 579, 571–579 (2007).
Koralek, A.C., Costa, R.M. & Carmena, J.M. Temporally precise cell-specific coherence develops in corticostriatal networks during learning. Neuron 79, 865–872 (2013).
Orsborn, A.L. et al. Closed-loop decoder adaptation shapes neural plasticity for skillful neuroprosthetic control. Neuron 82, 1380–1393 (2014).
Mitchell, J.F., Sundberg, K.A. & Reynolds, J.H. Spatial attention decorrelates intrinsic activity fluctuations in macaque area V4. Neuron 63, 879–888 (2009).
Watson, B.O., Levenstein, D., Greene, J.P., Gelinas, J.N. & Buzsáki, G. Network homeostasis and state dynamics of neocortical sleep. Neuron 90, 839–852 (2016).
Peyrache, A., Khamassi, M., Benchenane, K., Wiener, S.I. & Battaglia, F.P. Replay of rule-learning related neural patterns in the prefrontal cortex during sleep. Nat. Neurosci. 12, 919–926 (2009).
Ramanathan, D.S., Gulati, T. & Ganguly, K. Sleep-dependent reactivation of ensembles in motor cortex promotes skill consolidation. PLoS Biol. 13, e1002263 (2015).
Lansink, C.S., Goltstein, P.M., Lankelma, J.V., McNaughton, B.L. & Pennartz, C.M. Hippocampus leads ventral striatum in replay of place-reward information. PLoS Biol. 7, e1000173 (2009).
de Lavilléon, G., Lacroix, M.M., Rondi-Reig, L. & Benchenane, K. Explicit memory creation during sleep demonstrates a causal role of place cells in navigation. Nat. Neurosci. 18, 493–495 (2015).
Singer, A.C. & Frank, L.M. Rewarded outcomes enhance reactivation of experience in the hippocampus. Neuron 64, 910–921 (2009).
Churchland, M.M. et al. Stimulus onset quenches neural variability: a widespread cortical phenomenon. Nat. Neurosci. 13, 369–378 (2010).
Song, W. & Giszter, S.F. Adaptation to a cortex-controlled robot attached at the pelvis and engaged during locomotion in rats. J. Neurosci. 31, 3110–3128 (2011).
Miyamoto, D. et al. Top-down cortical input during NREM sleep consolidates perceptual memory. Science 352, 1315–1318 (2016).
Chuong, A.S. et al. Noninvasive optical inhibition with a red-shifted microbial rhodopsin. Nat. Neurosci. 17, 1123–1129 (2014).
Steriade, M., Nuñez, A. & Amzica, F. A novel slow (<1 Hz) oscillation of neocortical neurons in vivo: depolarizing and hyperpolarizing components. J. Neurosci. 13, 3252–3265 (1993).
Yang, G. et al. Sleep promotes branch-specific formation of dendritic spines after learning. Science 344, 1173–1178 (2014).
de Vivo, L. et al. Ultrastructural evidence for synaptic scaling across the wake/sleep cycle. Science 355, 507–510 (2017).
Maret, S., Faraguna, U., Nelson, A.B., Cirelli, C. & Tononi, G. Sleep and waking modulate spine turnover in the adolescent mouse cortex. Nat. Neurosci. 14, 1418–1420 (2011).
Gupta, A.S., van der Meer, M.A., Touretzky, D.S. & Redish, A.D. Hippocampal replay is not a simple function of experience. Neuron 65, 695–705 (2010).
O'Doherty, J.P., Cockburn, J. & Pauli, W.M. Learning, reward, and decision making. Annu. Rev. Psychol. 68, 73–100 (2017).
Schultz, W. Behavioral theories and the neurophysiology of reward. Annu. Rev. Psychol. 57, 87–115 (2006).
Ishii, S., Yoshida, W. & Yoshimoto, J. Control of exploitation-exploration meta-parameter in reinforcement learning. Neural Netw. 15, 665–687 (2002).
Wallstrom, G., Liebner, J. & Kass, R.E. An implementation of Bayesian adaptive regression splines (BARS) in C with S and R Wrappers. J. Stat. Softw. 26, 1–21 (2008).
Mitra, P. & Bokil, H. Observed Brain Dynamics (Oxford University Press, 2008).
Acknowledgements
This work was supported by awards from the Department of Veterans Affairs, Veterans Health Administration (VA Merit: 1I01RX001640 to K.G., VA CDA 1IK2BX003308 to D.S.R.); the National Institute of Neurological Disorders and Stroke (1K99NS097620 to T.G. and 5K02NS093014 to K.G.); the American Heart/Stroke Association (15POST25510020 to T.G.); the Burroughs Wellcome Fund (1009855 to K.G.); and start-up funds from the SFVAMC, NCIRE and UCSF Department of Neurology (to K.G.).
Author information
Authors and Affiliations
Contributions
T.G. and K.G. conceived of the experiments. L.G. and T.G. performed surgical procedures and collected the data. A.B., D.S.R. and T.G. analyzed the data. T.G. and K.G. wrote the manuscript. L.G. and D.S.R. edited the manuscript.
Corresponding author
Ethics declarations
Competing interests
The authors declare no competing financial interests.
Integrated supplementary information
Supplementary Figure 1 Modulations of TRD neurons are not significantly high during random time points in sleep.
a, Mean modulation of all TRD and TRI neurons at randomly picked times during Sleeppost. For this analysis, snippets of activity (50 ms window of spiking activity that was then binned at 5 ms resolution) were randomly sampled from all the reactivation times. b, Scatter plots showing the modulation depth for all the individual TRD and TRI (mean in solid line ± s.e.m. in box; unpaired t test t121 = 0.69, P = 0.49).
Supplementary Figure 2 Depth modulation during sleep reactivations predicts rescaling of task–related firing rate during BMI2.
MDreactivation from Sleeppost (in Fig 2c) are compared to changes in modulation from BMI1 to BMI2 for both TRD and TRI neurons (linear regression R2 = 0.17, P < 10-5).
Supplementary Figure 3 Characteristics of OPTOUP and OPTODOWN experiments.
a, Comparison of the total number of 100 ms optogenetic stimulation periods for OPTOUP and OPTODOWN experiments (mean in solid line ± s.e.m. in box; color convention same as Fig 5; unpaired t test t17 = 0.99, P = 0.33). b, Comparison of the proportion of total time that the LED was turned on for the respective Sleeppost during the OPTOUP and OPTODOWN experiments (mean in solid line ± s.e.m. in box; color convention same as Fig 5; unpaired t test t17 = 2.07, P = 0.054).
Supplementary Figure 4 Sleep durations in optogenetic experiments.
Comparison of the total sleep durations for Sleeppre and Sleeppost during the OPTOUP, OPTODOWN and OPTOOFF experiments (mean in solid line ± s.e.m. in box, one-way ANOVA, F5,48 = 0.92, P = 0.47; post hoc t test shows no significant difference between any pairwise comparison).
Supplementary Figure 5 PC weights for PCA based reactivation analysis.
Examples of the first principal components from two separate experiments. Notably, TRD and TRI neurons both had non-zero weights (TRD enclosed in red box).
Supplementary information
Supplementary Text and Figures
Supplementary Figures 1–5. (PDF 522 kb)
Rights and permissions
About this article
Cite this article
Gulati, T., Guo, L., Ramanathan, D. et al. Neural reactivations during sleep determine network credit assignment. Nat Neurosci 20, 1277–1284 (2017). https://doi.org/10.1038/nn.4601
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1038/nn.4601
This article is cited by
-
Spindle oscillations in communicating axons within a reconstituted hippocampal formation are strongest in CA3 without thalamus
Scientific Reports (2024)
-
Probing pathways by which rhynchophylline modifies sleep using spatial transcriptomics
Biology Direct (2023)
-
Emergence of cortical network motifs for short-term memory during learning
Nature Communications (2023)
-
Emergence of task-related spatiotemporal population dynamics in transplanted neurons
Nature Communications (2023)
-
Cortical–hippocampal coupling during manifold exploration in motor cortex
Nature (2023)