Information

Does STDP make the Hebbian learning rule redundant?

Does STDP make the Hebbian learning rule redundant?


We are searching data for your request:

Forums and discussions:
Manuals and reference books:
Data from registers:
Wait the end of the search in all databases.
Upon completion, a link will appear to access the found materials.

On Scholarpedia they introduce STDP (spike timing dependent plasticity) as a temporally asymmetric form of Hebbian learning, making it sound as if the original Hebbian rule still has relevance in neuroscience. On the wiki page on Hebbian theory, however, it seems SDTP and Hebbian theory are pretty much synonyms.

Do all cases of neuron types that were previously thought to follow the Hebbian learning rule (fire together, wire together) actually follow the STDP rule? If not, are there statistics about how often each learning rule occurs in the brain?


Short answer
Just like Mendel's ideas have been proven by the existence of DNA, Hebbian learning can be explained by STDP. That doesn't make Mendel's or Hebb's teachings obsolete.

Background
STDP is the long-term synaptic potentiation or depression governed by the timing of pre- and post synaptic cell firing. STDP depends on certain “critical windows” for spike timing. This critical window is on the order of tens of milliseconds (Dan & Poo, 2004). STDP is known for quite some time, but is by far a more novel idea than the idea of Hebbian learning posed in the 1940's (Markram et al., 2012). Hebbian learning has been popularly described as:

["C]ells that fire together, wire together.” In other words, if things keep happening more or less simultaneously, you may assume that there is a common cause for the firing. More importantly, if one of the cell is active systematically just slightly before another, the firing of the first one might have a causal link to the firing of the second one and this causal link could be remembered by increasing the wiring of connections, a notion we call synaptic plasticity. In short, timing matters because it may indicate causality.

In other words, Hebbian learning is mediated by STDP. Thus STDP may explain how Hebbian learning may come to exist.

But Markram et al. caution:

… this is not to say that STDP has been a panacea for all problems neuroscientific. Clearly cells that fire together wire together, there is no doubt about that, so the coming of STDP has not rendered the classical literature obsolete by any means.

In other words, Hebbian learning is more of a theoretical construct whereas STDP provides a neurophysiological explanation. Just like Mendel's ideas have been proven by the existence of DNA, and Darwin's ideas by a since buildup of fossil records, Hebbian learning has been explained by STDP, but that doesn't make Darwin nor Mendel's teachings obsolete.

References
- Dan & Poo, Neuron (2004); 44(1): 23-30
- Markram et al., Front Synaptic Neurosci (2012); 4: 2


Unsupervised Learning of Visual Features through Spike Timing Dependent Plasticity

Affiliations Centre de Recherche Cerveau et Cognition, Centre National de la Recherche Scientifique, Université Paul Sabatier, Faculté de Médecine de Rangueil, Toulouse, France , SpikeNet Technology SARL, Labege, France

Affiliations Centre de Recherche Cerveau et Cognition, Centre National de la Recherche Scientifique, Université Paul Sabatier, Faculté de Médecine de Rangueil, Toulouse, France , SpikeNet Technology SARL, Labege, France


Introduction

Modelling the brain at a neural level can lead to a better understanding of neural and psychological function, but can also lead to better AI systems. In particular, the authors are interested in developing agents and memory systems with good neuro-psychological fidelity. One task that these systems must perform is to learn new categories, and to be able to categorise novel input into one of several categories. Categorisation is also a standard machine learning task.

The scientific community has a growing understanding of brain function, and increasingly accurate models of that function. Nonetheless, current understanding and models of brain function are far from complete. In particular, it would be useful to develop a model of a novel categorisation task with good neuro-biological fidelity.

This paper develops a neural model of an abstract categorisation task that is commonly used as a machine learning problem. It takes into account some constraints from neural behaviour, but is not biologically or psychologically accurate. It is instead a step toward a neuro-psychologically faithful model that is also a reasonable machine learning algorithm.

Biological neuron models that perform psychological tasks have dynamics on two timescales that interact with each other. The first dynamic is the activation dynamic, where activation, typically involving neural firing, spreads from neuron to neuron. The second dynamic is a learning dynamic, where connections between neurons are changed. The activation dynamic is reviewed in “Neural models“ section, while the fatiguing leaky integrate and fire (FLIF) neural model used in this paper is described in 𠇏LIF model” section. The learning dynamic, which is based on Hebbian learning rules, is reviewed in “Hebbian learning” section. The post and pre-compensatory Hebbian learning rules used in this paper are described in 𠇌ompensatory learning” section.

“Iris categorisation” section describes simulations to categorise irises. Two modes are described in this section, one with two subnets and one with three subnets. Spontaneous firing from the neural model enables a second subnet to be used for categorising. A third subnet is added so that the categorisation is done based on a direct measurement of neural firing, and this makes the categorisation more plausible as a neuro-biological model of the task. This system is self-organising and is compared to a Kohonen self-organising map.

�tegorising with four subnets, and stability” section describes a four subnet system and some problems of learning stability. Solutions to this homeostatic problem are explored, eventually leading to systems that, as far as simulations have explored, continue to improve the longer they learn. Testing the homeostatic system on another categorisation task, yeast, gives poor results. 𠇍iscussion and conclusion” section concludes considering how this model might be included in a full neuro-psychological memory system.


Frank Rosenblatt in 1950, inferred that threshold neuron cannot be used for modeling cognition as it cannot learn or adopt from the environment or develop capabilities for classification, recognition or similar capabilities.

A perceptron draws inspiration from a biological visual neural model with three layers illustrated as follows :

  • Input Layer is synonymous to sensory cells in the retina, with random connections to neurons of the succeeding layer.
  • Association layers have threshold neurons with bi-directional connections to the response layer.
  • Response layer has threshold neurons that are interconnected with each other for competitive inhibitory signaling.

Response layer neurons compete with each other by sending inhibitory signals to produce output. Threshold functions are set at the origin for the association and response layers. This forms the basis of learning between these layers. The goal of the perception is to activate correct response neurons for each input pattern.


2 Background

Földiák foldiak1989adaptive developed a feedforward network with anti-Hebbian interconnections for visual feature extraction. The Hebbian rule in his model, shown in Eq. 1 , is inspired from Oja’s learning rule oja1982simplified that extracts the largest principal component from an input sequence,

where, w j i is the weight associated with the synapse connecting input (presynaptic) neuron i and representation (postsynaptic) unit j . x i and y j are input and linear output respectively. Over repeated trials, the term y j x i increases the weight when the input and the output are correlated. The second term ( − w j i y 2 j ) maintains the learning stability foldiak1989adaptive . With respect to binary (or spiking) units, a more appropriate assumption was made by Földiák foldiak1990forming . He modified the previous feedforward network by incorporating non-linear threshold units in the representation layer. The units are binary neurons with a threshold of 0.5 in which y j ∈ < 0 , 1 >(Note: y 2 j = y j ). Thus, the Hebbian rule in Eq. 1 is simplified to

The weight change rules defined in Eqs. 1 and 3 are based on the input and output correlation. Another interpretation for Eq. 3 can be explained in terms of vector quantization (or clustering in a WTA circuit) hammer2002generalized schneider2009distance in which the weights connected to each output neuron represent particular clusters (centroids). The weight change is also affected by the output neuron activation, y j . In this paper, we utilize the vector quantization concept to define an objective function. The objective function can be adapted to develop a spiking visual representation model equipped with a temporally local learning rule while still maintaining sparsity and independence. Our motivation is to use event-based, STDP-type learning rules. This requires the learning to be temporally local, specifically using spike times between pre- and postsynaptic neurons.


3 Simulation results

3.1 Method

We simulate random variations in a presynaptic neural firing rate ρ ( s j ) as well as random variations in the postsynaptic firing rates ρ ( s i ) induced by an externally driven voltage. By exploring many configurations of variations and levels at pre- and postsynaptic sides, we hope to cover the possible natural variations. We generate and record pre- and postsynaptic spikes sampled according to a binomial at each discrete t with probability proportional to ρ ( s i ) and ρ ( s j ) respectively, and record ˙ s i as well, in order to implement either a classical nearest neighbor STDP update rule or Eq. 1 , 1 1 1 Python scripts for those simulations are available at http://www.iro.umontreal.ca/

bengioy/src/STDP _ simulations . The nearest neighbor STDP rule is as follows. For every presynaptic spike, we consider a window of 20 time steps before and after. If there is one or more postsynaptic spike in both left and right windows, or no postsynaptic spike at all, the weight is not updated. Otherwise, we measure the time difference between the closest postsynaptic spike (nearest neighbor) and the presynaptic spike and compute the weight change using the current variable values. If both spikes coincide, we make no weight change. To compute the appropriate averages, 500 random sequences of rates are generated, each of length 160 time steps, and 1000 randomly sampled spike trains are generated according to these rates.

For measuring the effect of weight changes, we measure the average squared rate of change E [ | | ˙ s 2 i | | ] in two conditions: with weight changes (according to Eq. 1 ), and without.

3.2 Results

Examples of the spike sequences and underlying pre- and postsynaptic states s i and s j are illustrated in Fig. 3 .

Figure 3: Example of rate and spike sequence generated in the simulations, along with weight changes according to the spike-dependent variant of our update rule, Eq. 2 . Top: presynaptic spikes ξ j (which is when a weight change can occur). Middle: integrated postsynaptic activity s j . Bottom: value of the updated weight W i , j .

Fig. 1 (middle and right) shows the results of these simulations, comparing the weight change obtained at various spike timing differences for Eq. 1 and for the nearest neighbor STDP rule, both matching well the biological data (Fig. 1 , left). Fig. 2 shows that both update rules are strongly correlated, in the sense that for a given amount of weight change induced by one, we observe in average a linearly proportional weight change by the other.

3.3 Link to Stochastic Gradient Descent and Back-Propagation

Let us consider the common simplification in which postsynaptic neural activity s i is obtained as a sum with the usual terms proportional to the product of synaptic weight ( W i , j ) and presynaptic firing rate ( ρ ( s j ( t − 1 ) ), i.e., in discrete time,

from some quantities α and β . In that case,

Stochastic gradient descent on W with respect to an objective function J would follow

our STDP rule (Eq. 1 ) would produce stochastic gradient on J .

The assumption in Eq. 4 and the above consequence was first suggested by Hinton (2007) : neurons would change their average firing rate so as make the network as a whole produce configurations corresponding to better values of our objective function. This would make STDP do gradient descent on the prediction errors made by the network.

But how could neural dynamics have that property? That question is not completely answered, but as shown in Bengio and Fischer (2015) , a network with symmetric feedback weights would have the property that a small perturbation of “output units” towards better predictions would propagate to internal layers such that hidden units s i would move to approximately follow the gradient of the prediction error J with respect to s i , making our STDP rule correspond approximately to gradient descent on the prediction error. The experiments reported by Scellier and Bengio (2016) have actuallly shown that these approximations work and enable a supervised multi-layer neural network to be trained.


[D] State of Hebbian Learning Research

Current deep learning is based off of backprop, aka a global tweaking of an algorithm via propagation of an error signal. However I've heard that biological networks make updates via a local learning rule, which I interpret as an algo that is only provided the states of a neuron's immediate stimuli to decide how to tweak that neuron's weights. A local learning rule would also make sense considering brain circuitry consists of a huge proportion of feedback connections, and (classic) backprop only works on DAGs. Couple questions:

- How are 'weights' represented in neurons and by what mechanism are they tweaked?

- Is this local learning rule narrative even correct? Any clear evidence?

- What is the state of research regarding hebbian/local learning rules, why haven't they gotten traction? I was also specifically interested in research concerned w/ finding algorithms to discover an optimal local rule for a task (a hebbian meta-learner if that makes sense).

Iɽ love pointers to any resources/research, especially since I don't know where to start trying to understand these systems. I've studied basic ML theory and am caught up w/ deep learning, but want to better understand the foundational ideas of learning that people have come up with in the past.

* I use 'hebbian' and 'local' interchangeably, correct me if there is a distinction between the two *

Excellent questions, op. I will try to fully answer what I can tomorrow so I’ll just leave this short reply as a reminder. My PhD is in neuroscience and I study learning and memory, specifically synaptic plasticity in the hippocampus via electron microscopy, it’s nice to see some questions here I am actually qualified to answer.

many people view synapses as ‘weights’, we know larger ones are generally stronger, they can physically enlarge or diminish in area in response to different stimuli, and can very rapidly change functional states without measurable change in size.

adult neurons are mostly sessile, they can extend some processes and dendritic spines can be quite dynamic, but have very little access to information not delivered directly to their synapses by their presynaptic partners. A given neuron can’t really know what a neuron 3 or 4 synapses away is doing except via the intermediary neurons which may or may be transforming that information to an unknown degree. That’s not to say neurons have zero access to nonsynaptic information, the endocrine system does provide some signals globally, or sort of globally.

Evidence for local learning is enormous, the literature is hard to keep up with, I will provide examples.

3) this is a bit beyond my experience as to hebbian learning in machines, but likely is due to the current limitations of hardware. Biological neurons supply their own power, don’t follow a clock, exploit biophysical properties of their environment and their own structure in ways nodes in a graph cannot do yet, likely encode large amounts of information in their complex shapes, and have access to genetic information that is often unique enough to a specific neuron subtype that we use that to identify them.

Weights are a very clear and concrete concept in the context of networks of artificial neurons or nodes. The weight at a link between two nodes is simply a number that scales the input (also a number) in some arbitrary way, ie, positive, negative, or identity, and as far as I understand the weights are the only parameters of a node that change during learning. If the idea is to identify processes that could stand in for weights in neurons, then since the weight changes the response of the node, a weight for a neuron can be anything that can change its response to some stimuli.

The links between nodes are very roughly analogous to the synapses between neurons, but if one looks too hard the similarities are extremely shallow. We can start by only considering individual synapses themselves while ignoring neighboring synapses and other cellular processes for now.

First, to keep this under 50 pages we will also ignore neuromodulators and consider only the two main neurotransmitters, glutamate and GABA. A given synapse can grow or shrink, which is typically associated with their ‘strength’, though how one chooses what to measure to be able to say this will depend largely on what the experimenter is interested in. One can measure synaptic strength in several ways: current across the membrane, change in voltage potential at the soma or some distance from the synapse, or the spiking output of the measured cell. Unlike link weights, synapses are exclusively excitatory or inhibitory where a weight can be positive or negative.

Both excitatory and inhibitory synapses can get stronger or weaker depending on activity through numerous mechanisms operating at different time scales simultaneously. Short term potentiation and depression typically involve transient changes to the conductance or binding affinity of a receptor or ion channel, the voltage dependence of a channel or receptor, or the concentration of something and can be expressed either presynaptically, postsynaptically, or both and these occur at a few to a few hundred milliseconds. Changes in synaptic strength that involve physical growth or shrinkage of the synapse occur over timescales of


Results

Adaptation with the EH learning rule

We model the learning effects observed by Jarosiewicz et al. (2008) through adaptation at a single synaptic stage, from a set of hypothesized input neurons to our motor cortical neurons. Adaptation of these synaptic efficacies wij will be necessary if the actual decoding PDs pi do not produce efficient cursor trajectories. To make this more clear, assume that suboptimal dPDs p1, … , pn are used for decoding. Then for some input x(t), the movement of the cursor is not in the desired direction y*(t). The weights wij should therefore be adapted such that at every time step t, the direction of movement y(t) is close to the desired direction y*(t). We can quantify the angular match Rang(t) at time t by the cosine of the angle between movement direction y(t) and desired direction y*(t): This measure has a value of 1 if the cursor moves exactly in the desired direction, it is 0 if the cursor moves perpendicular to the desired direction, and it is −1 if the cursor movement is in the opposite direction. The angular match Rang(t) will be used as the reward signal for adaptation below. For desired directions y*(1), … , y*(T) and corresponding inputs x(1), … , x(T), the goal of learning is hence to find weights wij such that is maximized.

The plasticity model used in this article is based on the assumption that learning in motor cortex neurons has to rely on a single global scalar neuromodulatory signal that carries information about system performance. One way for a neuromodulatory signal to influence synaptic weight changes is by gating local plasticity. In the study by Loewenstein and Seung (2006), this idea was implemented by learning rules where the weight changes were proportional to the covariance between the reward signal R and some measure of neuronal activity N at the synapse, where N could correspond to the presynaptic activity, the postsynaptic activity, or the product of both. The authors showed that such learning rules can explain a phenomenon called Herrnstein's matching law. Interestingly, for the analysis of Loewenstein and Seung (2006), the specific implementation of this correlation-based adaptation mechanism is not important. From this general class, we investigate in this article the following learning rule: where (t) denotes the low-pass filtered version of some variable z with an exponential kernel we used (t) = 0.8(t − 1) + 0.2z(t). We call this rule the exploratory Hebb rule (EH rule). The important feature of this learning rule is that apart from variables that are locally available for each neuron (xj(t), ai(t), āi(t)), only a single scalar signal, the reward signal R(t), is needed to evaluate performance (we also explored a rule where the activation ai is replaced by the output si and obtained very similar results). This reward signal is provided by some neural circuit that evaluates performance of the system. In our simulations, we simply use the angular match Rang(t), corresponding to the deviation of the instantaneous trajectory from its ideal path to the target, as this reward signal. The rule measures correlations between deviations of the reward signal R(t) from its mean and deviations of the activation ai(t) from the mean activation and adjusts weights such that rewards above mean are reinforced. The EH rule approximates gradient ascent on the reward signal by exploring alternatives to the actual behavior with the help of some exploratory signal ξ(t). The exploratory signal could, for example, be interpreted as spontaneous activity, internal noise, or input from some other brain area. The deviation of the activation from the recent mean ai(t) − āi(t) is an estimate of the exploratory term ξi(t) at time t if the mean āi(t) is based on neuron activations σj wijxj(t′), which are similar to the activation σj wijxj(t) at time t. Here we make use of (1) the fact that weights are changing very slowly and (2) the continuity of the task (inputs x at successive time points are similar). If conditions 1 and 2 hold, the EH rule can be seen as an approximation of the following: This rule is a typical node-perturbation learning rule (Mazzoni et al., 1991 Williams, 1992 Baxter and Bartlett, 2001 Fiete and Seung, 2006) (see also the Discussion) that can be shown to approximate gradient ascent (see, e.g., Fiete and Seung, 2006). A simple derivation that shows the link between the EH rule and gradient ascent is given in the Appendix.

The EH learning rule is different from other node-perturbation rules in one important aspect. In standard node-perturbation learning rules, the noise needs to be accessible to the learning mechanism separately from the output signal. For example, in the studies by Mazzoni et al. (1991) and Williams (1992), binary neurons were used and the noise appears in the learning rule in the form of the probability of the neuron to output 1. In the study by Fiete and Seung (2006), the noise term is directly incorporated in the learning rule. The EH rule instead does not directly need the noise signal, but a temporally filtered version of the activation of the neuron, which is an estimate of the noise signal. Obviously, this estimate is only sufficiently accurate if the structure of the task is appropriate, i.e., if the input to the neuron is temporally stable on small timescales. We note that the filtering of postsynaptic activity makes the Hebbian part of the EH rule reminiscent of a linearized BCM rule (Bienenstock et al., 1982). The postsynaptic activity is compared with a threshold to decide whether the synapse is potentiated or depressed.

Comparison with experimentally observed learning effects

We simulated the two types of perturbation experiments reported by Jarosiewicz et al. (2008) in our model network with 40 recorded neurons. In the first set of simulations, we chose 25% of the recorded neurons to be rotated neurons, and in the second set of simulations, we chose 50% of the recorded neurons to be rotated. In each simulation, 320 targets were presented to the model, which is similar to the number of target presentations in the study by Jarosiewicz et al. (2008). The performance improvement and PD shifts for one example run are shown in Figure 3. To simulate the experiments as closely as possible, we fit the noise and the learning rate in our model to the experimental data (see Materials and Methods). All neurons showed a tendency to compensate the perturbation by a shift of their PDs in the direction of the perturbation rotation. This tendency is stronger for rotated neurons. The training-induced shifts in PDs of the recorded neurons were compiled from 20 independent simulated experiments, and analyzed separately for rotated and nonrotated neurons. The results are in good agreement with the experimental data (Fig. 4). In the simulated 25% perturbation experiment, the mean shift of the PD for rotated neurons was 8.2 ± 4.8°, whereas for nonrotated neurons, it was 5.5 ± 1.6°. This is a relatively small effect, similar to the effect observed by Jarosiewicz et al. (2008), where the PD shifts were 9.86° for rotated units and 5.25° for nonrotated units. A stronger effect can be found in the 50% perturbation experiment (see below). We also compared the deviation of the trajectory from the ideal straight line in rotation direction halfway to the target (see Materials and Methods) from early trials to the deviation of late trials. In early trials, the trajectory deviation was 9.2 ± 8.8 mm, which was reduced by learning to 2.4 ± 4.9 mm. In the simulated 50% perturbation experiment, the mean shift of the PD for rotated neurons was 18.1 ± 4.2°, whereas for nonrotated neurons, it was 12.1 ± 2.6°. Again, the PD shifts are very similar to those in the monkey experiments: 21.7° for rotated units and 16.11° for nonrotated units. The trajectory deviation was 23.1 ± 7.5 mm in early trials, and 4.8 ± 5.1 mm in late trials. Here, the early deviation was stronger than in the monkey experiment, while the late deviation was smaller.

One example simulation of the 50% perturbation experiment with the EH rule and data-derived network parameters. A, Angular match Rang as a function of learning time. Every 100th time point is plotted. B, PD shifts projected onto the rotation plane (the rotation axis points toward the reader) for rotated (red) and nonrotated (black) neurons from their initial values (light color) to their values after training (intense color, these PDs are connected by the shortest path on the unit sphere axes in arbitrary units). The PDs of rotated neurons are consistently rotated counter-clockwise to compensate for the perturbation. C, Tuning of an example rotated neuron to target directions of angle Φ in the rotation plane (y–z-plane) before (gray) and after (black) training. The target direction for a given Φ was defined as y*(Φ) = (1/ )(1, cos(Φ), sin(Φ)) T . Circles on the x-axis indicate projected preferred directions of the neuron before (gray) and after (black) training.

PD shifts in simulated perturbation sessions are in good agreement with experimental data [compare to Jarosiewicz et al. (2008), their Fig. 3A,B]. Shift in the PDs measured after simulated perturbation sessions relative to initial PDs for all units in 20 simulated experiments where 25% (A) or 50% (B) of the units were rotated. Dots represent individual data points and black circled dots represent the means of the rotated (red) and nonrotated (blue) units.

The EH rule falls into the general class of learning rules where the weight change is proportional to the covariance of the reward signal and some measure of neuronal activity (Loewenstein and Seung, 2006). Interestingly, the specific implementation of this idea influences the learning effects observed in our model. We performed the same experiment with slightly different correlation-based rules: and where the filtered postsynaptic activation or the filtered reward was not taken into account. Compare these to the EH rule (Eq. 16). These rules also converge with performance similar to the EH rule. However, no credit assignment effect can be observed with these rules. In the simulated 50% perturbation experiment, the mean shift of the PD of rotated neurons (nonrotated neurons) was 25.5 ± 4.0° (26.8 ± 2.8°) for the rule given by Equation 18 and 12.8 ± 3.6° (12.0 ± 2.4°) for the rule given by Equation 19 (Fig. 5). Only when deviations of the reward from its local mean and deviations of the activation from its local mean are both taken into account do we observe differential changes in the two populations of cells.

PD shifts in simulated 50% perturbation sessions with the learning rules in Equations 18 (A) and 19 (B). Dots represent individual data points and black circled dots represent the means of the rotated (red) and nonrotated (blue) units. No credit assignment effect can be observed for these rules.

In the monkey experiment, training in the perturbation session also resulted in a decrease of the modulation depth of rotated neurons, which led to a relative decrease of the contribution of these neurons to the cursor movement. A qualitatively similar result could be observed in our simulations. In the 25% perturbation simulation, modulation depths of rotated neurons changed on average by −2.7 ± 4.3 Hz, whereas modulation depths of nonrotated neurons changed on average by 2.2 ± 3.9 Hz (average over 20 independent simulations a negative change indicates a decreased modulation depth in the perturbation session relative to the control session). In the 50% perturbation simulation, the changes in modulation depths were on average −3.6 ± 5.5 Hz for rotated neurons and 5.4 ± 6.0 Hz for nonrotated neurons (when comparing these results to experimental results, one has to take into account that modulation depths in monkey experiments were around 10 Hz, whereas in the simulations, they were ∼25 Hz). Thus, the relative contribution of rotated neurons on cursor movement decreased during the perturbation session.

It was reported by Jarosiewicz et al. (2008) that after the perturbation session, PDs returned to their original values in a subsequent washout session where the original PDs were used as decoding PDs. We simulated such washout sessions after our simulated perturbation sessions in the model and found a similar effect (Fig. 6A,B). However, the retuning in our simulation is slower than observed in the monkey experiments. In the experiments, it took about 160 target presentations until mean PD shifts relative to PDs in the control session were around zero. This fast unlearning is consistent with the observation that adaptation and deadaptation in motor cortex can occur at substantially different rates, likely reflecting two separate processes (Davidson and Wolpert, 2004). We did not model such separate processes thus, the timescales for adaptation and deadaptation are the same in the simulations. In a simulated washout session with a larger learning rate, we found faster convergence of PDs to original values (Fig. 6C,D).

PDs shifts in simulated washout sessions. A–D, Shift in the PDs (mean over 20 trials) for rotated neurons (gray) and nonrotated neurons (black) relative to PDs of the control session as a function of the number of targets presented for 25% perturbation (A, C) and 50% perturbation (B, D). A, B, Simulations with the same learning rate as in the simulated perturbation session. C, D, Simulations with a five times larger learning rate.

The performance of the system before and after learning is shown in Figure 7. The neurons in the network after training are subject to the same amount of noise as the neurons in the network before training, but the angular match after training shows much less fluctuation than before training. We therefore conjectured that the network automatically suppresses jitter in the trajectory in the presence of high exploration levels υ. We quantified this conjecture by computing the mean angle between the cursor velocity vector with and without noise for 50 randomly drawn noise samples. In the mean over the 20 simulations and 50 randomly drawn target directions, this angle was 10 ± 2.7° (mean ± SD) before learning and 9.6 ± 2.5° after learning. Although only a slight reduction, it was highly significant when the mean angles before and after learning were compared for identical target directions and noise realizations (p < 0.0002, paired t test). This is not an effect of increased network weights, because weights increased only slightly and the same test where weights were normalized to their initial L2 norm after training produced the same significance value.

Comparison of network performance before and after learning for 50% perturbation. Angular match Rang(t) of the cursor movements in one reaching trial before (gray) and after (black) learning as a function of the time since the target was first made visible. The black curve ends prematurely because the target is reached faster. Note the reduced temporal jitter of the performance after learning, indicating reduced sensitivity to the noise signal.

Psychophysical studies in humans (Imamizu et al., 1995) and monkeys (Paz and Vaadia, 2004) showed that the learning of a new sensorimotor mapping generalizes poorly to untrained directions with better generalization for movements in directions close to the trained one. It was argued by Imamizu et al. (1995) that this is evidence for a neural network-like model of sensorimotor mappings. The model studied in this article exhibits similar generalization behavior. When training is constrained to a single target location, performance is optimized in this direction, while the performance clearly decreased as target direction increased from the trained angle (Fig. 8).

Generalization of network performance for a 50% perturbation experiment with cursor movements to a single target location during training. Twenty independent simulations with randomly drawn target positions (from the corners of the unit cube) and rotation axes (either the x-, y-, or z-axis) were performed. In each simulation the network model was first tested on 100 random target directions, then trained for 320 trials, and then tested again on 100 random target directions. Angular matches of the test trials before (gray) and after (black) training are plotted against the angle between the target direction vector in the test and the vector from the origin to the training target location. Shown is the mean and SD of the angular match Rang over movements with an angle to the training direction in [0, 20]°, (20, 40]°, (40, 60]°, etc. For clarity, the SD for movements before learning is not shown. It was quite constant over all angles being 0.15 in the mean.

Tuning changes depend on the exploration level

When we compare the results obtained by our simulations to those of monkey experiments [compare Fig. 4 to Jarosiewicz et al. (2008), their Fig. 3], it is interesting that quantitatively similar effects were obtained with noise levels that were measured in the experiments. We therefore explored whether the fitting of parameters to values extracted from experimental data was important by exploring the effect of different exploration levels and learning rates on performance and PD shifts.

The amount of noise was controlled by modifying the exploration level ν (see Eq. 13). For some extreme parameter settings, the EH rule can lead to large weights. We therefore implemented a homeostatic mechanism by normalizing the weight vector of each neuron after each update, i.e., the weight after the tth update step is given by the following: Employing the EH learning rule, the network converged to weight settings with good performance for most parameter settings, except for large learning rates and very large noise levels. Note that good performance is achieved even for large exploration levels of ν ≈ 60 Hz (Fig. 9A). The good performance of the system shows that already a very small network can use large amounts of noise for learning, while this noise does not interfere with performance.

Behavior of the EH rule in simulated perturbation sessions (50% perturbed neurons) for different parameter settings. All plotted values are means over 10 independent simulations. Logarithms are to the basis of 2. The black circle indicates the parameter setting used in Results. A, Dependence of network performance measured as the mean number of targets reached per time step on learning rate η and exploration level υ. Performance deteriorates for high learning rate and exploration levels. B, Mean PD shifts in rotation direction for rotated neurons. C, Mean PD shifts in rotation direction for nonrotated neurons. In comparison to rotated neurons, PD shifts of nonrotated neurons are small, especially for larger exploration levels.

We investigated the influence of learning on the PDs of circuit neurons. The amount of exploration and the learning rate η both turned out be important parameters. The tuning changes reported in neurons of monkeys subsumed under the term “credit assignment effect” were qualitatively met by our model networks for most parameter settings (Fig. 9), except for very large learning rates (when learning does not work) and very small learning rates (compare panels B and C). Quantitatively, the amount of PD shift especially for rotated neurons strongly depends on the exploration level, with shifts close to 50° for large exploration levels.

To summarize, for small levels of exploration, PDs change only slightly and the difference in PD change between rotated and nonrotated neurons is small, while for large noise levels, PD change differences can be quite drastic. Also the learning rate η influences the amount of PD shifts. This shows that the learning rule guarantees good performance and a qualitative match to experimentally observed PD shifts for a wide range of parameters. However, for the quantitative fit found in our simulations, the parameters extracted from experimental data turned out to be important.


The authors’ work is supported by a Royal Society Dorothy Hodgkin Fellowship (Alanna J. Watt), and by the Neurosciences Research Foundation and the G. Harold and Leila Y. Mathers Charitable Foundation (Niraj S. Desai).

Abbott, L. F., and Nelson, S. B. (2000). Synaptic plasticity: taming the beast. Nat. Neurosci. 3(Suppl.), 1178�.

Abraham, W. C. (2008). Metaplasticity: tuning synapses and networks for plasticity. Nat. Rev. Neurosci. 9, 387.

Abraham, W. C., Mason-Parker, S. E., Bear, M. F., Webb, S., and Tate, W. P. (2001). Heterosynaptic metaplasticity in the hippocampus in vivo: a BCM-like modifiable threshold for LTP. Proc. Natl. Acad. Sci. U.S.A. 98, 10924�.

Aizenman, C. D., Akerman, C. J., Jensen, K. R., and Cline, H. T. (2003). Visually driven regulation of intrinsic neuronal excitability improves stimulus detection in vivo. Neuron 39, 831�.

Amici, M., Doherty, A., Jo, J., Jane, D., Cho, K., Collingridge, G., and Dargan, S. (2009). Neuronal calcium sensors and synaptic plasticity. Biochem. Soc. Trans. 37, 1359�.

Aoto, J., Nam, C. I., Poon, M. M., Ting, P., and Chen, L. (2008). Synaptic signaling by all-trans retinoic acid in homeostatic synaptic plasticity. Neuron 60, 308�.

Appleby, P. A., and Elliott, T. (2005). Synaptic and temporal ensemble interpretation of spike-timing-dependent plasticity. Neural Comput. 17, 2316�.

Aptowicz, C. O., Kunkler, P. E., and Kraig, R. P. (2004). Homeostatic plasticity in hippocampal slice cultures involves changes in voltage-gated Na+ channel expression. Brain Res. 998, 155�.

Artola, A., and Singer, W. (1993). Long-term depression of excitatory synaptic transmission and its relationship to long-term potentiation. Trends Neurosci. 16, 480�.

Atkinson, S. E., and Williams, S. R. (2009). Postnatal development of dendritic synaptic integration in rat neocortical pyramidal neurons. J. Neurophysiol. 102, 735�.

Azdad, K., Chavez, M., Don Bischop, P., Wetzelaer, P., Marescau, B., De Deyn, P. P., Gall, D., and Schiffmann, S. N. (2009). Homeostatic plasticity of striatal neurons intrinsic excitability following dopamine depletion. PLoS ONE 4, e6908. doi: 10.1371/journal.pone.0006908.

Bartley, A. F., Huang, Z. J., Huber, K. M., and Gibson, J. R. (2008). Differential activity-dependent, homeostatic plasticity of two neocortical inhibitory circuits. J. Neurophysiol. 100, 1983�.

Bear, M. F. (2003). Bidirectional synaptic plasticity: from theory to reality. Philos. Trans. R. Soc. Lond., B, Biol. Sci. 358, 649�.

Bender, V. A., Bender, K. J., Brasier, D. J., and Feldman, D. E. (2006). Two coincidence detectors for spike timing-dependent plasticity in somatosensory cortex. J. Neurosci. 26, 4166�.

Benuskova, L., and Abraham, W. C. (2007). STDP rule endowed with the BCM sliding threshold accounts for hippocampal heterosynaptic plasticity. J. Comput. Neurosci. 22, 129�.

Bi, G. Q., and Poo, M. M. (1998). Synaptic modifications in cultured hippocampal neurons: dependence on spike timing, synaptic strength, and postsynaptic cell type. J. Neurosci. 18, 10464�.

Biel, M., Wahl-Schott, C., Michalakis, S., and Zong, X. (2009). Hyperpolarization-activated cation channels: from genes to function. Physiol. Rev. 89, 847�.

Bienenstock, E. L., Cooper, L. N., and Munro, P. W. (1982). Theory for the development of neuron selectivity: orientation specificity and binocular interaction in visual cortex. J. Neurosci. 2, 32�.

Billings, G., and van Rossum, M. C. (2009). Memory retention and spike-timing-dependent plasticity. J. Neurophysiol. 101, 2775�.

Brager, D. H., and Johnston, D. (2007). Plasticity of intrinsic excitability during long-term depression is mediated through mGluR-dependent changes in I(h) in hippocampal CA1 pyramidal neurons. J. Neurosci. 27, 13926�.

Branco, T., Staras, K., Darcy, K. J., and Goda, Y. (2008). Local dendritic activity sets release probability at hippocampal synapses. Neuron 59, 475�.

Breton, J. D., and Stuart, G. J. (2009). Loss of sensory input increases the intrinsic excitability of layer 5 pyramidal neurons in rat barrel cortex. J. Physiol. 587, 5107�.

Burkitt, A. N., Meffin, H., and Grayden, D. B. (2004). Spike-timing-dependent plasticity: the relationship to rate-based learning for models with weight dynamics determined by a stable fixed point. Neural Comput. 16, 885�.

Burrone, J., and Murthy, V. N. (2003). Synaptic gain control and homeostasis. Curr. Opin. Neurobiol. 13, 560�.

Burrone, J., O𠆛yrne, M., and Murthy, V. N. (2002). Multiple forms of synaptic plasticity triggered by selective suppression of activity in individual neurons. Nature 420, 414�.

Campanac, E., Daoudal, G., Ankri, N., and Debanne, D. (2008). Downregulation of dendritic I(h) in CA1 pyramidal neurons after LTP. J. Neurosci. 28, 8635�.

Campanac, E., and Debanne, D. (2008). Spike timing-dependent plasticity: a learning rule for dendritic integration in rat CA1 pyramidal neurons. J. Physiol. 586, 779�.

Caporale, N., and Dan, Y. (2008). Spike timing-dependent plasticity: a Hebbian learning rule. Annu. Rev. Neurosci. 31, 25�.

Carpenter, G. A., and Milenova, B. L. (2002). Redistribution of synaptic efficacy supports stable pattern learning in neural networks. Neural Comput. 14, 873�.

Castellani, G. C., Quinlan, E. M., Cooper, L. N., and Shouval, H. Z. (2001). A biophysical model of bidirectional synaptic plasticity: dependence on AMPA and NMDA receptors. Proc. Natl. Acad. Sci. U.S.A. 98, 12772�.

Chistiakova, M., and Volgushev, M. (2009). Heterosynaptic plasticity in the neocortex. Exp. Brain Res. 199, 377�.

Cho, K. K., Khibnik, L., Philpot, B. D., and Bear, M. F. (2009). The ratio of NR2A/B NMDA receptor subunits determines the qualities of ocular dominance plasticity in visual cortex. Proc. Natl. Acad. Sci. U.S.A. 106, 5377�.

Cingolani, L. A., Thalhammer, A., Yu, L. M., Catalano, M., Ramos, T., Colicos, M. A., and Goda, Y. (2008). Activity-dependent regulation of synaptic AMPA receptor composition and abundance by beta3 integrins. Neuron 58, 749�.

Clopath, C., Busing, L., Vasilaki, E., and Gerstner, W. (2010). Connectivity reflects coding: a model of voltage-based STDP with homeostasis. Nat. Neurosci. 13, 344�.

Cooper, L. N., Intrator, N., Blais, B. S., and Shouval, H. Z. (2004). Theory of Cortical Plasticity. River Edge, NJ: World Scientific.

Corlew, R., Brasier, D. J., Feldman, D. E., and Philpot, B. D. (2008). Presynaptic NMDA receptors: newly appreciated roles in cortical synaptic function and plasticity. Neuroscientist 14, 609�.

Corlew, R., Wang, Y., Ghermazien, H., Erisir, A., and Philpot, B. D. (2007). Developmental switch in the contribution of presynaptic and postsynaptic NMDA receptors to long-term depression. J. Neurosci. 27, 9835�.

Debanne, D., Gahwiler, B. H., and Thompson, S. M. (1998). Long-term synaptic plasticity between pairs of individual CA3 pyramidal cells in rat hippocampal slice cultures. J. Physiol. 507, 237�.

De Gois, S., Schafer, M. K., Defamie, N., Chen, C., Ricci, A., Weihe, E., Varoqui, H., and Erickson, J. D. (2005). Homeostatic scaling of vesicular glutamate and GABA transporter expression in rat neocortical circuits. J. Neurosci. 25, 7121�.

Delgado, J. Y., Gomez, J. F., and Desai, N. S. (2009). In vivo-like conductance levels control the induction of spike-timing dependent plasticity. Soc. Neurosci. Abstr. 35, 41.8.

Desai, N. S., Cudmore, R. H., Nelson, S. B., and Turrigiano, G. G. (2002). Critical periods for experience-dependent synaptic scaling in visual cortex. Nat. Neurosci. 5, 783�.

Desai, N. S., Rutherford, L. C., and Turrigiano, G. G. (1999a). BDNF regulates the intrinsic excitability of cortical neurons. Learn. Mem. 6, 284�.

Desai, N. S., Rutherford, L. C., and Turrigiano, G. G. (1999b). Plasticity in the intrinsic excitability of cortical pyramidal neurons. Nat. Neurosci. 2, 515�.

De Simoni, A., Griesinger, C. B., and Edwards, F. A. (2003). Development of rat CA1 neurones in acute versus organotypic slices: role of experience in synaptic morphology and activity. J. Physiol. 550, 135�.

Destexhe, A., Rudolph, M., and Pare, D. (2003). The high-conductance state of neocortical neurons in vivo. Nat. Rev. Neurosci. 4, 739�.

Dudek, S. M., and Bear, M. F. (1992). Homosynaptic long-term depression in area CA1 of hippocampus and effects of N-methyl-D-aspartate receptor blockade. Proc. Natl. Acad. Sci. U.S.A. 89, 4363�.

Duguid, I., and Sjostrom, P. J. (2006). Novel presynaptic mechanisms for coincidence detection in synaptic plasticity. Curr. Opin. Neurobiol. 16, 312�.

Echegoyen, J., Neu, A., Graber, K. D., and Soltesz, I. (2007). Homeostatic plasticity studied using in vivo hippocampal activity-blockade: synaptic scaling, intrinsic plasticity and age-dependence. PLoS ONE 2, e700. doi: 10.1371/journal.pone.0000700.

Ehlers, M. D., Heine, M., Groc, L., Lee, M. C., and Choquet, D. (2007). Diffusional trapping of GluR1 AMPA receptors by input-specific synaptic activity. Neuron 54, 447�.

Engert, F., and Bonhoeffer, T. (1999). Dendritic spine changes associated with hippocampal long-term synaptic plasticity. Nature 399, 66�.

Erickson, J. D., De Gois, S., Varoqui, H., Schafer, M. K., and Weihe, E. (2006). Activity-dependent regulation of vesicular glutamate and GABA transporters: a means to scale quantal size. Neurochem. Int. 48, 643�.

Fan, Y., Fricker, D., Brager, D. H., Chen, X., Lu, H. C., Chitwood, R. A., and Johnston, D. (2005). Activity-dependent decrease of excitability in rat hippocampal neurons through increases in I(h). Nat. Neurosci. 8, 1542�.

Feldman, D. E. (2000). Inhibition and plasticity. Nat. Neurosci. 3, 303�.

Feldman, D. E. (2009). Synaptic mechanisms for plasticity in neocortex. Annu. Rev. Neurosci. 32, 33�.

Frick, A., and Johnston, D. (2005). Plasticity of dendritic excitability. J. Neurobiol. 64, 100�.

Froemke, R. C., Tsay, I. A., Raad, M., Long, J. D., and Dan, Y. (2006). Contribution of individual spikes in burst-induced long-term synaptic modification. J. Neurophysiol. 95, 1620�.

Fuenzalida, M., Fernandez de Sevilla, D., and Buno, W. (2007). Changes of the EPSP waveform regulate the temporal window for spike-timing-dependent plasticity. J. Neurosci. 27, 11940�.

Gibson, J. R., Bartley, A. F., and Huber, K. M. (2006). Role for the subthreshold currents ILeak and IH in the homeostatic control of excitability in neocortical somatostatin-positive inhibitory neurons. J. Neurophysiol. 96, 420�.

Goddard, C. A., Butts, D. A., and Shatz, C. J. (2007). Regulation of CNS synapses by neuronal MHC class I. Proc. Natl. Acad. Sci. U.S.A. 104, 6828�.

Goel, A., and Lee, H. K. (2007). Persistence of experience-induced homeostatic synaptic plasticity through adulthood in superficial layers of mouse visual cortex. J. Neurosci. 27, 6692�.

Gutig, R., Aharonov, R., Rotter, S., and Sompolinsky, H. (2003). Learning input correlations through nonlinear temporally asymmetric Hebbian plasticity. J. Neurosci. 23, 3697�.

Hardingham, N. R., Hardingham, G. E., Fox, K. D., and Jack, J. J. (2007). Presynaptic efficacy directs normalization of synaptic strength in layer 2/3 rat neocortex after paired activity. J. Neurophysiol. 97, 2965�.

Harms, K. J., and Craig, A. M. (2005). Synapse composition and organization following chronic activity blockade in cultured hippocampal neurons. J. Comp. Neurol. 490, 72�.

Haydon, P. G. (2001). GLIA: listening and talking to the synapse. Nat. Rev. Neurosci. 2, 185�.

Hebb, D. O. (1949). The Organization of Behavior: A Neuropsychological Theory. New York, NY: John Wiley and Sons.

Hou, Q., Zhang, D., Jarzylo, L., Huganir, R. L., and Man, H. Y. (2008). Homeostatic regulation of AMPA receptor expression at single hippocampal synapses. Proc. Natl. Acad. Sci. U.S.A. 105, 775�.

Ibata, K., Sun, Q., and Turrigiano, G. G. (2008). Rapid synaptic scaling induced by changes in postsynaptic firing. Neuron 57, 819�.

Izhikevich, E. M., and Desai, N. S. (2003). Relating STDP to BCM. Neural Comput. 15, 1511�.

Johnston, D., and Narayanan, R. (2008). Active dendrites: colorful wings of the mysterious butterflies. Trends Neurosci. 31, 309�.

Ju, W., Morishita, W., Tsui, J., Gaietta, G., Deerinck, T. J., Adams, S. R., Garner, C. C., Tsien, R. Y., Ellisman, M. H., and Malenka, R. C. (2004). Activity-dependent regulation of dendritic synthesis and trafficking of AMPA receptors. Nat. Neurosci. 7, 244�.

Kampa, B. M., Letzkus, J. J., and Stuart, G. J. (2007). Dendritic mechanisms controlling spike-timing-dependent synaptic plasticity. Trends Neurosci. 30, 456�.

Karmarkar, U. R., and Buonomano, D. V. (2002). A model of spike-timing dependent plasticity: one or two coincidence detectors? J. Neurophysiol. 88, 507�.

Karmarkar, U. R., and Buonomano, D. V. (2006). Different forms of homeostatic plasticity are engaged with distinct temporal profiles. Eur. J. Neurosci. 23, 1575�.

Kempter, R., Gerstner, W., and van Hemmen, J. L. (2001). Intrinsic stabilization of output rates by spike-based Hebbian learning. Neural Comput. 13, 2709�.

Kilman, V., van Rossum, M. C., and Turrigiano, G. G. (2002). Activity deprivation reduces miniature IPSC amplitude by decreasing the number of postsynaptic GABA(A) receptors clustered at neocortical synapses. J. Neurosci. 22, 1328�.

Kirkwood, A., Rioult, M. C., and Bear, M. F. (1996). Experience-dependent modification of synaptic plasticity in visual cortex. Nature 381, 526�.

Law, C. C., and Cooper, L. N. (1994). Formation of receptive fields in realistic visual environments according to the Bienenstock, Cooper, and Munro (BCM) theory. Proc. Natl. Acad. Sci. U.S.A. 91, 7797�.

Lazar, A., Pipa, G., and Triesch, J. (2009). SORN: a self-organizing recurrent neural network. Front. Comput. Neurosci. 3:23. doi: 10.3389/neuro.10.023.2009.

LeMasson, G., Marder, E., and Abbott, L. F. (1993). Activity-dependent regulation of conductances in model neurons. Science 259, 1915�.

Lisman, J. (2003). Long-term potentiation: outstanding questions and attempted synthesis. Philos. Trans. R. Soc. Lond., B, Biol. Sci. 358, 829�.

Lisman, J., and Spruston, N. (2005). Postsynaptic depolarization requirements for LTP and LTD: a critique of spike timing-dependent plasticity. Nat. Neurosci. 8, 839�.

Lissin, D. V., Gomperts, S. N., Carroll, R. C., Christine, C. W., Kalman, D., Kitamura, M., Hardy, S., Nicoll, R. A., Malenka, R. C., and von Zastrow, M. (1998). Activity differentially regulates the surface expression of synaptic AMPA and NMDA glutamate receptors. Proc. Natl. Acad. Sci. U.S.A. 95, 7097�.

Lu, Y., Christian, K., and Lu, B. (2008). BDNF: a key regulator for protein synthesis-dependent LTP and long-term memory? Neurobiol. Learn. Mem. 89, 312�.

Maffei, A., and Fontanini, A. (2009). Network homeostasis: a matter of coordination. Curr. Opin. Neurobiol. 19, 168�.

Maffei, A., Nataraj, K., Nelson, S. B., and Turrigiano, G. G. (2006). Potentiation of cortical inhibition by visual deprivation. Nature 443, 81�.

Maffei, A., Nelson, S. B., and Turrigiano, G. G. (2004). Selective reconfiguration of layer 4 visual cortical circuitry by visual deprivation. Nat. Neurosci. 7, 1353�.

Maffei, A., and Turrigiano, G. G. (2008). Multiple modes of network homeostasis in visual cortical layer 2/3. J. Neurosci. 28, 4377�.

Malenka, R. C., and Nicoll, R. A. (1999). Long-term potentiation – a decade of progress? Science 285, 1870�.

Marder, E., and Bucher, D. (2007). Understanding circuit dynamics using the stomatogastric nervous system of lobsters and crabs. Annu. Rev. Physiol. 69, 291�.

Markram, H., Lubke, J., Frotscher, M., and Sakmann, B. (1997). Regulation of synaptic efficacy by coincidence of postsynaptic APs and EPSPs. Science 275, 213�.

Markram, H., and Tsodyks, M. (1996). Redistribution of synaptic efficacy between neocortical pyramidal neurons. Nature 382, 807�.

McCoy, M. K., and Tansey, M. G. (2008). TNF signaling inhibition in the CNS: implications for normal brain function and neurodegenerative disease. J. Neuroinflammation 5, 45.

McCoy, P. A., Huang, H. S., and Philpot, B. D. (2009). Advances in understanding visual cortex plasticity. Curr. Opin. Neurobiol. 19, 298�.

Minichiello, L. (2009). TrkB signalling pathways in LTP and learning. Nat. Rev. Neurosci. 10, 850�.

Misonou, H., Mohapatra, D. P., Park, E. W., Leung, V., Zhen, D., Misonou, K., Anderson, A. E., and Trimmer, J. S. (2004). Regulation of ion channel localization and phosphorylation by neuronal activity. Nat. Neurosci. 7, 711�.

Mohapatra, D. P., Misonou, H., Pan, S. J., Held, J. E., Surmeier, D. J., and Trimmer, J. S. (2009). Regulation of intrinsic excitability in hippocampal neurons by activity-dependent modulation of the KV2.1 potassium channel. Channels (Austin) 3, 46�.

Morrison, A., Aertsen, A., and Diesmann, M. (2007). Spike-timing-dependent plasticity in balanced random networks. Neural Comput. 19, 1437�.

Morrison, A., Diesmann, M., and Gerstner, W. (2008). Phenomenological models of synaptic plasticity based on spike timing. Biol. Cybern. 98, 459�.

Mrsic-Flogel, T. D., Hofer, S. B., Ohki, K., Reid, R. C., Bonhoeffer, T., and Hubener, M. (2007). Homeostatic regulation of eye-specific responses in visual cortex during ocular dominance plasticity. Neuron 54, 961�.

Mu, Y., Otsuka, T., Horton, A. C., Scott, D. B., and Ehlers, M. D. (2003). Activity-dependent mRNA splicing controls ER export and synaptic delivery of NMDA receptors. Neuron 40, 581�.

Murthy, V. N., Schikorski, T., Stevens, C. F., and Zhu, Y. (2001). Inactivity produces increases in neurotransmitter release and synapse size. Neuron 32, 673�.

Narayanan, R., and Johnston, D. (2008). The h channel mediates location dependence and plasticity of intrinsic phase response in rat hippocampal neurons. J. Neurosci. 28, 5846�.

Nelson, A. B., Krispel, C. M., Sekirnjak, C., and du Lac, S. (2003). Long-lasting increases in intrinsic excitability triggered by inhibition. Neuron 40, 609�.

Neves, G., Cooke, S. F., and Bliss, T. V. (2008). Synaptic plasticity, memory and the hippocampus: a neural network approach to causality. Nat. Rev. Neurosci. 9, 65�.

O𠆛rien, R. J., Kamboj, S., Ehlers, M. D., Rosen, K. R., Fischbach, G. D., and Huganir, R. L. (1998). Activity-dependent modulation of synaptic AMPA receptor accumulation. Neuron 21, 1067�.

O’Leary, T., van Rossum, M. C., and Wyllie, D. J. (2010). Homeostasis of intrinsic excitability in hippocampal neurones: dynamics and mechanism of the response to chronic depolarization. J. Physiol. 588, 157�.

Pfister, J. P., and Gerstner, W. (2006). Triplets of spikes in a model of spike timing-dependent plasticity. J. Neurosci. 26, 9673�.

Philpot, B. D., Cho, K. K., and Bear, M. F. (2007). Obligatory role of NR2A for metaplasticity in visual cortex. Neuron 53, 495�.

Philpot, B. D., Espinosa, J. S., and Bear, M. F. (2003). Evidence for altered NMDA receptor function as a basis for metaplasticity in visual cortex. J. Neurosci. 23, 5583�.

Philpot, B. D., Sekhar, A. K., Shouval, H. Z., and Bear, M. F. (2001). Visual experience and deprivation bidirectionally modify the composition and function of NMDA receptors in visual cortex. Neuron 29, 157�.

Poo, M. M. (2001). Neurotrophins as synaptic modulators. Nat. Rev. Neurosci. 2, 24�.

Poolos, N. P. (2005). The h-channel: a potential channelopathy in epilepsy? Epilepsy Behav. 7, 51�.

Pratt, K. G., and Aizenman, C. D. (2007). Homeostatic regulation of intrinsic excitability and synaptic transmission in a developing visual circuit. J. Neurosci. 27, 8268�.

Quinlan, E. M., Philpot, B. D., Huganir, R. L., and Bear, M. F. (1999). Rapid, experience-dependent expression of synaptic NMDA receptors in visual cortex in vivo. Nat. Neurosci. 2, 352�.

Rabinowitch, I., and Segev, I. (2008). Two opposing plasticity mechanisms pulling a single synapse. Trends Neurosci. 31, 377�.

Ramocki, M. B., and Zoghbi, H. Y. (2008). Failure of neuronal homeostasis results in common neuropsychiatric phenotypes. Nature 455, 912�.

Rao, A., and Craig, A. M. (1997). Activity regulates the synaptic localization of the NMDA receptor in hippocampal neurons. Neuron 19, 801�.

Renart, A., Song, P., and Wang, X. J. (2003). Robust spatial working memory through homeostatic synaptic scaling in heterogeneous cortical networks. Neuron 38, 473�.

Reyes, A., and Sakmann, B. (1999). Developmental switch in the short-term modification of unitary EPSPs evoked in layer 2/3 and layer 5 pyramidal neurons of rat neocortex. J. Neurosci. 19, 3827�.

Rial Verde, E. M., Lee-Osbourne, J., Worley, P. F., Malinow, R., and Cline, H. T. (2006). Increased expression of the immediate-early gene arc/arg3.1 reduces AMPA receptor-mediated synaptic transmission. Neuron 52, 461�.

Rittenhouse, C. D., Shouval, H. Z., Paradiso, M. A., and Bear, M. F. (1999). Monocular deprivation induces homosynaptic long-term depression in visual cortex. Nature 397, 347�.

Roth-Alpermann, C., Morris, R. G., Korte, M., and Bonhoeffer, T. (2006). Homeostatic shutdown of long-term potentiation in the adult hippocampus. Proc. Natl. Acad. Sci. U.S.A. 103, 11039�.

Royer, S., and Pare, D. (2003). Conservation of total synaptic weight through balanced synaptic depression and potentiation. Nature 422, 518�.

Rubin, J. E. (2001). Steady states in an iterative model for multiplicative spike-timing-dependent plasticity. Network 12, 131�.

Rubin, J. E., Gerkin, R. C., Bi, G. Q., and Chow, C. C. (2005). Calcium time course as a signal for spike-timing-dependent plasticity. J. Neurophysiol. 93, 2600�.

Rumsey, C. C., and Abbott, L. F. (2006). Synaptic democracy in active dendrites. J. Neurophysiol. 96, 2307�.

Rutherford, L. C., Nelson, S. B., and Turrigiano, G. G. (1998). BDNF has opposite effects on the quantal amplitude of pyramidal neuron and interneuron excitatory synapses. Neuron 21, 521�.

Seeburg, D. P., Feliu-Mojer, M., Gaiottino, J., Pak, D. T., and Sheng, M. (2008). Critical role of CDK5 and Polo-like kinase 2 in homeostatic synaptic plasticity during elevated activity. Neuron 58, 571�.

Shepherd, J. D., Rumbaugh, G., Wu, J., Chowdhury, S., Plath, N., Kuhl, D., Huganir, R. L., and Worley, P. F. (2006). Arc/Arg3.1 mediates homeostatic synaptic scaling of AMPA receptors. Neuron 52, 475�.

Shin, M., and Chetkovich, D. M. (2007). Activity-dependent regulation of h channel distribution in hippocampal CA1 pyramidal neurons. J. Biol. Chem. 282, 33168�.

Shouval, H. Z., Bear, M. F., and Cooper, L. N. (2002). A unified model of NMDA receptor-dependent bidirectional synaptic plasticity. Proc. Natl. Acad. Sci. U.S.A. 99, 10831�.

Sjostrom, P. J., Rancz, E. A., Roth, A., and Hausser, M. (2008). Dendritic excitability and synaptic plasticity. Physiol. Rev. 88, 769�.

Sjostrom, P. J., Turrigiano, G. G., and Nelson, S. B. (2001). Rate, timing, and cooperativity jointly determine cortical synaptic plasticity. Neuron 32, 1149�.

Sjostrom, P. J., Turrigiano, G. G., and Nelson, S. B. (2003). Neocortical LTD via coincident activation of presynaptic NMDA and cannabinoid receptors. Neuron 39, 641�.

Smith, S. J. (1998). Glia help synapses form and function. Curr. Biol. 8, R158–R160.

Song, S., Miller, K. D., and Abbott, L. F. (2000). Competitive Hebbian learning through spike-timing-dependent synaptic plasticity. Nat. Neurosci. 3, 919�.

Song, S., Sjostrom, P. J., Reigl, M., Nelson, S., and Chklovskii, D. B. (2005). Highly nonrandom features of synaptic connectivity in local cortical circuits. PLoS Biol. 3, e68. doi: 10.1371/journal.pbio.0030068.

Stellwagen, D., and Malenka, R. C. (2006). Synaptic scaling mediated by glial TNF-alpha. Nature 440, 1054�.

Sutton, M. A., Ito, H. T., Cressy, P., Kempf, C., Woo, J. C., and Schuman, E. M. (2006). Miniature neurotransmission stabilizes synaptic function via tonic suppression of local dendritic protein synthesis. Cell 125, 785�.

Sutton, M. A., and Schuman, E. M. (2009). Partitioning the synaptic landscape: distinct microdomains for spontaneous and spike-triggered neurotransmission. Sci. Signal. 2, pe19.

Swanwick, C. C., Murthy, N. R., and Kapur, J. (2006). Activity-dependent scaling of GABAergic synapse strength is regulated by brain-derived neurotrophic factor. Mol. Cell. Neurosci. 31, 481�.

Thiagarajan, T. C., Lindskog, M., and Tsien, R. W. (2005). Adaptation to synaptic inactivity in hippocampal neurons. Neuron 47, 725�.

Tononi, G., Sporns, O., and Edelman, G. M. (1999). Measures of degeneracy and redundancy in biological networks. Proc. Natl. Acad. Sci. U.S.A. 96, 3257�.

Toyoizumi, T., Pfister, J. P., Aihara, K., and Gerstner, W. (2005). Generalized Bienenstock𠄼ooper–Munro rule for spiking neurons that maximizes information transmission. Proc. Natl. Acad. Sci. U.S.A. 102, 5239�.

Trasande, C. A., and Ramirez, J. M. (2007). Activity deprivation leads to seizures in hippocampal slice cultures: is epilepsy the consequence of homeostatic plasticity? J. Clin. Neurophysiol. 24, 154�.

Tsodyks, M. (2002). Spike-timing-dependent synaptic plasticity – the long road towards understanding neuronal mechanisms of learning and memory. Trends Neurosci. 25, 599�.

Tsodyks, M. V., and Markram, H. (1997). The neural code between neocortical pyramidal neurons depends on neurotransmitter release probability. Proc. Natl. Acad. Sci. U.S.A. 94, 719�.

Turrigiano, G., Abbott, L. F., and Marder, E. (1994). Activity-dependent changes in the intrinsic properties of cultured neurons. Science 264, 974�.

Turrigiano, G. G. (2008). The self-tuning neuron: synaptic scaling of excitatory synapses. Cell 135, 422�.

Turrigiano, G. G., Leslie, K. R., Desai, N. S., Rutherford, L. C., and Nelson, S. B. (1998). Activity-dependent scaling of quantal amplitude in neocortical neurons. Nature 391, 892�.

Vacher, H., Mohapatra, D. P., and Trimmer, J. S. (2008). Localization and targeting of voltage-dependent ion channels in mammalian central neurons. Physiol. Rev. 88, 1407�.

van Rossum, M. C., Bi, G. Q., and Turrigiano, G. G. (2000). Stable Hebbian learning from spike timing-dependent plasticity. J. Neurosci. 20, 8812�.

van Welie, I., Remme, M. W., van Hooft, J. A., and Wadman, W. J. (2006a). Different levels of Ih determine distinct temporal integration in bursting and regular-spiking neurons in rat subiculum. J. Physiol. 576, 203�.

van Welie, I., van Hooft, J. A., and Wadman, W. J. (2006b). Background activity regulates excitability of rat hippocampal CA1 pyramidal neurons by adaptation of a K+ conductance. J. Neurophysiol. 95, 2007�.

van Welie, I., van Hooft, J. A., and Wadman, W. J. (2004). Homeostatic scaling of neuronal excitability by synaptic modulation of somatic hyperpolarization-activated Ih channels. Proc. Natl. Acad. Sci. U.S.A. 101, 5123�.

Wang, H., and Wagner, J. J. (1999). Priming-induced shift in synaptic plasticity in the rat hippocampus. J. Neurophysiol. 82, 2024�.

Watt, A. J., Sjostrom, P. J., Hausser, M., Nelson, S. B., and Turrigiano, G. G. (2004). A proportional but slower NMDA potentiation follows AMPA potentiation in LTP. Nat. Neurosci. 7, 518�.

Watt, A. J., van Rossum, M. C., MacLeod, K. M., Nelson, S. B., and Turrigiano, G. G. (2000). Activity coregulates quantal AMPA and NMDA currents at neocortical synapses. Neuron 26, 659�.

Wierenga, C. J., Ibata, K., and Turrigiano, G. G. (2005). Postsynaptic expression of homeostatic plasticity at neocortical synapses. J. Neurosci. 25, 2895�.

Wierenga, C. J., Walsh, M. F., and Turrigiano, G. G. (2006). Temporal regulation of the expression locus of homeostatic plasticity. J. Neurophysiol. 96, 2127�.

Wilhelm, J. C., Rich, M. M., and Wenner, P. (2009). Compensatory changes in cellular excitability, not synaptic scaling, contribute to homeostatic recovery of embryonic network activity. Proc. Natl. Acad. Sci. U.S.A. 106, 6760�.

Wu, W. W., Chan, C. S., Surmeier, D. J., and Disterhoft, J. F. (2008). Coupling of L-type Ca 2+ channels to KV7/KCNQ channels creates a novel, activity- dependent, homeostatic intrinsic plasticity. J. Neurophysiol. 100, 1897�.

Xu, T. X., Sotnikova, T. D., Liang, C., Zhang, J., Jung, J. U., Spealman, R. D., Gainetdinov, R. R., and Yao, W. D. (2009a). Hyperdopaminergic tone erodes prefrontal long-term potential via a D2 receptor-operated protein phosphatase gate. J. Neurosci. 29, 14086�.

Xu, Z., Chen, R. Q., Gu, Q. H., Yan, J. Z., Wang, S. H., Liu, S. Y., and Lu, W. (2009b). Metaplastic regulation of long-term potentiation/long-term depression threshold by activity-dependent changes of NR2A/NR2B ratio. J. Neurosci. 29, 8764�.

Yashiro, K., and Philpot, B. D. (2008). Regulation of NMDA receptor subunit expression and its implications for LTD, LTP, and metaplasticity. Neuropharmacology 55, 1081�.

Yasui, T., Fujisawa, S., Tsukamoto, M., Matsuki, N., and Ikegaya, Y. (2005). Dynamic synapses as archives of synaptic history: state-dependent redistribution of synaptic efficacy in the rat hippocampal CA1. J. Physiol. 566, 143�.

Yeung, L. C., Shouval, H. Z., Blais, B. S., and Cooper, L. N. (2004). Synaptic homeostasis and input selectivity follow from a calcium-dependent plasticity model. Proc. Natl. Acad. Sci. U.S.A. 101, 14943�.

Zhang, L. I., Tao, H. W., Holt, C. E., Harris, W. A., and Poo, M. (1998). A critical window for cooperation and competition among developing retinotectal synapses. Nature 395, 37�.

Zhang, W., and Linden, D. J. (2003). The other side of the engram: experience-driven changes in neuronal intrinsic excitability. Nat. Rev. Neurosci. 4, 885�.

Zucker, R. S., and Regehr, W. G. (2002). Short-term synaptic plasticity. Annu. Rev. Physiol. 64, 355�.

Keywords: homeostatic plasticity, synaptic scaling, intrinsic plasticity, STDP, BCM, LTP, LTD, stability

Citation: Watt AJ and Desai NS (2010) Homeostatic plasticity and STDP: keeping a neuron’s cool in a fluctuating world. Front. Syn. Neurosci. 2:5. doi: 10.3389/fnsyn.2010.00005

Received: 18 February 2010 Paper pending published: 23 April 2010
Accepted: 17 May 2010 Published online: 07 June 2010

Henry Markram, Ecole Polytechnique Federale de Lausanne, Switzerland

Gina Turrigiano, Brandeis University, USA

Copyright: © 2010 Watt and Desai. This is an open-access article subject to an exclusive license agreement between the authors and the Frontiers Research Foundation, which permits unrestricted use, distribution, and reproduction in any medium, provided the original authors and source are credited.


Discussion

While a real environment comprises several different contexts, humans and animals retain the experience of past contexts to perform well when they find themselves in the same context in the future. This ability is known as conservation of learning or cognitive flexibility 25 . Although analogous learning is likely to happen during BSS, the conventional biological BSS algorithms 37,38 must forget the memory of past contexts to learn a new one. Thereby, when the agent subsequently encounters a previously experienced context, it needs to relearn it from the very beginning. We overcame this limitation by using the described algorithm, the EGHR. The crucial property of the EGHR is that when the number of inputs is larger than the number of sources, the synaptic matrix contains a null space in which synaptic strengths are equally optimized for performing BSS. Hence, with sufficiently redundant inputs, the EGHR can make the synaptic matrix optimal for every experienced context. This is an ability that the conventional biologically plausible BSS algorithms do not have, due to the constraint that the number of inputs and outputs must be equal 15 however, we argue that this ability is crucial for animals to perceive and adapt to dynamically changing multi-context environments. It is also crucial for animals to generalize past learning to inexperienced contexts. We also found that, if there is a common feature shared across the training contexts, the EGHR can extract it and generalize the BSS result to inexperienced test contexts. This speaks to a generalization capability and transfer learning, implying the prevention of overfitting to a specific context alternatively, one might see this as an extraction of a general concept across contexts. Therefore, we argue that the EGHR is a good candidate model for describing the neural mechanism of conservation of learning or cognitive flexibility for BSS.

Moreover, the process of extracting hidden sources in a multi-context BSS setup can be seen as a novel concept of dimensionality reduction 42 . If the dimensions of input are greater than the product of the number of sources and the number of contexts, the EGHR can extract the low-dimensional sources (up to context-dependent permutations and sign-flips), while filtering out a large number of context-dependent signals induced by changes in the mixing matrix. ICA algorithms for multi-context BSS 39,40,41 and undercomplete ICA for compressing data dimensionality 15,17,53 have been separately developed. Nevertheless, conventional ICA algorithms for multi-context BSS cannot learn efficient dimensionality reduction, and thus, to our knowledge, our study is the first to attempt dimensionality reduction in the multi-context BSS setup. This method is particularly powerful when a common feature is shared across the contexts, because the EGHR can make each neuron encode an identical source across all contexts. Our results are different from those obtained using standard dimensionality reduction approaches by PCA 18,19 , because PCA is used for extracting subspaces of high-variance principal components and hence would preferentially extract the context-dependent varying features, given that each source has the same variance. Therefore, our study proposes an attractive use of the EGHR for dimensionality reduction.

It is worth noting that the application of standard ICA algorithms to high-pass filtered inputs cannot solve the multi-context BSS problem. This is because context-dependent changes in the mixing matrix not only change the means of the inputs, which can be removed by high-pass filtering, but also change the gain of how fluctuations of each source are propagated to input fluctuations. Hence, the difference in contexts cannot be expressed as a linear ICA problem after high-pass input filtering. Therefore, selective extraction of context-invariant features is an advantage of the EGHR. Moreover, if provided with redundant input, the EGHR can solve multi-context BSS even if the context changes continuously in time, as we demonstrated in Figs. 4, 5.

We demonstrated that a neural network learns to distinguish individual birdsongs from their superposition. Young songbirds learn songs by mimicking adult birds’ songs 54,55,56,57 . A study reported that neurons in songbirds’ higher auditory cortex exhibit a teacher specific activity 58 . One can imagine those neurons correspond to the expectation of hidden sources (u), as considered in this study. Importantly, the natural environment that young songbirds encounter is dynamic, as we considered in Fig. 5. Therefore, the conventional BSS setup, which assumes a static environment or context, is not suitable for explaining this problem. It is interesting to consider that young songbirds might employ some computational mechanism similar to the EGHR to distinguish a teacher’s song from other songs in a dynamically changing environment.

Biological neural networks implement an EGHR-like learning rule. The main ingredients of the EGHR are Hebbian plasticity and the third scalar factor that modulates it. Hebbian plasticity occurs in the brain depending on the activity level 44,59 , spike timings 60,61,62,63 , or burst timings 64 of pre- and post-synaptic neurons. In contrast, the third scalar factor can modify the learning rate and even invert Hebbian to anti-Hebbian plasticity 50 , similarly to what we propose for the EGHR. In general, such a modulation forms the basis of a three-factor learning rule, a concept that has recently received attention (see 20,65,66 for reviews), and is supported by experiments on various neuromodulators and neurotransmitters, such as dopamine 45,46,47 , noradrenaline 48,49 , muscarine 67 , and GABA 50,51 , as well as glial factors 52 . (These factors may encode reward 68,69,70,71,72 , likelihood 73 , novelty/surprise 74 , or error from a prior belief 15,17 to achieve various types of learning, implying the existence of a unified three-factor learning framework.) Importantly, the EGHR only requires such a signal that conveys global information to neurons to achieve learning. Furthermore, a study using in vitro neural networks suggested that neurons perform simple BSS using a plasticity rule that is different from the most basic form of Hebbian plasticity, by which synaptic strengths are updated purely as a product of pre- and postsynaptic activity 75,76 . A candidate implementation of the EGHR can be made for cortical pyramidal cells and inhibitory neurons the former constituting the EGHR output neurons and encoding the expectations of hidden sources, and the latter constituting the third scalar factor and calculating the nonlinear sum of activity in surrounding pyramidal cells. This view is consistent with the circuit structure reported for the visual cortex 77,78 . These empirical evidences support the biological plausibility of the EGHR as a candidate model of neuronal BSS.

A local computation of the EGHR is highly desirable for neuromorphic engineering 23,24,79,80 . The EGHR updates synapses by a simple product of pre- and postsynaptic neurons’ activity and a global scalar factor. Because of this, less information transfer between neurons is required, compared to conventional ICA methods that require non-local information 10,11,12 , all-to-all plastic lateral inhibition between output neurons 37,38 , or an additional processing step for decorrelation 13 . The simplicity of the EGHR is a great advantage when implemented in a neuromorphic chip because it can reduce the space for wiring and the energy consumption. Furthermore, unlike the conventional ICA algorithms that assume an equal number of input and output neurons, a neuromorphic chip that employs the EGHR with redundant inputs would perform BSS in multiple contexts, as allowed by the network memory capacity, without requiring readaptation. The generalization capability of the EGHR, as demonstrated in Fig. 6, is an additional benefit, as the EGHR captures the common features shared across training contexts to perform BSS in inexperienced test contexts.

Notably, although we considered a linear BSS problem in this study, multi-context BSS can be extended to non-linear BSS, in which the inputs are generated through a non-linear mixture of sources 81,82 . To solve this problem, a promising approach would be to use a linear neural network. A recent study showed that when the ratio of input-to-source dimensions and source number are large, a linear neural network can find an optimal linear encoder that separates the true sources through PCA and ICA, thus asymptotically achieving zero BSS error 83 . Because both the asymptotic linearization and multi-context BSS by the EGHR are based on high-dimensional sensory inputs, combining these two might be a useful approach to solve the multi-context and non-linear BSS problem.

In summary, we demonstrated that the EGHR can retain memories of past contexts and, once the learning is achieved for every context, it can perform multi-context BSS without further updating synapses. Moreover, the EGHR can find common features shared across contexts, if present, and uses them to generalize the learning result to inexperienced contexts. Therefore, the EGHR will be useful for understanding the neural mechanisms of flexible inference and sensory representation under dynamically changing environments, and for creating brain-inspired artificial general intelligence.


Watch the video: Hebb Rule with detailed Example (May 2022).