Spiking Neurons with Nonlinear Dynamics for Fully Memristive Neural Networks

Like Comment
Read the paper

Artificial device elements that can implement similar functionalities of biological neurons or synapses are key building blocks for the construction of neuromorphic computing systems. Extensive efforts have been devoted to exploit the nonvolatile (or even analog) states of emerging memory devices, such as memristors, so as to efficiently perform the multiply-accumulate (MAC) functions based on physical laws, and this approach has found wide applications in building neural network accelerators.

Despite the encouraging processes, the above method is only applicable to linear operations. Recent studies in neuroscience have shown that even an individual neuron in our brain can have a variety of nonlinear mechanisms to convert synaptic inputs into neuronal outputs, and it is such nonlinear dynamics that endows single biological cells with considerable computational power. Traditionally, complex circuits and bulky capacitors are used to simulate this kind of bio-dynamics, but the conventional approach faces area and energy inefficiencies.

Published in Nature Communications, we use NbOx volatile memristors to build a type of artificial neuron based on the Pearson-Anson circuit configuration, and by exploiting the physics in such device/structure, the artificial neuron can not only realize traditional all-or-nothing, threshold-driven spiking but also enable spatiotemporal integration and dynamic logic, including XOR function that cannot be realized by a linear neural network. In addition, our spiking neurons can also achieve multiplicative gain modulation among different dendritic inputs, which clearly surpasses neuronal functions described by a simple point neuron model.

The NbOx neurons with enriched functionalities, together with the artificial synapses based on nonvolatile memristors such as TaOx and HfOx that have been more maturely developed, make it possible to construct a fully memristive neural network, where the neurons and synapses are formed by memristors with different dynamics. As a proof of concept, we build a 4×4 fully memristive neural network consisting of volatile NbOx memristor based neurons and nonvolatile TaOx memristor based synapses, monolithically integrated in a single crossbar array.

Figure 1. (a) Schematic diagram of gain modulation in biological neurons, where the driving input and modulatory input affect output firing rate jointly. (b) Experimental responses in NbOx neuron showing multiplicative gain modulation when applying different modulatory inputs (m = 0, 0.4, 0.5, 0.6 and 0.7 V). The background lines indicate the overall trend in data. (c) Spatiotemporal integration in NbOx neuron. (d) Monolithically integrated memristive neural network consisting of Pt/Ti/NbOx/Pt neurons and Pt/Ta/TaOx/Pt synapses.

The fully memristive neural network not only demonstrates application in pattern recognition through online learning, but can achieve more complex functions at the network level based on the rich nonlinear dynamics. Given the spatiotemporal integration in the NbOx neuron, highly efficient coincidence detection can be performed among spike sequences fed from different TaOx synapses, where high firing rate is obtained when synchronous inputs are transmitted. Moreover, the multiplicative gain modulation of the NbOx neuron forms an enabling factor in receptive field remapping, which can significantly enhance the stability of artificial visual systems, hence showing significantly enhanced computational power after utilizing the dynamics.

As a matter of fact, the beauty of memristor lies in the fact that it has a lot more complex dynamics than static resistors. We expect the nonlinear dynamics in memristors and memristive systems, if further exploited, can greatly boost the construction of more powerful, intelligent and energy-efficient computing systems.

 

Spiking Neurons with Spatiotemporal Dynamics and Gain Modulation for Monolithically Integrated Memristive Neural Networks, Nature Communications, 11, 3399 (2020).

Yuchao Yang

Director, Center for Brain Inspired Chips, Institute for Artificial Intelligence, Department of Micro/nanoelectronics, Peking University

No comments yet.