Skip to Main content Skip to Navigation
Theses

Deep learning in event-based neuromorphic systems

Abstract : Inference and training in deep neural networks require large amounts of computation, which in many cases prevents the integration of deep networks in resource constrained environments. Event-based spiking neural networks represent an alternative to standard artificial neural networks that holds the promise of being capable of more energy efficient processing. However, training spiking neural networks to achieve high inference performance is still challenging, in particular when learning is also required to be compatible with neuromorphic constraints. This thesis studies training algorithms and information encoding in such deep networks of spiking neurons. Starting from a biologically inspired learning rule, we analyze which properties of learning rules are necessary in deep spiking neural networks to enable embedded learning in a continuous learning scenario. We show that a time scale invariant learning rule based on spike-timing dependent plasticity is able to perform hierarchical feature extraction and classification of simple objects of the MNIST and N-MNIST dataset. To overcome certain limitations of this approach we design a novel framework for spike-based learning, SpikeGrad, which represents a fully event-based implementation of the gradient backpropagation algorithm. We show how this algorithm can be used to train a spiking network that performs inference of relations between numbers and MNIST images. Additionally, we demonstrate that the framework is able to train large-scale convolutional spiking networks to competitive recognition rates on the MNIST and CIFAR10 datasets. In addition to being an effective and precise learning mechanism, SpikeGrad allows the description of the response of the spiking neural network in terms of a standard artificial neural network, which allows a faster simulation of spiking neural network training. Our work therefore introduces several powerful training concepts for on-chip learning in neuromorphic devices, that could help to scale spiking neural networks to real-world problems.
Complete list of metadatas

Cited literature [128 references]  Display  Hide  Download

https://tel.archives-ouvertes.fr/tel-02417462
Contributor : Abes Star :  Contact
Submitted on : Wednesday, December 18, 2019 - 11:14:08 AM
Last modification on : Tuesday, September 29, 2020 - 4:46:28 AM
Long-term archiving on: : Thursday, March 19, 2020 - 2:48:53 PM

File

84730_THIELE_2019_archivage.pd...
Version validated by the jury (STAR)

Identifiers

  • HAL Id : tel-02417462, version 1

Collections

Citation

Johannes C. Thiele. Deep learning in event-based neuromorphic systems. Artificial Intelligence [cs.AI]. Université Paris Saclay (COmUE), 2019. English. ⟨NNT : 2019SACLS403⟩. ⟨tel-02417462⟩

Share

Metrics

Record views

337

Files downloads

596