Skip to Main content Skip to Navigation

Codage hippocampique par transitions spatio-temporelles pour l'apprentissage autonome de comportements dans des tâches de navigation sensori-motrice et de planification en robotique

Abstract : This thesis takes interest in the mechanisms facilitating the autonomous acquisition of be- haviors in animals and proposes to use these mechanisms in the frame of robotic tasks. Artificial neural networks are used to model cerebral structures, both to understand how these structures work and to design robust and adaptive algorithms for robot control. The work presented here is based on a model of the hippocampus capable of learning the temporal relationship between perceptive events. The neurons performing this learning, called transition cells, can predict which future events the robot could encounter. These transitions support the building of a cognitive map, located in the prefrontal and/or parietal cortex. The map can be learned by a mobile robot exploring an unknown environment and then be used to plan paths in order to reach one or several goals. Apart from their use in building a cognitive map, transition cells are also the basis for the design of a model of reinforcement learning. A biologically plausible neural implementation of the Q-learning algorithm, using transitions, is made by taking inspiration from the basal ganglia. This architecture provides an alternative strategy to the cognitive map planning strategy. The reinforcement learning strategy requires a longer learning period but corresponds more to an au- tomatic low-level behavior. Experiments are carried out with both strategies used in cooperation and lesions of the prefrontal cortex and basal ganglia allow to reproduce experimental results obtained with rats. Transition cells can learn temporally precise relations predicting the exact timing when an event should be perceived. In a model of interactions between the hippocampus and prefrontal cortex, we show how these predictions can explain in-vivo recordings in these cerebral struc- tures, in particular when rat is carrying out a task during which it must remain stationary for 2 seconds on a goal location to obtain a reward. The learning of temporal information about the environment and the behavior of the robot allows the system to detect regularity. On the contrary, the absence of a predicted event can signal a failure in the behavior of the robot, which can be detected and acted upon in order to modulate the failing behavior. Consequently, a fail- ure detection system is developed, taking advantage of the temporal predictions provided by the hippocampus and the interaction between behavior modulation functions in the prefrontal cortex and reinforcement learning in the basal ganglia. Several robotic experiments are conducted, in which the failure signal is used to modulate, immediately at first, the behavior of the robot in order to stop selecting actions which lead to failures and explore other strategies. The signal is then used in a more lasting way by modulating the learning of the associations leading to the selection of an action so that the repeted failures of an action in a particular context lead to the suppression of this association. Finally, after having used the model in the frame of navigation, we demonstrate its general- ization capabilities by using it to control a robotic arm in a trajectory planning task. This work constitutes an important step towards obtaining a generic and unified model allowing the control of various robotic setups and the learning of tasks of different natures.
Complete list of metadatas
Contributor : Julien Hirel <>
Submitted on : Tuesday, January 17, 2012 - 6:10:19 PM
Last modification on : Monday, January 25, 2021 - 3:16:03 PM
Long-term archiving on: : Tuesday, December 13, 2016 - 11:33:38 PM


  • HAL Id : tel-00660862, version 1


Julien Hirel. Codage hippocampique par transitions spatio-temporelles pour l'apprentissage autonome de comportements dans des tâches de navigation sensori-motrice et de planification en robotique. Apprentissage [cs.LG]. Université de Cergy Pontoise, 2011. Français. ⟨tel-00660862⟩



Record views


Files downloads