Skip to Main content Skip to Navigation

Exploration of reinforcement learning algorithms for autonomous vehicle visual perception and control

Abstract : Reinforcement learning is an approach to solve a sequential decision making problem. In this formalism, an autonomous agent interacts with an environment and receives rewards based on the decisions it makes. The goal of the agent is to maximize the total amount of rewards it receives. In the reinforcement learning paradigm, the agent learns by trial and error the policy (sequence of actions) that yields the best rewards.In this thesis, we focus on its application to the perception and control of an autonomous vehicle. To stay close to human driving, only the onboard camera is used as input sensor. We focus in particular on end-to-end training, i.e. a direct mapping between information from the environment and the action chosen by the agent. However, training end-to-end reinforcement learning for autonomous driving poses some challenges: the large dimensions of the state and action spaces as well as the instability and weakness of the reinforcement learning signal to train deep neural networks.The approaches we implemented are based on the use of semantic information (image segmentation). In particular, this work explores the joint training of semantic information and navigation.We show that these methods are promising and allow to overcome some limitations. On the one hand, combining segmentation supervised learning with navigation reinforcement learning improves the performance of the agent and its ability to generalize to an unknown environment. On the other hand, it enables to train an agent that will be more robust to unexpected events and able to make decisions limiting the risks.Experiments are conducted in simulation, and numerous comparisons with state of the art methods are made.
Complete list of metadata
Contributor : ABES STAR :  Contact
Submitted on : Tuesday, June 29, 2021 - 3:24:17 PM
Last modification on : Wednesday, May 11, 2022 - 3:20:03 PM
Long-term archiving on: : Thursday, September 30, 2021 - 7:06:32 PM


Version validated by the jury (STAR)


  • HAL Id : tel-03273748, version 1



Florence Carton. Exploration of reinforcement learning algorithms for autonomous vehicle visual perception and control. Machine Learning [cs.LG]. Institut Polytechnique de Paris, 2021. English. ⟨NNT : 2021IPPAE007⟩. ⟨tel-03273748⟩



Record views


Files downloads