Skip to Main content Skip to Navigation

Motion forecasting of the objects in road scenes

Abstract : The automotive industry is moving toward intelligent systems. The aim for more safety, comfort, or even full self-driving systems requires the vehicles to take actions instead of the driver. It may use light signals, brake, accelerate, turn, and shift gears. Performing safely such actions requires a decision-making process that anticipates the resulting situation. This is broken down into three "p" steps: perception, prediction, and planning. The perception system recognizes the current surrounding road-scene situation. The prediction system forecasts the future of the road-scene including the other road users. The planning system produces a driving intention.If the future was perfectly known, planning would be straight forward. However, the perception system makes partial and noisy observations. The future cannot be perfectly predicted because of the uncertainties in the observation and because the human driver's decisions are modeled imperfectly. Therefore, improving the motion forecasting model that can work with a noisy observation system in complex situations is a critical point for safety, comfort, and future applications toward autonomous vehicles.Motion forecasting can be modeled from various perspectives. The three main ones are kinematics and heuristics, statistics, and decision process. The kinematics and heuristics are the historical approaches that offer efficiency and robustness. Most vehicles using short-term forecasting systems on the road nowadays rely on such forecasting models. However, it leads to an overly conservative behavior when used in complex situations. The approach using decision processes shows good performance in complex scenarios. It is able to perform well in simulations and controlled environments. It combines forecasting and planning into a single task and may account for the reactions of the other drivers in the decisions it plans to take. However, in a noisy and wider, uncontrolled environment, statistical approaches still produce the best performances.In this work, we use neural networks to learn the statistics of the vehicle trajectories. We progressively build a motion forecasting model from a constant velocity baseline to a complex interaction-aware neural network model. The evaluation criteria that we establish are used to judge the quality of the forecast and to compare the models. The likelihood of future observation for the forecasted distribution is the main criterion that is commonly used in the applications. However, there is no universal forecasting quality criterion and it remains an open problem.We begin our tests with vanilla neural networks for maximum likelihood motion forecasting. These models are then modified to also learn to fit the expected forecasting error. They optimize the NLL. It is a likelihood criterion for a given dataset of trajectories. This does not lead to much improvement over the constant velocity model. We go on to add interaction awareness to the model using multi-head attention. This brings much improvement to the forecasts but it is still insufficient. We improve the model further by considering different future possibilities. Finally, the multi-head attention model is extended to also attend to the surrounding lane shapes. The resulting neural network counts several blocks that account for different aspects of the task. Experiments show that each part of the model is useful. We show that the trained model has learned specialized attention patterns and is able to make multi-modal forecasts. Our results were the winning entry at the two first Argoverse forecasting competition.
Document type :
Complete list of metadata
Contributor : Abes Star :  Contact
Submitted on : Tuesday, July 20, 2021 - 12:21:12 PM
Last modification on : Monday, August 2, 2021 - 10:32:30 AM


Version validated by the jury (STAR)


  • HAL Id : tel-03292330, version 1


Jean Mercat. Motion forecasting of the objects in road scenes. Automatic. Université Paris-Saclay, 2021. English. ⟨NNT : 2021UPASG008⟩. ⟨tel-03292330⟩



Record views


Files downloads