Trajectory and intention forecasting
There are many challenges to overcome for a safe coexistence of autonomous cars and pedestrians to be possible: the limited on-board observations, the very different weather conditions (rain, snow, dust, etc), the presence of multiple road users in the same scene, the very different motion patterns users may have, the large variety of situations, to give only a few ones.
Our proposal is oriented to pedestrian/cyclist motion prediction at short (1-2 s) and mid term (10-20 s) for anticipating potential collision risks. Our key insight is that pedestrians/cyclists movements and intents are governed by their context, e.g. social forces and environment constraints or semantics. For an effective estimation of pedestrian or cyclist intent, situation-awareness is of paramount importance, which implies to rely on the immediate surroundings of the users (presence of other users, presence of obstacles, presence of road crossings…), but also on scene semantics. One of the direction we are considering is to use public geo-tagged information (e.g. from search engines such as Google Maps or OpenStreetMap) to extract would-be relevant semantic cues to be used in the inference system. For example, the presence of an elementary school near a crossing may make probable some massive road crossing at peak hours; the presence of a bar may raise the odds for erratic behaviours in late night. Similarly, some latent variables are important to consider for motion prediction: for example, the surrounding obstacle map and the specific spatial goal of a pedestrian on the other side of a crossing will probably influence the shape of his path along the crossing; the streetlight color should be an important factor for the crossing decision; also, would we suspect that a road user is currently looking at his mobile phone, as it occurs frequently in modern day streets, this should modify critically our prediction of its short-term motion and may also lower the odds that the pedestrian takes into account the street light; last, the presence of other pedestrians has also impact on its trajectory.
Related thesis
Juan Luis Baldelomar Cabrera. "Trajectory forecasting for autonomous vehicles". MSc. Thesis, CIMAT A.C., 2023.
Javad Amirian. "Human Motion Trajectory Prediction for Robot Navigation". PhD. Thesis, U. de Rennes 1, 2021.
Andre Berenice Ek Hobak. "Contribuciones a la predicción de Trayectorias Haciendo Uso de Apariencia y Flujo Óptico". MSc. Thesis, CIMAT A.C., 2021.
José Gallardo Monroy. "VAE models in trajectory prediction". MSc. Thesis, CIMAT A.C., 2020.
Related publications
Pei Xu, Jean-Bernard Hayet, Ioannis Karamouzas, "Context-Aware Timewise VAEs for Real-Time Vehicle Trajectory Prediction." IEEE Robotics and Automation Letters, 2023.
Pei Xu, Jean{-}Bernard Hayet, Ioannis Karamouzas, "SocialVAE: Human Trajectory Prediction Using Timewise Latents." the proceedings of Computer Vision - ECCV 2022 - 17th European Conference, Tel Aviv, Israel, October 23-27, 2022, Proceedings, Part IV, 2022.
Javad Amirian, Bingqing Zhang, Francisco Castro, Juan Baldelomar, Jean-Bernard Hayet, Julien Pettre, "OpenTraj: Assessing Prediction Complexity in Human Trajectories Datasets." the proceedings of Proceedings of the Asian Conference on Computer Vision (ACCV), 2020.
Javad Amirian, Jean-Bernard Hayet, Julien Pettré, "Social Ways: Learning Multi-Modal Distributions of Pedestrian Trajectories With GANs." the proceedings of IEEE Conference on Computer Vision and Pattern Recognition Workshops, CVPR Workshops 2019, Long Beach, CA, USA, June 16-20, 2019, 2019.