Program

8:50: Welcome by the main organizers

9:00-10:00 Session 1: Machine & Deep Learning

Chairman: M. Ang

• Title:  Learning from Human Driver Data for Humanized Autonomous Driving at Dynamic Scenes  presentation video 9:00-9:40
  Keynote speaker: Huijing Zhao (Peking University)

  • Abstract:  This talk contains three parts: 1) Naturalistic driving behavior study at PKU-POSS Lab since 2011. A large amount of driving data was collected during human driver’s naturalistic driving on real-world traffic, which contains multi-modal sensing data, driving trajectories of the ego and all-around scene vehicles, and lane-change and car following samples. Based on such data, driving behavior modeling and reason methods were studied, which have the focus on aware the scene. 2) Learning from naturalistic driving data for human-like autonomous highway driving. This work follows the traditional modular-based approach, whereas, the idea is to learn a cost function from human driving data, based on this, trajectory candidates with higher similarity to human driven one at the situation are more likely to be selected by the trajectory planning module. 3) Imitation learning for humanized autonomous navigation at crowded intersections. The study is conducted using a high-fidelity simulator CARLA, and an end-to-end approach is studied to learn control policy at dense intersection scenes. We share our results and findings, followed by discussions of challenges.

  • Biography: Huijing Zhao received B.S. degree in computer science from Peking University in 1991. She obtained M.E. degree in 1996 and Ph.D. degree in 1999 in civil engineering from the University of Tokyo, Japan. From 1999 to 2007, she was a postdoctoral researcher and visiting associate professor at the Center for Space Information Science, University of Tokyo. In 2007, she joined Peking University as a tenure-track professor at the School of Electronics Engineering and Computer Science. She became an associate professor with tenure on 2013 and was promoted to full professor on 2020. She is now a full professor at the School of Artificial Intelligence, Peking University. She has research interest in several areas in connection with intelligent vehicle and mobile robot, such as machine perception, behavior learning and motion planning, and she has special interests on the studies through real world data collection. She has co-authored more than 100 research papers published in refereed journals and topic level conferences. She serves as the PIs of a number of national and bi-national projects, and broad collaborations with industry. She is a co-chair of the IEEE RAS Technical Committee AGV-ITS. She serves as an associate editor of the IEEE Trans. on Intelligent Vehicle since 2016, and also at the conferences such as IROS17,21-22, IV17-22, ITSC18-22.

• Title: Highly Compressive Visual Self-localization Using Sequential Semantic Scene Graph and Graph Convolutional Neural Network paper presentation  9:40-09:55

  Authors: Yoshida Mitsuki, Yamamoto Ryogo, Tanaka Kanji

  Abstract: In this paper, we address the problem of image sequence-based self-localization from a new highly compressive scene representation called sequential semantic scene graph (S3G). Recent developments in deep graph convolutional neural networks (GCNs) have enabled a highly compressive visual place classifier (VPC) that can use a scene graph as the input modality. However, in such a highly compressive application, the amount of information lost in the image-to-graph mapping is significant and can damage the classification performance. To address this issue, we propose a pair of similarity-preserving mappings, image-to-nodes and image-to-edges, such that the nodes and edges act as absolute and relative features, respectively, that complement each other. Moreover, the proposed GCN-VPC is applied to a new task of viewpoint planning of the query image sequence, which contributes to further improvement in the VPC performance. Experiments using the public NCLT dataset validated the effectiveness of the proposed method.

• Title: TAS-NIR: A VIS+NIR Dataset for Fine-grained Semantic Segmentation in Unstructured Outdoor Environments paper  presentation 9:55-10:10
  Authors: Peter Mortimer, Hans-Joachim Wünsche

  Abstract: Vegetation Indices based on paired images of the visible color spectrum (VIS) and near infrared spectrum (NIR) have been widely used in remote sensing applications. These vegetation indices are extended for their application in autonomous driving in unstructured outdoor environments. In this domain we can combine traditional vegetation indices like the Normalized Difference Vegetation Index (NDVI) and Enhanced Vegetation Index (EVI) with Convolutional Neural Networks (CNNs) pre-trained on available VIS datasets. By laying a focus on learning calibrated CNN outputs, we can provide an approach to fuse known hand-crafted image features with CNN predictions for different domains as well. The method is evaluated on a VIS-NIR dataset of semantically annotated images in unstructured outdoor environments. The dataset will be released as part of this publication.

10:10-10:30 Coffee Break

10:30-12:00 Session 2: Perception & Situation  Awareness

Chairman: C. Laugier

• Title: Lidar-based 3D objection detection using deep learning for autonomous vehicles applications, a review.  presentation   10:30-11:10
  Keynote speaker: Vincent Fremont (LS2N, ECole Centrale de Nantes, France)

  • Abstract: 3D object detection for autonomous vehicles is an active research topic since it provides input for downstream tasks such as prediction of other road users motion that will strongly influence the motion planning of the ego vehicle. Moreover, due to the need for localizing objects in 3D space, object detection for autonomous vehicles is frequently performed on point cloud acquired by 3D LiDAR.
    Over the past years, deep neural networks have proven their efficiency in object detection tasks using camera images. Common approaches aim to extend these techniques to 3D point clouds. However, due to the intrinsic differences between 2D images and 3D LiDAR information, the application of classical neural networks on 3D point cloud data is still an open problem.
    In this talk, recent advances and breakthroughs about Lidar-based 3D object detection will be reviewed. Focus will be given on detection architectures that show robustness to change in Lidar resolution, real-time inference capabilities and network architecture flexibility for Lidar/camera fusion.

  • Biography: Prof. Vincent Frémont received the M.S. degree in automatic control and computer science from the Ecole Centrale de Nantes, France, in 2000 and the Ph.D. degree in automatic control and computer science from the Ecole Centrale de Nantes, France, in 2003. From 2005 to 2018, he was an Associate Professor at the Université de Technologie de Compiègne (UTC) within the Heudiasyc Lab, UMR CNRS 7253. Since 2018, he is a Full Professor at Ecole Centrale de Nantes within the ARMEN team at the LS2N Lab, UMR CNRS 6004. His research interests belong to perception systems and scene understanding for autonomous mobile robotics with an emphasis on 3D vision, deep learning and multi-sensor fusion for self-driving cars. He has co-authored more than 80 papers in refereed journals and conference papers.

• Title: Modeling and Using the Context of Navigation: Towards Context-Aware Navigation of Autonomous Vehicles  paper   presentation  11:10-11:25
  Authors: Sélim Chefchaouni Moussaoui, Alessandro Corrêa Victorino, Marie-Hélène Abel

  Abstract: The problem of the autonomous navigation of intelligent vehicles has been studied for years now, and nowadays efficient sensing-based controllers have been developed and are even included by autonomous car manufacturers. However, these controllers do not take into account the contextual information from the vehicle’s environment. Recently, a few studies tried to consider the possibility of considering this information in control laws, in order to adapt the vehicle’s behavior to each different situation it may be in. In this paper, we develop a method to model a context of navigation which provides information on both local and global navigation, using ontologies and reasoners. Then we explain how this information can be handled in local and global control laws. Finally, we illustrate our method with a small-sized ontology of the context of navigation, and its integration to be taken into account by a local driving controller.

• Title:  Small Object Change Detection for Small Obstacle Avoidance in Everyday Robot  Navigation  paper  presentation   11:25-11:40
  Authors: Koji Takeda, Kanji Tanaka, Yoshimasa Nakamura

  Abstract: This paper addresses the problem of small object change detection for small obstacle avoidance in everyday robot navigation. Despite recent research progress in the field of object detection and change detection, the problem of detecting semantically non-distinctive and visually small objects is still a challenging problem.We developed a practical image processing pipeline by combining state-of-the-art techniques from image retrieval, image registration, and image change detection. We then integrated the image processing pipeline into a traditional plan-sense-act cycle to realize a reactive collision avoidance system. Experiments using a real mobile robot verified the
  effectiveness of the proposed approach.

• Title:  Towards a Mixed-Reality framework for autonomous driving paper presentation 11:40-11:55
   Authors:  Imane Argui, Maxime Gueriau, Samia Ainouz

  Abstract: Testing autonomous driving algorithms on mobile systems in simulation is an essential step to validate the models and train the system for a large set of (possibly unpredictable and critical) situations. Yet, the transfer of the model from simulation to reality is challenging due to the reality gap (i.e., discrepancies between reality and simulation models). Mixed-reality environments enable testing models on real vehicles without taking financial and safety risks. Additionally, it can reduce the development costs of the system by providing faster testing and debugging for mobile robots. This paper proposes a preliminary work towards a mixed-reality framework for autonomous navigation based on RGB-D cameras. The aim is to represent the objects in two environments within a single display using an augmentation strategy. We tested a first prototype by introducing a differential robot able to navigate in its environment, visualize augmented objects and detect them correctly using a pre-trained model based on Faster R-CNN.

12:00-13:00 Lunch Break

13:00-14:30 Session 3 : Motion Planning &Navigation

Chairman: P. Martinet

• Title: Toward socially aware navigation : from pedestrian’s behavior modeling to proactive navigation presentation 13:00-13:40
  Keynote speaker: Anne Spalanzani (CHROMA team, Inria, France)

  • Abstract: This talk will present recent advancement in the field of autonomous navigation while cars are among humans. AV start to share the urban space with other road users. To be accepted, autonomous vehicle need to be seen as a social robot that transport people. That implies that the people inside must feel integrated in the environment, as they would be in a driven car. They expect, as well as people in the surroundings, the cybercar to behave accordingly adhering to social and urban conventions and negotiating its path among crowded environments. This talk will explore the complex problem of navigating autonomously in shared-space environments, where pedestrians and cars share the same environment.

  • Biography: Anne Spalanzani received the Ph.D. degree in cognitive and computer science from the University Joseph Fourier of Grenoble, France, in 1999. Since 2003, she is an Associate Professor at Grenoble Alps University, France, and since 2013 she is a Researcher in the Chroma Team at the National Institute for Research in Computer Science and Automation (Inria), Grenoble-Rhône-Alpes, France. Her research work is focused on safe navigation of robotic systems in dynamic and human populated environments. She has been working on the adaptability of autonomous systems to their environments and the consideration of the human environment by robotic systems for the last 20 years, and has published many articles in this field.

• • Title: Alignability maps for ensuring high-precision localization paper presentation 13:40-13:55
    Authors: Manuel Castellano-Quero, Tomasz Piotr Kucner, Martin Magnusson

    Abstract: Localization methods for mobile robots have been proven to work successfully in a wide variety of situations. However, there still exist certain conditions that may lead these methods to fail when deployed in real-world contexts. One of the most common issues is the lack or scarcity of geometric features in the environment, which is considered a main cause behind localization error, especially for those methods relying on laser-based sensory information. In this paper, we present a map that aims to spatially capture the risk of getting localization errors. We base our proposal on a so-called alignability metric, which represents the capacity of a given range scan to be aligned with subsequent ones. Through different experiments, we demonstrate that our approach serves to correctly encode the variety of visible features from a given position and that such variety has a decisive impact on localization error. Also, the obtained results from our tests enable us to affirm that our proposal can be used as a reliable prediction of the risk of getting localization error.

  • Title: Trajectory monitoring for a drone using intevral analysis  paper presentation 13:55-14:10
    Authors: Sylvain Largent, Julien Alexandre dit Sandretto

    Abstract: When modelizing a robot, uncertainties are bound to be taken into account.  Uncertainties may appear because of approximations linked to the model. Sometimes  uncertainties are unavoidable as they are linked to the sensors’ accuracies, or inherent to the control of the robot. For instance, interval observers could be used for parameter estimation and state estimation. This paper proposes a method to consider all these uncertainties and to monitor the reliance of trajectories using interval analysis. The case study of this article is to monitor the trajectory of a holonomic drone controlled by its velocity, but the monitoring could be extended to more complex dynamic systems.

14:30-15:00 Coffee break

15:00-15:30 Session 4: AI based challenge based on Dataset

• Introduction to the challenge :   Huijing Zhao  video   15:00-15:20
• Results of the challenge :   Yancheng Pan  video   15:00-15:25

15:30-17:00 Round table: Computer vision & AI in Assisted/Autonomous driving

Chairmen: C. Laugier, M. Ang

Computer Vision is one of the most important sensing modality for perception towards understanding the environment and situational awareness. This understanding is critical for planning the maneuvers required for autonomous vehicles (AVs). AI and specifically Machine Learning has shown tremendous improvements in providing perception capability to AVs. This roundtable discusses the latest developments and focus on the technical gaps and challenges. For example, an unsolved problem is predicting the motion of surrounding dynamic obstacles such as pedestrians, cyclists and human driven cars. One aim of the roundtable is to facilitate and nurture new collaborations among and between academia and industry.

• Introduction to the round Table 
• Title: Understanding the Challenges When 3D Semantic Segmentation Faces Class Imbalanced and OOD Data   presentation   video 15:30-15:50
  Speaker: Huijing Zhao (Peking University Beijing, China)

  • Abstract: Real-world datasets are very limited and often suffer from the class imbalance (long-tail) problem. Out-of-distribution (OOD) data is another key issue in open world, where an autonomous driving system needs to deal with new objects not present in the dataset. This research aims to understand the challenges of deep models facing class imbalanced and OOD data at driving scenes. We take 3D semantic segmentation task as an example, seek answers to the following questions: how does the class imbalance problem affect 3DSS model performance? Facing class imbalanced and OOD data, can the model be aware of its unsureness about semantic categories or ID/OOD predictions? We share our latest results and our findings in this talk.

  • Biography: Huijing Zhao is a full professor with tenure at the School of Intelligence and Technology, Peking University. She has research interest in several areas in connection with intelligent vehicle and mobile robot, such as machine perception, behavior learning and motion planning, and she has special interests on the studies through real world data collection.

• Title: Event-based vision and Deep Learning for dynamic scene analysis  presentation 15:50-16:10
  Speaker:  Vincent Fremont (Ecole Centrale de Nantes, France)

  • Abstract: Event-based cameras offer a new paradigm for vision-based dynamic scene analysis in the context of autonomous vehicles. However, compare to frame-based cameras, the local pixel-level changes caused by movement in the perceived environment are transmitted at the time they occur. Therefore, the loss of spatial consistency between the pixels implies the design new processing algorithms and new Deep Learning architectures. In this talk, the idea is to present recent advances and breakthroughs in this area, and to discuss how this kind of sensors can be efficiently integrated in autonomous vehicles multi-sensors perception systems.

  • Biography: Prof. Vincent Frémont is a Full Professor at Ecole Centrale de Nantes (France) and member of the LS2N Lab, UMR CNRS 6004. His research interests belong to perception systems and scene understanding for autonomous mobile robotics with an emphasis on 3D vision, deep learning and multi-sensor fusion for self-driving cars. He has co-authored more than 80 papers in refereed journals and conference papers.

• Title: The multidimensional complexity of Safety Assurance applied to Computer Vision for ADS & AD  presentation 16:10-16:30
  Speaker: Dr Javier-Ibanez Guzman  (Renault S.A., France)
  

  • Abstract: Currently, AI is an integral part of current computer vision – based solutions. Its introduction whilst providing undeniable progress in the field, has introduced a different level of complexity.  The training of AI-based models depends on the data used for such purpose.  Most systems are expected to contribute to accident reduction. However, to obtain data of scenarios where accidents occur is difficult. How can we be sure that the perception systems are to have a minimum number of false positives?  This discussion will center on the uncertainties associated with data driven systems leading to constraints with regards to their safety assurance.

  • Biography: Javier Ibanez-Guzman, is currently expert on Autonomous Systems at Renault S.A. Owner for the functional architecture for the Level 4 Research prototypes. Co-director of the SIVALab, a common laboratory between UTC Compiegne – CNRS – Renault working on maps, perception and localization applicable to Intelligent Vehicles./p>

• Title: Leveraging Machine Learning for Vehicle Control in Academia and Industry 
   presentation   16:30-16:50
  Speaker: Dr Nathan Spielberg  (Motional, Boston, USA)
  

  • Abstract: Professional race car drivers can seamlessly leverage the vehicle’s full capabilities to drive as fast as possible on the race track, all while learning and improving over time. In order to be safe on the road under all situations including emergencies, autonomous vehicles should similarly be capable of leveraging all of the available road-tire friction to avoid a collision. By leveraging data over time, learned vehicle models show the ability to not only improve predictive performance at the limits, but also improve lap after lap as skilled humans do. By demonstrating learned models for control, this talk shows that autonomous racing vehicles can approach the control performance of skilled human race car drivers, even while driving on ice and snow. Secondly this talk examines how some of these techniques are applicable in industry, where there is an abundance of data to learn from.

  • Biography: Nathan Spielberg received the S.B degree in mechanical engineering from MIT in 2015 and masters in ME from Stanford in 2017. He worked at Stanford in the Dynamic Design lab with Prof. J. Christian Gerdes, where he obtained his Ph.D in 2021. His work focused on leveraged machine learning models to model and control the vehicle at its performance limits. Currently, Nathan works at Motional as a senior control engineer focusing on trajectory optimization and machine learning approaches for control.

     

16:50 Closing