1. Vehicle On-board Multi-target Tracking and Motion Prediction Method for Vulnerable Road User

• Research Background

Vulnerable road users, such as pedestrians, riders or other cyclists, are common and diverse in actual traffic scenarios, yet they are susceptible and need to be protected. The detection is the first step to protect them. It needs to further track the positions of the same target between consecutive frames, form a motion trajectory and achieve multi-target tracking, so as to predict their motion intentions subsequently.

• Research Achievements

Aiming at the trajectory prediction of vulnerable road users (VRU, including pedestrians and riders) around self-driving vehicles in autonomous driving environment, an in-depth recurrent neural network trajectory prediction method integrating multiple trajectory prediction factors is proposed, taking into account the motion cues such as the positions and shapes of targets between consecutive frames and the deep convolutional feature of local rectangular frame.

In the multi-target trajectory prediction database of vulnerable road users established by Tsinghua-Daimler Joint Research Center for Sustainable Transportation, different variants of recurrent neural network (RNN / LSTM / GRU), target trajectory predictors and network parameter optimization strategies are comprehensively evaluated, and the effectiveness of the proposed VRU multi-target trajectory prediction method is verified.

• Leading Research Member: XIONG Hui
• Representative Papers

1. Xiong H, Flohr F. B., Wang S, Wang, B, Wang, J, and Li, K. Recurrent Neural Network Architectures for Vulnerable Road User Trajectory Prediction[C].2019 IEEE Intelligent Vehicles Symposium (IV),IEEE, 2019: 171-178.
2. Huang B, Xiong H, Wang J, Xu Q, Li X, and Li K. Detection-level fusion for multi-object perception in dense traffic environment[C].2017 IEEE International Conference on Multisensor Fusion and Integration for Intelligent Systems (MFI),IEEE, 2017: 411-416.

2. Multi-target Tracking Based on Roadside Camera 

• Research Background

Accurate and robust multi-target tracking method is the premise of cognition and decision-making for intelligent and connected vehicles, and plays an important role in the automatic driving perception technology. And the roadside camera has the prominent advantages of low cost, rich sensing information and wide field of vision, etc. Therefore, the multi-target tracking method based on roadside camera is of great significance for the realization of vehicle-road coordinated perception and high-level automatic driving.

3. Estimation Method of Tire-road Adhesion Coefficient Based on Current On-board Sensors

• Research Background

Tire-road adhesion coefficient is a considerable parameter for describing the current tire-road condition, and also an important prior knowledge of automatic driving system decision-making and control algorithm. It is of great significance to the safety and comfort of automatic driving system. Besides, for the connected traffic system, the estimation results of the adhesion coefficient can be shared, providing early warning for the driver and automatic driving system, thereby greatly enhancing the safety of the whole traffic system.

• Research Achievements

Based on 2D-LuGre tire model and Unscented Kalman Filtering, the real-time estimation of tire-road adhesion coefficient under three driving conditions of braking, acceleration and steering is realized. The observable condition of the established tire system is proved, and the theoretical boundary of dynamic-based estimation method of tire-road adhesion coefficient is put forward. Through the co-simulation of CarSim-MATLAB and Simulink, the effectiveness of the proposed method is preliminarily verified.

• Leading Research Member: LIN Xuewu

1. Research on Intelligent Vehicle Environment Perception Technology Based on Multi-sensor Fusion

• Research Background

Environmental perception is essential in the field of autonomous driving. Due to the advantages and disadvantages of different sensors, multi-sensor fusion has become a hot and challenging research topic in environmental perception. The research on multi-sensor fusion perception method will provide technical support for the construction of intelligent transportation and the development of automatic driving, and will play a significant and positive role in advancing the intelligent development of automobile industry.

• Research Achievements

In order to tackle the difficulty of moving vehicle detection caused by the incomplete target contour in remote sparse point cloud, a moving vehicle detection algorithm based on PE-CPD is proposed. In this approach, the moving targets in the scenario are filtered by the adaptive threshold which changes with the distance of the target and the resolution of the virtual scanning mapping angle. Then, the pose of the target is estimated by using the point cloud registration information combined with the process of motion state calibration and model selection. And it is introduced into the process of moving vehicle detection. Experimental results show that the proposed method performs well in moving vehicle detection, especially in the remote sparse point cloud scenario.

To solve the problem of inaccurate ground segmentation in the case of undulating ground, occlusion of obstacles and sparse point cloud, a point cloud ground modeling algorithm based on hybrid regression technology is initiated. RLWR and gradient filter are used to establish the ground seed model along the scanning radial direction, to obtain the overall ground seed skeleton. Based on this, GPR is used to conduct the ground autonomous modeling along the scanning circumferential direction, and the ground circumferential continuity is used to make up for the modeling error caused by the lack of radial point cloud. The method uses the whole plane continuity to predict the height of the ground point cloud, which improves the robustness of the algorithm to the environment.

At the same time, a pedestrian detection algorithm based on single template matching is proposed. KDE pedestrian clustering algorithm based on hierarchical segmentation and multi-level fusion is used to solve the problem of under-segmentation of neighboring pedestrians. Secondly, the projection map of potential pedestrian clustering is constructed by main plane projection, from which the LARK feature matrix reflecting the target contour is extracted, and the similarity with the template is calculated to determine the target type. The algorithm avoids the need for a large number of training data, and can effectively distinguish pedestrians from other similar objects in the scenario.

• Leading Research Member: LIU Kaiqi
• Representative Papers

1. Liu K, Wang W, Tharmarasa R, et al. Dynamic vehicle detection with sparse point clouds based on PE-CPD[J]. IEEE Transactions on Intelligent Transportation Systems, 2018, 20(5): 1964-1977.
2. Liu K, Wang W, Wang J. Pedestrian Detection with Lidar Point Clouds Based on Single Template Matching[J]. Electronics, 2019, 8(7): 780.
3. Liu K, Wang W, Tharmarasa R, et al. Ground surface filtering of 3D point clouds based on hybrid regression technique[J]. IEEE Access, 2019, 7: 23270-23284.
4. Liu K, Wang J. Fast dynamic vehicle detection in road scenarios based on pose estimation with Convex-Hull model[J]. Sensors, 2019, 19(14): 3136.
5. Liu K, Wang W. Pedestrian detection on the slope using multi-layer laser scanner[C]//2017 20th International Conference on Information Fusion (Fusion). IEEE, 2017: 1-7.
6. Liu K, Wang W, Sun Z. Recognition of SAR image based on combined templates[C]//2013 IEEE International Conference on Imaging Systems and Techniques (IST). IEEE, 2013: 284-287.

2. Optimal Sensor Selection and Deployment Planning Based on Multi-objective Optimization

• Research Background

Due to the advantages of high computing efficiency and wide sensing range, Vehicle-Road-Cloud collaborative fusion sensing is an inevitable trend in the development of intelligent connected vehicle. Traditional researches on sensor deployment optimization often only optimize for a single index and variable and lack a reasonable road network model. Therefore, the abstract modeling of multi-sensor perception systems and urban roads, together with sensor-deployment-multi-objective optimization algorithms for perception system, have positive and important significance for the improvement of intelligent connected vehicle perception systems and the full use of resources.

• Leading Research Member: LU Zhengye

3. Target Positioning and Tracking Based on Multi-sensor Information Fusion

• Research Background

Intelligent connected vehicles rely on the data of sensors. In practical applications, different sensors have different working principles and application characteristics, which cannot be accurately applied in some scenarios. Therefore, multi-sensor fusion has become a consensus. The application of sensors in various information environments, to resist various types of noise interference, accurately identify targets, fuse asynchronous heterogeneous information , and obtain invariant features, is the focus of multi-sensor fusion research with great significance.

• Research Achievements

In order to study how to reduce noise interference in multi-sensor fusion perception, improve the accuracy of positioning and tracking of targets after multi-sensor fusion, we need to analyze the source of the noise,improve the fusion algorithm for the noise of the sensor's original data and the noise caused by the unstable reference of the vehicle, and then reduce the positioning and tracking error by comparing the improved algorithm with the unimproved algorithm.

• Leading Research Member: ZHU Shihao

Autonomous vehicles play a positive role in traffic safety, traffic efficiency, environmental protection and energy-saving. With the development of intelligent vehicles, how to interact with these dynamic, random and diversified traffic participants is a potential challenge to realize high-level automatic driving in mixed traffic. Therefore, research on human-like decision-making system of autonomous driving is necessary. However, there remains some challenges as follows: 1) how to identify the intentions of surrounding traffic participants, instead of passively accelerating or braking according to the current state of the surrounding vehicles; 2) how to quantify the driving risks caused by different traffic factors in complex environment; 3) how to plan specific driving strategies and improve the decision-making ability in complex environments.

1. Pedestrian Trajectory Prediction Based on Intention and Behavior Identification

• Research Background

Pedestrian trajectory prediction plays an important role in vehicle anti-collision protection system and automatic driving. At present, the researches of pedestrian trajectory prediction are mainly divided into three categories, which are based on motion models, pedestrian intention identification and learning methods. However, these methods always neglect pedestrian behaviors, and the motion models are not adaptive to pedestrian's unique characteristics. Combined with the complex behaviors of pedestrian, a new pedestrian motion model is proposed to identify the pedestrian's intention, and finally integrate the behavior and intention to obtain good prediction results, which is of great significance to improve the safety of automatic driving.

• Research Achievements

To get accurate prediction results, a pedestrian trajectory prediction method based on pedestrian's behavior and intention is proposed. Pedestrian’s behaviors are identified into standing, walking and running, through 18 captured key points. The result shows its adaptability for general pedestrian crossing scenarios. DBN is used to obtain pedestrian’s crossing intentions with all parameters obtained from empirical data and announced data for general crossing scenarios. The trajectory prediction method combines the behavior and the intention results together and performs well in defined 8 typical pedestrian crossing scenarios in our provided BPI dataset, especially for stopping scenarios. The results show that the method can accurately predict the pedestrian trajectory for general pedestrian crossing scenarios without training of pedestrian trajectory.

• Leading research member: WU Haoran