A Neural Network-based Multisensor Data Fusion Approach for Enabling Situational Awareness of Vehicles

Abstract

With the growing number of research studies on Vehicle-to-Vehicle (V2V) communication applications, situational awareness becomes one of major challenges for autonomous vehicles. Autonomous vehicle needs to predict the movement and trajectories of surrounding vehicles accurately in order to make a better decision making. The ability to recognize vehicles surroundings has become important in order to enable situational awareness and navigate the vehicle safely. In this paper, we propose a neural network called Mapping Decision Feedback Neural Network (MDFNN) to tackle the vehicle identification (VID) issue in V2V communication. According to the MDFNN infrastructure, two types of MDFNN namely as Grid-based MDFNN and Bounding box-based MDFNN are proposed. The MDFNN fuses image, V2V interface, GPS, magnetometer, and speedometer data (i.e., multi-sensor data and V2V communication) to enable situational awareness. MDFNN utilizes the mapping decision feedback information in the proposed deep learning neural network structure. With this improvement, a greatly improved accuracy can help to resolve the VID issue. Our experiments result shows 85% of accuracy for Grid-based MDFNN.