LiDAR and Camera-based Convolutional Neural Network Detection for Autonomous Driving

Autonomous vehicles are currently a subject of great interest and there is heavy research on creating and improving algorithms for detecting objects in their vicinity. Object classification and detection are crucial tasks that need to be solved accurately and robustly in order to achieve higher automation levels. Current approaches for classification and detection use either cameras or light detection and ranging (LiDAR) sensors. Cameras can work at high frame-rate, and provide dense information over a long range under good illumination and fair weather. LiDARs scan the environment by using their own emitted pulses of laser light and they are only marginally affected by the external lighting conditions. LiDARs provide accurate distance measurements. However, they have a limited range, typically between 10 and 100 m, and provide sparse data. A ROS-based deep learning approach has been developed to detect objects using point cloud data. With encoded raw camera and LiDAR data, several basic statistics such as elevation and density are generated. The system leverages simple and fast convolutional neural network (CNN) solution for object classification and generation of a bounding box to detect vehicles, pedestrians and cyclists. The system is implemented on an Nvidia Jetson TX2 embedded computing platform, the classification and location of the objects are determined by the neural network. Coordinates and other properties of the object are published to various ROS topics which are then serviced by visualization and data handling routines. Performance of the system is scrutinized with regards to hardware capability, software reliability, and real-time performance. The final product is a mobile-platform capable of identifying pedestrians, cars, trucks and cyclists.