Real-time Detection and Avoidance of Obstacles in the Path of Autonomous Vehicles using Monocular RGB Camera
2022-01-0080
03/29/2022
- Event
- Content
- Collision avoidance is one of the most critical functions of an autonomous vehicle. This requires a highly accurate and low latency perception of the environment using in-vehicle sensors. Nevertheless, due to the uncertainty and diversity in object attributes and dynamic backgrounds in on-road scenarios, obstacle detection becomes a challenging problem. Moreover, high-quality data that represents this diversity can be difficult to acquire. In addition, processing complex perception algorithms in low compute embedded hardware is an ongoing research problem. In this paper, we present an end-to-end real-time detection and collision avoidance framework in an autonomous vehicle using a monocular RGB camera. The proposed system is able to run on embedded hardware in the vehicle to perform real-time detection of small objects. RetinaNet architecture with ResNet50 backbone is used to develop the object detection model using RGB images. A quantized version of the object detection inference model is implemented in the vehicle using NVIDIA Jetson AGX Xavier. A geometric method is used to estimate the distance to the detected object which is forwarded to a MicroAutoBox device that implements the control system of the vehicle and is responsible for maneuvering around the detected objects. The pipeline is implemented on a passenger vehicle and demonstrated using different obstacles on a predefined set of waypoints. Our results show that the system is capable of detecting objects that appear in an image area as small as 20x30 pixels in a 1280x720 image and can run at a speed of 24 frames per second (FPS) on the embedded device in the vehicle. A data analyzer is also employed to visualize the real-time performance of the system.
- Citation
- Mallik, A., Gaopande, M., Singh, G., Ravindran, A. et al., "Real-time Detection and Avoidance of Obstacles in the Path of Autonomous Vehicles using Monocular RGB Camera," SAE Technical Paper 2022-01-0080, 2022, .