Performance Evaluation of an Autonomous Vehicle Using Resilience Engineering
2022-01-0069
03/29/2022
- Event
- Content
- Standard operation of autonomous vehicles on public roads results in significant exposure to high levels of risk. There is a significant need to develop metrics that evaluate safety of an automated system without reliance on the rate of vehicle accidents and fatalities compared to the number of miles driven; a proactive rather than a reactive metric is needed. Resilience engineering is a new paradigm for safety management that focuses on evaluating complex systems and their interaction with the environment. This paper presents the overall methodology of resilience engineering and the resilience assessment grid (RAG) as an evaluation tool to measure autonomous systems' resilience. This assessment tool was used to evaluate the path tracking capabilities of an autonomous vehicle and measure the ability of the control subsystem to respond. A Pure Pursuit controller was developed and utilized as the path tracking control algorithm, and the Carla simulator was used to implement the algorithm and develop the testing environment for this methodology. The path tracking control algorithm was tested at different speeds and evaluated using RAG. The results first show that, although no crashes were observed, the higher speed vehicle demonstrated lower overall resilience and tells us the algorithm is less susceptible to overcome disturbances. We conclude that this metric can be successfully used to proactively evaluate the safety of automated vehicle subsystems or the system's overall performance. In future work, we plan on expanding our evaluation to include commercially available products such as SuperCruise, BlueCruise, and the Full Self-Driving product.
- Citation
- Fanas Rojas, J., Brown, N., Rupp, J., Bradley, T. et al., "Performance Evaluation of an Autonomous Vehicle Using Resilience Engineering," SAE Technical Paper 2022-01-0069, 2022, .