Aspects to consider while defining acceptance criteria and validation target
2022-01-0072
03/29/2022
- Event
- Content
- Two major steps involved in SOTIF analysis are defining acceptance criteria and estimating the validation target. While acceptance criteria aids in determining if we have an acceptable residual risk corresponding to a hazardous behavior, the validation target specifies the amount of testing effort (in hours or representative miles) that is needed to ensure that the acceptance criteria are met. The existing approaches for defining acceptance criteria heavily rely on existing fatality databases or naturalistic driving study data sets. The criterion is selected based on average number of fatalities or crashes per mile or per an hour of operation. The validation target is calculated based on these acceptance criteria. However, are these values really reflecting the acceptable risk criteria and targets? According to statistics, for a given data set and a random sample derived from the dataset, only the mean of population of the data set and the sample can considered to be equal. In the case of autonomous vehicles, this implies we can only generalize the mean of crash statistics of all vehicles in a country to a mean of crash statistics of a sample fleet of vehicles. However, the current acceptance criteria consider arithmetic mean across states but not vehicles, and often does not take into account the operational design domain (ODD) factors, the driver characteristics, and justifications behind their selection. Similarly, the validation target is defined from the acceptance criteria but no minimal requirements that need to be met within the target with respect to scenarios are mentioned. To overcome these limitations, in this paper, we discuss the various aspects we need to consider for defining acceptance criteria and validation target. We illustrate with an example how the results can vary drastically when the aspects we defined are considered and compare our results with the existing approaches.
- Citation
- Madala, K., Krishnamoorthy, J., Avalos Gonzalez, C., SHIVKUMAR, A. et al., "Aspects to consider while defining acceptance criteria and validation target," SAE Technical Paper 2022-01-0072, 2022, .