BEST PRACTICE
PUBLICATIONS
AVSC Best Practice for Metrics and Methods for Assessing Safety Performance of Automated Driving Systems (ADS)
AVSC0006202103
RATIONALE & INTRODUCTION
In order to garner public trust and confidence, automated driving systems (ADS) must demonstrate and measure safety performance in a manner that is credible, practicable, and consistent. Though several approaches to safety performance measurement have been put forward, none have been broadly adopted by the larger industry. This AVSC best practice recommends a practical set of ADS safety metrics which enable foundational system-level analysis for developers and manufacturers, as well as support emerging ADS standards. Performance-based and technology neutral, they work towards the larger goals of safe automated vehicle deployments and improved communication among developers and the general public.
Measuring safety outcomes in any field is a complex task that requires considerable time and exposure to achieve statistical significance. Because of this, ADS safety assessment must utilize predictive indicators of performance. Prior to and during commercial deployment, metrics will, and should, be used to track safety performance and inform risk management.
Consistency in the type of safety performance metrics used — across the industry, across operational design domains (ODDs), and across the product development lifecycle — facilitates discourse on many important ADS deployment-related topics. To inform their decision-making process, developers and manufacturers measure many elements associated with safety during development and testing. These recommended foundational metrics grounds communication among stakeholders, with the goal of accelerating industry-wide continuous improvement in safety performance.
RATIONALE & INTRODUCTION
In order to garner public trust and confidence, automated driving systems (ADS) must demonstrate and measure safety performance in a manner that is credible, practicable, and consistent. Though several approaches to safety performance measurement have been put forward, none have been broadly adopted by the larger industry. This AVSC best practice recommends a practical set of ADS safety metrics which enable foundational system-level analysis for developers and manufacturers, as well as support emerging ADS standards. Performance-based and technology neutral, they work towards the larger goals of safe automated vehicle deployments and improved communication among developers and the general public.
Measuring safety outcomes in any field is a complex task that requires considerable time and exposure to achieve statistical significance. Because of this, ADS safety assessment must utilize predictive indicators of performance. Prior to and during commercial deployment, metrics will, and should, be used to track safety performance and inform risk management.
Consistency in the type of safety performance metrics used — across the industry, across operational design domains (ODDs), and across the product development lifecycle — facilitates discourse on many important ADS deployment-related topics. To inform their decision-making process, developers and manufacturers measure many elements associated with safety during development and testing. These recommended foundational metrics grounds communication among stakeholders, with the goal of accelerating industry-wide continuous improvement in safety performance.
SCOPE
This AVSC Best Practice for Metrics and Methods for Assessing Safety Performance of Automated Driving Systems (ADS) (AVSC00006202103) recommends a set of metrics that may be used to assess ADS safety performance of the dynamic driving task (DDT). These metrics and methods are principally designed to provide evidence of safety performance for a manufacturer’s decision to deploy (and monitor) fleet-operated/managed SAE level 4 and 5 ADS-dedicated vehicles (ride-hailing or product delivery).
This document lays out a performance-based, technology-neutral approach for measuring and analyzing safety performance. It supports long-term, socially-important safety goals (like reducing crashes). ADS safety performance metrics in this document support system-level analyses, i.e. they are practical to implement for any system regardless of architecture.
The metrics and methods provided in this document are intended for use by the technical community (developers, manufacturers, testers, etc.) to aid in the safe development and deployment of ADS. They may also be useful to stakeholders who have interest in better understanding the safety posture of ADS deployments.
