RATIONALE & INTRODUCTION
 

Successful scaling of automated driving system (ADS) technology and realization of its full potential will require developers and service providers to continuously monitor performance of their fleet and enact prompt improvements should issues arise. Additionally, the safety performance of automated driving system-dedicated vehicles (ADS-DVs) is expected to evolve over time. Continuous monitoring of ADS-DVs makes it possible to identify and address new risks related to changes in the environment. This also means the validity of operational design domain (ODD) related assumptions is continually assessed. 

Large volumes of operational data produced by ADS-DVs during deployment can be analyzed to identify patterns as well as anomalies. These results can be indicative of actual or potential performance changes. In the future, iterative improvements to automated driving system (ADS) performance may occur, deployed as software updates—over-the-air (OTA) or otherwise—or hardware updates that have been enabled by new data or technology advancements. 
It is expected that ADS-DVs will encounter scenarios and situations that occasionally challenge its perception or control algorithms. These types of events may be due to changes in the deployed operating environment of the ADS-DV that were unknown or unaccounted for at the time the operational design domain (ODD) was developed and tested. They may also be due to incorrect assumptions about the ADS’s performance in such situations. Capturing and analyzing performance data from such scenarios enables ADS developers to make updates to their fleet(s).

This best practice complements other AVSC best practices that provide metrics and methods which can be used to monitor safety [AVSC00006202103, AVSC00008202111] while considering important factors pertaining to how data is collected, analyzed, and used [AVSC00004202009]. Applying the methods outline in this best practice to deployed ADS-DVs will drive continuous improvement in ADS safety performance, as well as promote public trust.

RATIONALE & INTRODUCTION
 

Successful scaling of automated driving system (ADS) technology and realization of its full potential will require developers and service providers to continuously monitor performance of their fleet and enact prompt improvements should issues arise. Additionally, the safety performance of automated driving system-dedicated vehicles (ADS-DVs) is expected to evolve over time. Continuous monitoring of ADS-DVs makes it possible to identify and address new risks related to changes in the environment. This also means the validity of operational design domain (ODD) related assumptions is continually assessed. 

Large volumes of operational data produced by ADS-DVs during deployment can be analyzed to identify patterns as well as anomalies. These results can be indicative of actual or potential performance changes. In the future, iterative improvements to automated driving system (ADS) performance may occur, deployed as software updates—over-the-air (OTA) or otherwise—or hardware updates that have been enabled by new data or technology advancements. 

It is expected that ADS-DVs will encounter scenarios and situations that occasionally challenge its perception or control algorithms. These types of events may be due to changes in the deployed operating environment of the ADS-DV that were unknown or unaccounted for at the time the operational design domain (ODD) was developed and tested. They may also be due to incorrect assumptions about the ADS’s performance in such situations. Capturing and analyzing performance data from such scenarios enables ADS developers to make updates to their fleet(s).

This best practice complements other AVSC best practices that provide metrics and methods which can be used to monitor safety [AVSC00006202103, AVSC00008202111] while considering important factors pertaining to how data is collected, analyzed, and used [AVSC00004202009]. Applying the methods outline in this best practice to deployed ADS-DVs will drive continuous improvement in ADS safety performance, as well as promote public trust.

SCOPE 

This Automated Vehicle Safety Consortium™ (AVSC) best practice describes an approach for continuous monitoring of the safety metrics identified in AVSC00006202103. In addition, this best practice describes data sources and methods that can be used to identify safety-relevant information. The insights gained about the operating environment can support manufacturers or fleet operator’s continuous improvement of ADS-DV safety performance post deployment. 

The described approach enables continuous improvement of safety performance by identifying unknowns and regularly evaluating the validity of assumptions made about the ADS-DVs’ operating environment. It also references change risk management approaches [AVSC00010202304] and describes approaches to analyzing and segmenting data according to intended behavioral competencies [AVSC00008202111]