Reachability Verification Based Reliability Assessment for Deep Reinforcement Learning Controlled Robotics and Autonomous Systems

Dong, Yi ORCID: 0000-0003-3047-7777, Zhao, Xingyu ORCID: 0000-0002-3474-349X, Wang, Sen and Huang, Xiaowei ORCID: 0000-0001-6267-0366 (2024) Reachability Verification Based Reliability Assessment for Deep Reinforcement Learning Controlled Robotics and Autonomous Systems IEEE Robotics and Automation Letters, 9 (4). pp. 1-8. ISSN 2377-3766, 2377-3766

Text RAL.pdf - Author Accepted Manuscript Available under License Creative Commons Attribution. Download (3MB) | Preview

Abstract

Deep Reinforcement Learning (DRL) has achieved impressive performance in robotics and autonomous systems (RAS). A key challenge to its deployment in real-life operations is the presence of spuriously unsafe DRL policies. Unexplored states may lead the agent to make wrong decisions that could result in hazards, especially in applications where DRL-trained end-to-end controllers govern the behaviour of RAS. This letter proposes a novel quantitative reliability assessment framework for DRL-controlled RAS, leveraging verification evidence generated from formal reliability analysis of neural networks. A two-level verification framework is introduced to check the safety property with respect to inaccurate observations that are due to, e.g., environmental noise and state changes. Reachability verification tools are leveraged locally to generate safety evidence of trajectories. In contrast, at the global level, we quantify the overall reliability as an aggregated metric of local safety evidence, corresponding to a set of distinct tasks and their occurrence probabilities. The effectiveness of the proposed verification framework is demonstrated and validated via experiments on real RAS.

Item Type:	Article
Uncontrolled Keywords:	Safety, Power system reliability, Trajectory, Reachability analysis, Software reliability, Robots, Reinforcement learning, Robot safety, formal methods in robotics and automation, AI-enabled robotics
Divisions:	Faculty of Science & Engineering > School of Electrical Engineering, Electronics and Computer Science
Depositing User:	Symplectic Admin
Date Deposited:	26 Feb 2024 10:48
Last Modified:	28 Feb 2026 03:15
DOI:	10.1109/lra.2024.3364471
Related Websites:	Author Publisher
URI:	https://livrepository.liverpool.ac.uk/id/eprint/3178886
Disclaimer:	The University of Liverpool is not responsible for content contained on other websites from links within repository metadata. Please contact us if you notice anything that appears incorrect or inappropriate.