Dependability Analysis of Deep Reinforcement Learning based Robotics and Autonomous Systems through Probabilistic Model Checking

Dong, Yi ORCID: 0000-0003-3047-7777, Zhao, Xingyu ORCID: 0000-0002-3474-349X and Huang, Xiaowei ORCID: 0000-0001-6267-0366 (2021) Dependability Analysis of Deep Reinforcement Learning based Robotics and Autonomous Systems through Probabilistic Model Checking In: IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS 2022), 2022-10-24 - ?, Kyoto, Japan.

Text IROS2022.pdf - Author Accepted Manuscript Download (11MB) | Preview

Abstract

While Deep Reinforcement Learning (DRL) provides transformational capabilities to the control of Robotics and Autonomous Systems (RAS), the black-box nature of DRL and uncertain deployment environments of RAS pose new challenges on its dependability. Although existing works impose constraints on the DRL policy to ensure successful completion of the mission, it is far from adequate to assess the DRL-driven RAS in a holistic way considering all dependability properties. In this paper, we formally define a set of dependability properties in temporal logic and construct a Discrete-Time Markov Chain (DTMC) to model the dynamics of risk/failures of a DRL-driven RAS interacting with the stochastic environment. We then conduct Probabilistic Model Checking (PMC) on the designed DTMC to verify those properties. Our experimental results show that the proposed method is effective as a holistic assessment framework while uncovering conflicts between the properties that may need trade-offs in training. Moreover, we find that the standard DRL training cannot improve dependability properties, thus requiring bespoke optimisation objectives. Finally, our method offers sensitivity analysis of dependability properties to disturbance levels from environments, providing insights for the assurance of real RAS.

Item Type:	Conference Item (Unspecified)
Additional Information:	Accepted by IROS2022
Uncontrolled Keywords:	cs.RO, cs.RO, cs.AI, cs.SE
Divisions:	Faculty of Science & Engineering > School of Electrical Engineering, Electronics and Computer Science
Depositing User:	Symplectic Admin
Date Deposited:	30 Jun 2022 14:49
Last Modified:	22 Apr 2026 11:00
Related Websites:	Author Publisher
URI:	https://livrepository.liverpool.ac.uk/id/eprint/3137379
Disclaimer:	The University of Liverpool is not responsible for content contained on other websites from links within repository metadata. Please contact us if you notice anything that appears incorrect or inappropriate.