Improved Deep Reinforcement Learning for Robotics Through Distribution-based Experience Retention



de Bruin, T, Kober, J, Tuyls, K and Babushka, R
(2016) Improved Deep Reinforcement Learning for Robotics Through Distribution-based Experience Retention. In: The 2016 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS 2016), 2016-10-9 - 2016-10-14, Daejeon, Korea.

[img] Text
improved-deep-reinforcement (6).pdf - Author Accepted Manuscript

Download (3MB)

Abstract

Recent years have seen a growing interest in the use of deep neural networks as function approximators in reinforcement learning. In this paper, an experience replay method is proposed that ensures that the distribution of the experiences used for training is between that of the policy and a uniform distribution. Through experiments on a magnetic manipulation task it is shown that the method reduces the need for sustained exhaustive exploration during learning. This makes it attractive in scenarios where sustained exploration is in-feasible or undesirable, such as for physical systems like robots and for life long learning. The method is also shown to improve the generalization performance of the trained policy, which can make it attractive for transfer learning. Finally, for small experience databases the method performs favorably when compared to the recently proposed alternative of using the temporal difference error to determine the experience sample distribution, which makes it an attractive option for robots with limited memory capacity.

Item Type: Conference or Workshop Item (Unspecified)
Depositing User: Symplectic Admin
Date Deposited: 11 Jul 2016 13:45
Last Modified: 19 Jan 2023 07:34
DOI: 10.1109/iros.2016.7759581
Related URLs:
URI: https://livrepository.liverpool.ac.uk/id/eprint/3002247