SCC-rFMQ Learning in Cooperative Markov Games with Continuous Actions



Zhang, Chengwei, Li, Xiaohong, Hao, Jianye, Chen, Siqi, Tuyls, Karl, Feng, Zhiyong, Xue, Wanli and Chen, Rong
(2018) SCC-rFMQ Learning in Cooperative Markov Games with Continuous Actions. In: AAMAS 2018, 2018-7-10 - 2018-7-15, Stockholm, Sweden.

Access the full-text of this item by clicking on the Open Access link.

Abstract

Although many reinforcement learning methods have been proposed for learning the optimal solutions in single-agent continuous-action domains, multiagent coordination domains with continuous actions have received relatively few investigations. In this paper, we propose an independent learner hierarchical method, named Sample Continuous Coordination with recursive Frequency Maximum Q-Value (SCC-rFMQ), which divides the cooperative problem with continuous actions into two layers. The first layer samples a finite set of actions from the continuous action spaces by a re-sampling mechanism with variable exploratory rates, and the second layer evaluates the actions in the sampled action set and updates the policy using a reinforcement learning cooperative method. By constructing cooperative mechanisms at both levels, SCC-rFMQ can handle cooperative problems in continuous action cooperative Markov games effectively. The effectiveness of SCC-rFMQ is experimentally demonstrated on two well-designed games, i.e., a continuous version of the climbing game and a cooperative version of the boat problem. Experimental results show that SCC-rFMQ outperforms other reinforcement learning algorithms.

Item Type: Conference or Workshop Item (Unspecified)
Uncontrolled Keywords: cs.AI, cs.AI
Depositing User: Symplectic Admin
Date Deposited: 07 Mar 2019 11:00
Last Modified: 19 Jan 2023 00:57
Open Access URL: http://ifaamas.org/Proceedings/aamas2018/pdfs/p216...
Related URLs:
URI: https://livrepository.liverpool.ac.uk/id/eprint/3033889