Market Making with Learned Beta Policies

Wang, Yongzhao, Savani, Rahul ORCID: 0000-0003-1262-7831, Gu, Anri, Mascioli, Chris, Turocy, Theodore and Wellman, Michael (2024) Market Making with Learned Beta Policies In: Proceedings of the 5th ACM International Conference on AI in Finance.

Access the full-text of this item by clicking on the Open Access link.

Abstract

In market making, a market maker (MM) can concurrently place many buy and sell limit orders at various prices and volumes, resulting in a vast action space. To handle this large action space, beta policies were introduced, utilizing a scaled beta distribution to concisely represent the volume distribution of an MM's orders across different price levels. However, in these policies, the parameters of the scaled beta distributions are either fixed or adjusted only according to predefined rules based on the MM's inventory. As we show, this approach potentially limits the effectiveness of market-making policies and overlooks the significance of other market characteristics in a dynamic market. To address this limitation, we introduce a general adaptive MM based on beta policies by employing deep reinforcement learning (RL) to dynamically control the scaled beta distribution parameters and generate orders based on current market conditions. A sophisticated market simulator is employed to evaluate a wide range of existing market-making policies and to train the RL policy in markets with varying levels of inventory risk, ensuring a comprehensive assessment of their performance and effectiveness. By carefully designing the reward function and observation features, we demonstrate that our RL beta policy outperforms baseline policies across multiple metrics in different market settings. We emphasize the strong adaptability of the learned RL beta policy, underscoring its pivotal role in achieving superior performance compared to other market-making policies.

Item Type:	Conference Item (Unspecified)
Uncontrolled Keywords:	Market Making, Deep Reinforcement Learning, Beta Policy
Divisions:	Faculty of Science & Engineering Faculty of Science & Engineering > School of Electrical Engineering, Electronics and Computer Science
Depositing User:	Symplectic Admin
Date Deposited:	18 Nov 2024 10:34
Last Modified:	23 May 2026 09:31
DOI:	10.1145/3677052.3698623
Open Access URL:	https://dl.acm.org/doi/pdf/10.1145/3677052.3698623
Related Websites:	Author Publisher
URI:	https://livrepository.liverpool.ac.uk/id/eprint/3188063
Disclaimer:	The University of Liverpool is not responsible for content contained on other websites from links within repository metadata. Please contact us if you notice anything that appears incorrect or inappropriate.