From Deterministic to Stochastic: An Interpretable Stochastic Model-free Reinforcement Learning Framework for Portfolio Optimization

Coenen, Frans ORCID: 0000-0003-1026-6649, Wang, Yining, Song, Zitao, Qian, Pin, Song, Sifan, Jiang, Zhengyong and Su, Jionglong (2022) From Deterministic to Stochastic: An Interpretable Stochastic Model-free Reinforcement Learning Framework for Portfolio Optimization. Applied Intelligence, 53 (12). pp. 15188-15203.

Text
SN_SPDQ_Revise_LP.pdf - Author Accepted Manuscript
Download (751kB) | Preview

Official URL: http://dx.doi.org/10.1007/s10489-022-04217-5

Abstract

As a fundamental problem in algorithmic trading, portfolio optimization aims to maximize the cumulative return by continuously investing in various financial derivatives within a given time period. Recent years have witnessed the transformation from traditional machine learning trading algorithms to reinforcement learning algorithms due to their superior nature of sequential decision making. However, the exponential growth of the imperfect and noisy financial data that is supposedly leveraged by the deterministic strategy in reinforcement learning, makes it increasingly challenging for one to continuously obtain a profitable portfolio. Thus, in this work, we first reconstruct several deterministic and stochastic reinforcement algorithms as benchmarks. On this basis, we introduce a risk-aware reward function to balance the risk and return. Importantly, we propose a novel interpretable stochastic reinforcement learning framework which tailors a stochastic policy parameterized by Gaussian Mixtures and a distributional critic realized by quantiles for the problem of portfolio optimization. In our experiment, the proposed algorithm demonstrates its superior performance on U.S. market stocks with a 63.1% annual rate of return while at the same time reducing the market value max drawdown by 10% when back-testing during the stock market crash around March 2020.

Item Type:	Article
Uncontrolled Keywords:	Portfolio management, Reinforcement learning, Deep learning, Quantitative finance
Divisions:	Faculty of Science and Engineering > School of Electrical Engineering, Electronics and Computer Science
Depositing User:	Symplectic Admin
Date Deposited:	16 Mar 2023 08:28
Last Modified:	11 Nov 2023 02:30
DOI:	10.1007/s10489-022-04217-5
Related URLs:	Author Publisher
URI:	https://livrepository.liverpool.ac.uk/id/eprint/3169124