On the first passage g-mean-variance optimality for discounted continuous-time Markov decision processes

Guo, X, Huang, X and Zhang, Y ORCID: 0000-0002-3200-6306 (2015) On the first passage g-mean-variance optimality for discounted continuous-time Markov decision processes SIAM Journal on Control and Optimization, 53 (3). pp. 1406-1424. ISSN 0363-0129, 1095-7138

Text SICON-096887R.pdf - Author Accepted Manuscript Download (168kB)

Abstract

This paper considers the discounted continuous-time Markov decision processes (MDPs) in Borel spaces and with unbounded transition rates. The discount factors are allowed to depend on states and actions. Main attention is concentrated on the set F<inf>g</inf> of stationary policies attaining a given mean performance g up to the first passage of the continuous-time MDP to an arbitrarily fixed target set. Under suitable conditions, we prove the existence of a g-mean-variance optimal policy that minimizes the first passage variance over the set F<inf>g</inf> using a transformation technique, and also give the value iteration and policy iteration algorithms for computing the g-variance value function and a g-mean-variance optimal policy, respectively. Two examples are analytically solved to demonstrate the application of our results.

Item Type:	Article
Uncontrolled Keywords:	continuous-time Markov decision processes, state-action-dependent discount factors, first passage mean-optimality, first passage g-mean-based variance minimization
Depositing User:	Symplectic Admin
Date Deposited:	01 Jul 2016 15:22
Last Modified:	24 Jan 2026 00:16
DOI:	10.1137/140968872
Related Websites:	Author Publisher
URI:	https://livrepository.liverpool.ac.uk/id/eprint/3001985
Disclaimer:	The University of Liverpool is not responsible for content contained on other websites from links within repository metadata. Please contact us if you notice anything that appears incorrect or inappropriate.