ON THE FIRST PASSAGE <i>g</i>-MEAN-VARIANCE OPTIMALITY FOR DISCOUNTED CONTINUOUS-TIME MARKOV DECISION PROCESSES



Guo, Xianping, Huang, Xiangxiang and Zhang, Yi ORCID: 0000-0002-3200-6306
(2015) ON THE FIRST PASSAGE <i>g</i>-MEAN-VARIANCE OPTIMALITY FOR DISCOUNTED CONTINUOUS-TIME MARKOV DECISION PROCESSES. SIAM JOURNAL ON CONTROL AND OPTIMIZATION, 53 (3). pp. 1406-1424.

[img] Text
SICON-096887R.pdf - Author Accepted Manuscript

Download (168kB)

Abstract

This paper considers the discounted continuous-time Markov decision processes (MDPs) in Borel spaces and with unbounded transition rates. The discount factors are allowed to depend on states and actions. Main attention is concentrated on the set F<inf>g</inf> of stationary policies attaining a given mean performance g up to the first passage of the continuous-time MDP to an arbitrarily fixed target set. Under suitable conditions, we prove the existence of a g-mean-variance optimal policy that minimizes the first passage variance over the set F<inf>g</inf> using a transformation technique, and also give the value iteration and policy iteration algorithms for computing the g-variance value function and a g-mean-variance optimal policy, respectively. Two examples are analytically solved to demonstrate the application of our results.

Item Type: Article
Uncontrolled Keywords: continuous-time Markov decision processes, state-action-dependent discount factors, first passage mean-optimality, first passage g-mean-based variance minimization
Depositing User: Symplectic Admin
Date Deposited: 01 Jul 2016 15:22
Last Modified: 15 Mar 2024 11:13
DOI: 10.1137/140968872
Related URLs:
URI: https://livrepository.liverpool.ac.uk/id/eprint/3001985