Discounted continuous-time Markov decision processes with unbounded rates and randomized history-dependent policies: the dynamic programming approach

Piunovskiy, Alexey ORCID: 0000-0002-9683-4856 and Zhang, Yi (2014) Discounted continuous-time Markov decision processes with unbounded rates and randomized history-dependent policies: the dynamic programming approach. 4OR-A QUARTERLY JOURNAL OF OPERATIONS RESEARCH, 12 (1). pp. 49-75.

Text QJOR2013.pdf - Unspecified Access to this file is embargoed until Unspecified. Download (316kB)

Abstract

This paper deals with a continuous-time Markov decision process in Borel state and action spaces and with unbounded transition rates. Under history-dependent policies, the controlled process may not be Markov. The main contribution is that for such non-Markov processes we establish the Dynkin formula, which plays important roles in establishing optimality results for continuous-time Markov decision processes. We further illustrate this by showing, for a discounted continuous-time Markov decision process, the existence of a deterministic stationary optimal policy (out of the class of history-dependent policies) and characterizing the value function through the Bellman equation. © 2013 Springer-Verlag Berlin Heidelberg.

Item Type:	Article
Uncontrolled Keywords:	Bellman equation, Continuous-time Markov decision process, Dynamic programming, Dynkin's formula
Depositing User:	Symplectic Admin
Date Deposited:	10 Feb 2015 10:12
Last Modified:	15 Mar 2024 07:07
DOI:	10.1007/s10288-013-0236-1
Related URLs:	Author
URI:	https://livrepository.liverpool.ac.uk/id/eprint/2005304