Impulsive control for continuous-time Markov decision processes

Dufour, François and Piunovskiy, Alexey (2015) Impulsive control for continuous-time Markov decision processes. Advances in Applied Probability, 47 (01). pp. 106-127.

Text ProbTrust13shortened.pdf - Unspecified Access to this file is embargoed until Unspecified. Download (415kB)

Abstract

In this paper our objective is to study continuous-time Markov decision processes on a general Borel state space with both impulsive and continuous controls for the infinite time horizon discounted cost. The continuous-time controlled process is shown to be nonexplosive under appropriate hypotheses. The so-called Bellman equation associated to this control problem is studied. Sufficient conditions ensuring the existence and the uniqueness of a bounded measurable solution to this optimality equation are provided. Moreover, it is shown that the value function of the optimization problem under consideration satisfies this optimality equation. Sufficient conditions are also presented to ensure on the one hand the existence of an optimal control strategy, and on the other hand the existence of a ε-optimal control strategy. The decomposition of the state space into two disjoint subsets is exhibited where, roughly speaking, one should apply a gradual action or an impulsive action correspondingly to obtain an optimal or ε-optimal strategy. An interesting consequence of our previous results is as follows: the set of strategies that allow interventions at time t = 0 and only immediately after natural jumps is a sufficient set for the control problem under consideration.

Item Type:	Article
Uncontrolled Keywords:	Impulsive control, continuous control, continuous-time Markov decision process, discounted cost
Depositing User:	Symplectic Admin
Date Deposited:	10 Feb 2015 10:19
Last Modified:	15 Dec 2022 16:14
DOI:	10.1239/aap/1427814583
Related URLs:	Author Publisher
URI:	https://livrepository.liverpool.ac.uk/id/eprint/2005306