Piunovskiy, Alexey 
ORCID: 0000-0002-9683-4856 and Zhang, Yi ORCID: 0000-0002-3200-6306
(2024)
EXTREME OCCUPATION MEASURES IN MARKOV DECISION PROCESSES WITH AN ABSORBING STATE
SIAM JOURNAL ON CONTROL AND OPTIMIZATION, 62 (1).
pp. 65-90.
ISSN 0363-0129, 1095-7138
|
Text
SICONfinalForProof.pdf - Author Accepted Manuscript Available under License Creative Commons Attribution. Download (367kB) | Preview |
Abstract
In this paper, we consider a Markov decision process (MDP) with a Borel state space X U {∆}, where ∆ is an absorbing state (cemetery), and a Borel action space A. We consider the space of finite occupation measures restricted on X X A and the extreme points in it. It is possible that some strategies have infinite occupation measures. Nevertheless, we prove that every finite extreme occupation measure is generated by a deterministic stationary strategy. Then, for this MDP, we consider a constrained problem with total undiscounted criteria and J constraints, where the cost functions are nonnegative. By assumption, the strategies inducing infinite occupation measures are not optimal. Then our second main result is that, under mild conditions, the solution to this constrained MDP is given by a mixture of no more than J + 1 occupation measures generated by deterministic stationary strategies.
| Item Type: | Article |
|---|---|
| Uncontrolled Keywords: | Markov decision process, total cost, occupation measure, mathematical program- ming, extreme point, mixture |
| Divisions: | Faculty of Science & Engineering > School of Physical Sciences |
| Depositing User: | Symplectic Admin |
| Date Deposited: | 11 Jan 2024 08:41 |
| Last Modified: | 04 Jun 2026 08:05 |
| DOI: | 10.1137/23M1572398 |
| Related Websites: | |
| URI: | https://livrepository.liverpool.ac.uk/id/eprint/3177797 |
| Disclaimer: | The University of Liverpool is not responsible for content contained on other websites from links within repository metadata. Please contact us if you notice anything that appears incorrect or inappropriate. |
Altmetric
Altmetric