Piunovskiy, Alexey 
ORCID: 0000-0002-9683-4856 and Zhang, Yi ORCID: 0000-0002-3200-6306
(2024)
Extreme occupation measures in Markov decision processes with an absorbing state.
SIAM Journal on Control and Optimization, 62 (1).
pp. 65-90.
ISSN 0363-0129, 1095-7138
|
Text
SICONfinalForProof.pdf - Author Accepted Manuscript Available under License Creative Commons Attribution. Download (367kB) | Preview |
Abstract
In this paper, we consider a Markov decision process (MDP) with a Borel state space X U {∆}, where ∆ is an absorbing state (cemetery), and a Borel action space A. We consider the space of finite occupation measures restricted on X X A and the extreme points in it. It is possible that some strategies have infinite occupation measures. Nevertheless, we prove that every finite extreme occupation measure is generated by a deterministic stationary strategy. Then, for this MDP, we consider a constrained problem with total undiscounted criteria and J constraints, where the cost functions are nonnegative. By assumption, the strategies inducing infinite occupation measures are not optimal. Then our second main result is that, under mild conditions, the solution to this constrained MDP is given by a mixture of no more than J + 1 occupation measures generated by deterministic stationary strategies.
| Item Type: | Article |
|---|---|
| Uncontrolled Keywords: | 4901 Applied Mathematics, 5108 Quantum Physics, 49 Mathematical Sciences, 51 Physical Sciences |
| Divisions: | Faculty of Science & Engineering > School of Physical Sciences |
| Depositing User: | Symplectic Admin |
| Date Deposited: | 11 Jan 2024 08:41 |
| Last Modified: | 24 Jan 2026 04:43 |
| DOI: | 10.1137/23M1572398 |
| Related Websites: | |
| URI: | https://livrepository.liverpool.ac.uk/id/eprint/3177797 |
| Disclaimer: | The University of Liverpool is not responsible for content contained on other websites from links within repository metadata. Please contact us if you notice anything that appears incorrect or inappropriate. |
Altmetric
Altmetric