Quality Assessment of MORL Algorithms: A Utility-Based Approach

Zintgraf, Luisa M, Kanters, Timon V, Roijers, Diederik M, Oliehoek, Frans ORCID: 0000-0003-4372-5055 and Beau, Philipp (2015) Quality Assessment of MORL Algorithms: A Utility-Based Approach. Benelearn 2015: Proceedings of the 24th Annual Machine Learning Conference of Belgium and the Netherlands.

Text Zintgraf15Benelearn.pdf - Unspecified Download (299kB)

Abstract

Sequential decision-making problems with multiple objectives occur often in practice. In such settings, the utility of a policy depends on how the user values different trade-offs between the objectives. Such valuations can be expressed by a so-called scalarisation function. However, the exact scalarisation function can be unknown when the agents should learn or plan. Therefore, instead of a single solution, the agents aim to produce a solution set that contains an optimal solution for all possible scalarisations. Because it is often not possible to produce an exact solution set, many algorithms have been proposed that produce approximate solution sets instead. We argue that when comparing these algorithms we should do so on the basis of user utility, and on a wide range of problems. In practice however, comparison of the quality of these algorithms have typically been done with only a few limited benchmarks and metrics that do not directly express the utility for the user. In this paper, we propose two metrics that express either the expected utility, or the maximal utility loss with respect to the optimal solution set. Furthermore, we propose a generalised benchmark in order to compare algorithms more reliably.

Item Type:	Article
Subjects:	?? QA75 ??
Depositing User:	Symplectic Admin
Date Deposited:	26 Nov 2015 09:10
Last Modified:	17 Dec 2022 01:30
Related URLs:	Publisher
URI:	https://livrepository.liverpool.ac.uk/id/eprint/2039202