Hyperparameter Optimization for Bayesian Decision Trees

Drousiotis, E, Habibi, S, Varsi, A ORCID: 0000-0003-2218-4720, Maskell, S ORCID: 0000-0003-1917-2913 and Spirakis, PG ORCID: 0000-0001-5396-3749 (2026) Hyperparameter Optimization for Bayesian Decision Trees In: 11th International Conference on Machine Learning , Optimization and Data Science (LOD 2025), 2025-9-11 - 2025-9-24, Tuscany , Italy.

Text Hyperparameter_Optimization_for_Bayesian_Decision_Trees (1).pdf - Author Accepted Manuscript Available under License Creative Commons Attribution. Download (537kB) | Preview

Abstract

Bayesian Decision Trees (DTs) are strong probabilistic models that integrate observed data with prior beliefs, providing robust interpretability in Machine Learning uncertainty. They are highly valuable in several fields such as Medicine, Finance and Education because of their ability to balance generalization and adaptability. However, tuning the Poisson prior parameter λ, which controls the expected number of leaf nodes, is crucial as a poor choice can lead to inefficient sampling, suboptimal posterior approximations, and increased computational cost. In this work, we propose a novel Recursive Interval Optimization (RIO) algorithm that, with high probability, preserves the most promising interval at each subdivision and has quasi-polynomial computational complexity in the interval width under mild assumptions. Furthermore, we experimentally demonstrate that RIO-tuned SMC on multiple real-world classification datasets outperforms traditional Random Search (RS) and standard Random Forest (RF) in accuracy while producing more compact trees. These results highlight RIO’s effectiveness and theoretical guarantees for hyperparameter tuning of Bayesian DTs.

Item Type:	Conference Item (Unspecified)
Uncontrolled Keywords:	46 Information and Computing Sciences, 4611 Machine Learning, Machine Learning and Artificial Intelligence, Networking and Information Technology R&D (NITRD)
Divisions:	Faculty of Science & Engineering Faculty of Science & Engineering > School of Electrical Engineering, Electronics and Computer Science
Depositing User:	Symplectic Admin
Date Deposited:	03 Jun 2025 08:48
Last Modified:	09 Jun 2026 15:05
DOI:	10.1007/978-3-032-21477-5_10
Related Websites:	Publisher
URI:	https://livrepository.liverpool.ac.uk/id/eprint/3192923
Disclaimer:	The University of Liverpool is not responsible for content contained on other websites from links within repository metadata. Please contact us if you notice anything that appears incorrect or inappropriate.