Durdy, Samantha, Gaultois, Michael W 
ORCID: 0000-0003-2172-2507, Gusev, Vladimir V ORCID: 0000-0002-2815-607X, Bollegala, Danushka ORCID: 0000-0003-4476-7003 and Rosseinsky, Matthew J ORCID: 0000-0002-1910-2483
(2022)
Random projections and kernelised leave one cluster out cross validation: universal baselines and evaluation tools for supervised machine learning of material properties
DIGITAL DISCOVERY, 1 (6).
pp. 763-778.
ISSN 2635-098X, 2635-098X
Abstract
With machine learning being a popular topic in current computational materials science literature, creating representations for compounds has become common place. These representations are rarely compared, as evaluating their performance – and the performance of the algorithms that they are used with – is nontrivial. With many materials datasets containing bias and skew caused by the research process, leave one cluster out cross validation (LOCO-CV) has been introduced as a way of measuring the performance of an algorithm in predicting previously unseen groups of materials. This raises the question of the impact, and control, of the range of cluster sizes on the LOCO-CV measurement outcomes. We present a thorough comparison between composition-based representations, and investigate how kernel approximation functions can be used to better separate data to enhance LOCO-CV applications. We find that domain knowledge does not improve machine learning performance in most tasks tested, with band gap prediction being the notable exception. We also find that the radial basis function improves the linear separability of chemical datasets in all 10 datasets tested and provides a framework for the application of this function in the LOCO-CV process to improve the outcome of LOCO-CV measurements regardless of machine learning algorithm, choice of metric, and choice of compound representation. We recommend kernelised LOCO-CV as a training paradigm for those looking to measure the extrapolatory power of an algorithm on materials data.
| Item Type: | Article |
|---|---|
| Uncontrolled Keywords: | 46 Information and Computing Sciences, 4611 Machine Learning, Machine Learning and Artificial Intelligence, Bioengineering, Networking and Information Technology R&D (NITRD), Data Science |
| Divisions: | Faculty of Science & Engineering > School of Electrical Engineering, Electronics and Computer Science Faculty of Science & Engineering > School of Physical Sciences |
| Depositing User: | Symplectic Admin |
| Date Deposited: | 26 Sep 2023 14:07 |
| Last Modified: | 16 Jun 2026 17:07 |
| DOI: | 10.1039/d2dd00039c |
| Open Access URL: | https://doi.org/10.1039/D2DD00039C |
| Related Websites: | |
| URI: | https://livrepository.liverpool.ac.uk/id/eprint/3173076 |
| Disclaimer: | The University of Liverpool is not responsible for content contained on other websites from links within repository metadata. Please contact us if you notice anything that appears incorrect or inappropriate. |
Altmetric
Altmetric