Cross-Datasets Evaluation of Machine Learning Models for Intrusion Detection Systems

Al-Riyami, Said, Lisitsa, Alexei and Coenen, Frans ORCID: 0000-0003-1026-6649
(2022) Cross-Datasets Evaluation of Machine Learning Models for Intrusion Detection Systems. .

[thumbnail of icict2021.pdf] Text
icict2021.pdf - Author Accepted Manuscript

Download (257kB) | Preview


The conventional way to evaluate the performance of machine learning models intrusion detection systems (IDS) is by using the same dataset to train and test. This method might lead to the bias from the computer network where the traffic is generated. Because of that, the applicability of the learned models might not be adequately evaluated. We argued in Al-Riyami et al. (ACM, pp 2195-2197 [1]) that a better way is to use cross-datasets evaluation, where we use two different datasets for training and testing. Both datasets should be generated from various networks. Using this method as it was shown in Al-Riyami et al. (ACM, pp 2195-2197 [1]) may lead to a significant drop in the performance of the learned model. This indicates that the models learn very little knowledge about the intrusion, which would be transferable from one setting to another. The reasons for such behaviour were not fully understood in Al-Riyami et al. (ACM, pp 2195-2197 [1]). In this paper, we investigate the problem and show that the main reason is the different definitions of the same feature in both models. We propose the correction and further empirically investigate cross-datasets evaluation for various machine learning methods. Further, we explored cross-dataset evaluation in the multiclass classification of attacks, and we show for most models that learning traffic normality is more robust than learning intrusions.

Item Type: Conference or Workshop Item (Unspecified)
Uncontrolled Keywords: Machine Learning and Artificial Intelligence, Networking and Information Technology R&D (NITRD)
Divisions: Faculty of Science and Engineering > School of Electrical Engineering, Electronics and Computer Science
Depositing User: Symplectic Admin
Date Deposited: 05 Jul 2021 13:49
Last Modified: 06 Jun 2024 19:40
DOI: 10.1007/978-981-16-2102-4_73
Related URLs: