Evaluating Feature Selection Stability and Efficiency in Multiclass Intrusion Detection Systems Using Machine Learning Algorithms
DOI:
https://doi.org/10.65405/gjq8v425Keywords:
Intrusion detection systems, feature selection, machine learning, multi-class classification, Jaccard coefficient, UNSW-NB15.Abstract
This study aims to evaluate the stability and efficiency of feature selection algorithms in multi-class intrusion detection systems (IDS) that utilize machine learning. It examines the impact of feature selection on the stability of the selected feature subsets, classification performance, and computational efficiency. Using the UNSW-NB15 dataset, the study applied the Mutual Information (MI) and Chi-Square algorithms to select the top 20 features out of 194. Selection stability was assessed using the Jaccard coefficient across five random samples, while classification performance was evaluated using metrics such as Accuracy, Precision, Recall, and F1-score. Additionally, training and prediction times were measured.
The results indicate that the MI algorithm achieved the highest stability in feature selection, with an average Jaccard coefficient of 1.0000, compared to 0.8883 for the Chi-Square algorithm. MI also yielded the highest Macro F1-score (0.5226) and the highest overall accuracy (0.7487). Conversely, the Chi-Square algorithm demonstrated superior temporal efficiency, with a training time of 326.42 seconds and a prediction time of 1.09 seconds. These findings underscore the importance of evaluating feature selection algorithms across multiple dimensions—including stability, classification quality, and computational efficiency—rather than relying on a single metric to determine the most suitable algorithm for multi-class intrusion detection systems.
Downloads
References
1. More, S., Idrissi, M., Mahmoud, H., & Asyhari, A. T. (2024). Enhanced intrusion detection systems performance with UNSW-NB15 data analysis. Algorithms, 17(2), 64. https://doi.org/10.3390/a17020064
2. Sowmya, T., & Mary Anita, E. A. (2025). A novel stable feature selection algorithm for machine learning based intrusion detection system. Procedia Computer Science, 252, 738–747. https://doi.org/10.1016/j.procs.2025.01.034
3. Umer, M., Tahir, M., Sardaraz, M., Sharif, M., Elmannai, H., & Algarni, A. D. (2025). Network intrusion detection model using wrapper based feature selection and multi head attention transformers. Scientific Reports, 15, Article 28718. https://doi.org/10.1038/s41598-025-11348-5
4. Saputra, A., Ramli, K., Nugroho, A. S., Nugraha, I. G. D., & Pranggono, B. (2026). A novel hybrid IGL1 feature selection method for high-performance intrusion detection on the UNSW-NB15 dataset using multiple machine learning models. Big Data and Cognitive Computing, 10(6), 182. https://doi.org/10.3390/bdcc10060182
5. Khraisat, A., Gondal, I., Vamplew, P., & Kamruzzaman, J. (2019). Survey of intrusion detection systems: techniques, datasets and challenges. Cybersecurity, 2, Article 20. https://doi.org/10.1186/s42400-019-0038-7.
6. Dina, A. S., & Manivannan, D. (2021). Intrusion detection based on Machine Learning techniques in computer networks. Internet of Things, 16, 100462. https://doi.org/10.1016/j.iot.2021.100462.
7. Moustafa, N., & Slay, J. (2015). UNSW-NB15: A comprehensive data set for network intrusion detection systems (UNSW-NB15 network data set). In 2015 Military Communications and Information Systems Conference (MilCIS) (pp. 1–6). IEEE. https://doi.org/10.1109/MilCIS.2015.7348942
