UPSI Digital Repository (UDRep)
|
UPSI Digital Repository (UDRep)
|
|
|
|
||||||||||||||||||||||||||||
| Abstract : Universiti Pendidikan Sultan Idris |
| Efficient evaluation for machine learning (ML)-based intrusion detection systems (IDSs) for federated learning (FL) in the Internet of Medical Things (IoMTs) environment falls under the standardisation and multicriteria decision-making (MCDM) problems. Thus, this study is developing an MCDM framework for standardising and benchmarking the ML-based IDSs used in the FL architecture of IoMT applications. In the methodology, firstly, the evaluation criteria of ML-based IDSs are standardised using the fuzzy Delphi method (FDM). Secondly, the evaluation decision matrix (DM) is formulated based on the intersection of standardised evaluation criteria and a list of ML-based IDSs. Such formulation is achieved using a dataset with 125,973 records, and each record comprises 41 features. Thirdly, the integration of MCDM methods is formulated to determine the importance weights of the main and sub standardised security and performance criteria, followed by benchmarking and selecting the optimal ML-based IDSs. In this phase, the Borda voting method is used to unify the different ranks and perform a group benchmarking context. The following results are confirmed. (1) Using FDM, 17 out of 20 evaluation criteria (14 for security and 3 for performance) reach the consensus of experts. (2) The area under curve criterion has the lowest set of weights, whilst the CPU time criterion has the highest one. (3) VIKOR group ranking shows that the BayesNet is a best classifier, whilst SVM is the last choice. For evaluation, three assessments, namely, systematic ranking, computational cost and comparative analysis, are used. 2022 IEEE. |
| References |
Abdulkareem, K. H., Arbaiy, N., Zaidan, A. A., Zaidan, B. B., Albahri, O. S., Alsalem, M. A., & Salih, M. M. (2021). A new standardisation and selection framework for real-time image dehazing algorithms from multi-foggy scenes based on fuzzy Delphi and hybrid multi-criteria decision analysis methods. Neural Computing and Applications, 33(4), 1029–1054. https://doi.org/10.1007/s00521-020-05020-4 Agrawal, S., Chowdhuri, A., Sarkar, S., Selvanambi, R., & Gadekallu, T. R. (2021). Temporal Weighted Averaging for Asynchronous Federated Intrusion Detection Systems. Computational Intelligence and Neuroscience, 2021. https://doi.org/10.1155/2021/5844728 Ahmad, I., Abdullah, A., & Alghamdi, A. (2010). Towards the selection of best neural network system for intrusion detection. International Journal of Physical Sciences, 5(12), 1830–1839. Ahmad, I., Basheri, M., Iqbal, M. J., & Rahim, A. (2018). Performance Comparison of Support Vector Machine, Random Forest, and Extreme Learning Machine for Intrusion Detection. IEEE Access, 6, 33789–33795. https://doi.org/10.1109/ACCESS.2018.2841987 Albahri, A. S., Albahri, O. S., Zaidan, A. A., Zaidan, B. B., Hashim, M., Alsalem, M. A., Mohsin, A. H., Mohammed, K. I., Alamoodi, A. H., Enaizan, O., al Shafeey, G. A., & Baqer, M. J. (2019). Based Multiple Heterogeneous Wearable Sensors: A Smart Real-Time Health Monitoring Structured for Hospitals Distributor. IEEE Access, 7, 37269–37323. https://doi.org/10.1109/ACCESS.2019.2898214 Albahri, A. S., Alwan, J. K., Taha, Z. K., Ismail, S. F., Hamid, R. A., Zaidan, A. A., Albahri, O. S., Zaidan, B. B., Alamoodi, A. H., & Alsalem, M. A. (2021). IoT-based telemedicine for disease prevention and health promotion: State-of-the-Art. Journal of Network and Computer Applications, 173. https://doi.org/10.1016/j.jnca.2020.102873 Albahri, O. S., Zaidan, A. A., Salih, M. M., Zaidan, B. B., Khatari, M. A., Ahmed, M. A., Albahri, A. S., & Alazab, M. (2021). Multidimensional benchmarking of the active queue management methods of network congestion control based on extension of fuzzy decision by opinion score method. International Journal of Intelligent Systems, 36(2), 796–831. https://doi.org/10.1002/int.22322 Aldweesh, A., Derhab, A., & Emam, A. Z. (2020). Deep learning approaches for anomaly-based intrusion detection systems: A survey, taxonomy, and open issues. Knowledge-Based Systems, 189. https://doi.org/10.1016/j.knosys.2019.105124 Alsalem, M. A., Alamoodi, A. H., Albahri, O. S., Dawood, K. A., Mohammed, R. T., Alnoor, A., Zaidan, A. A., Albahri, A. S., Zaidan, B. B., Jumaah, F. M., Jumaah, F. M., & Al-Obaidi, J. R. (2022). Multi-criteria decision-making for coronavirus disease 2019 applications: a theoretical analysis review. Artificial Intelligence Review, 55(6), 4979–5062. https://doi.org/10.1007/s10462-021-10124-x Alsalem, M. A., Mohammed, R., Albahri, O. S., Zaidan, A. A., Alamoodi, A. H., Dawood, K., Alnoor, A., Albahri, A. S., Zaidan, B. B., Aickelin, U., Alazab, M., & Jumaah, F. (2022). Rise of multiattribute decision-making in combating COVID-19: A systematic review of the state-of-the-art literature. International Journal of Intelligent Systems, 37(6), 3514–3624. https://doi.org/10.1002/int.22699 Alsalem, M. A., Zaidan, A. A., Zaidan, B. B., Albahri, O. S., Alamoodi, A. H., Albahri, A. S., Mohsin, A. H., & Mohammed, K. I. (2019). Multiclass Benchmarking Framework for Automated Acute Leukaemia Detection and Classification Based on BWM and Group-VIKOR. Journal of Medical Systems, 43(7). https://doi.org/10.1007/s10916-019-1338-x Bodjanova, S. (2006). Median alpha-levels of a fuzzy number. Fuzzy Sets and Systems, 157(7), 879–891. https://doi.org/10.1016/j.fss.2005.10.015 Bottomley, P. A., & Doyle, J. R. (2001). A comparison of three weight elicitation methods: Good, better, and best. Omega, 29(6), 553–560. https://doi.org/10.1016/S0305-0483(01)00044-5 Dawood, K. A., Sharif, K. Y., Ghani, A. A., Zulzalil, H., Zaidan, A. A., & Zaidan, B. B. (2021). Towards a unified criteria model for usability evaluation in the context of open source software based on a fuzzy Delphi method. Information and Software Technology, 130. https://doi.org/10.1016/j.infsof.2020.106453 Emerson, P. (2013). The original Borda count and partial voting. Social Choice and Welfare, 40(2), 353–358. https://doi.org/10.1007/s00355-011-0603-9 Ghaleb, A. M., Kaid, H., Alsamhan, A., Mian, S. H., & Hidri, L. (2020). Assessment and Comparison of Various MCDM Approaches in the Selection of Manufacturing Process. Advances in Materials Science and Engineering, 2020. https://doi.org/10.1155/2020/4039253 Ghaznavi, M., Shahriar, N., Kamali, S., Ahmed, R., & Boutaba, R. (2017). Distributed service function chaining. IEEE Journal on Selected Areas in Communications, 35(11), 2479–2489. https://doi.org/10.1109/JSAC.2017.2760178 Gong, M., Xie, Y., Pan, K., Feng, K., & Qin, A. K. (2020). A Survey on Differentially Private Machine Learning [Review Article]. IEEE Computational Intelligence Magazine, 15(2), 49–64. https://doi.org/10.1109/MCI.2020.2976185 Guo, W., Chen, Y., Cai, Y., Wang, T., & Tian, H. (2017). DIntrusion detection in WSN with an improved NSA based on the DE-CMOP. KSII Transactions on Internet and Information Systems, 11(11), 5574–5591. https://doi.org/10.3837/tiis.2017.11.022 Lian, Z., Wang, W., & Su, C. (2021). COFEL: Communication-Efficient and Optimized Federated Learning with Local Differential Privacy. IEEE International Conference on Communications. https://doi.org/10.1109/ICC42927.2021.9500632 Lim, W. Y. B., Luong, N. C., Hoang, D. T., Jiao, Y., Liang, Y.-C., Yang, Q., Niyato, D., & Miao, C. (2020). Federated Learning in Mobile Edge Networks: A Comprehensive Survey. IEEE Communications Surveys and Tutorials, 22(3), 2031–2063. https://doi.org/10.1109/COMST.2020.2986024 Liu, D., Zhang, Y., Wang, W., Dev, K., & Khowaja, S. A. (2023). Flexible Data Integrity Checking with Original Data Recovery in IoT-Enabled Maritime Transportation Systems. IEEE Transactions on Intelligent Transportation Systems, 24(2), 2618–2629. https://doi.org/10.1109/TITS.2021.3125070 Milenkoski, A., Vieira, M., Kounev, S., Avritzer, A., & Payne, B. D. (2015). Evaluating computer intrusion detection systems: A survey of common practices. ACM Computing Surveys, 48(1). https://doi.org/10.1145/2808691 Mohammed, K. I., Zaidan, A. A., Zaidan, B. B., Albahri, O. S., Albahri, A. S., Alsalem, M. A., & Mohsin, A. H. (2020). Novel technique for reorganisation of opinion order to interval levels for solving several instances representing prioritisation in patients with multiple chronic diseases. Computer Methods and Programs in Biomedicine, 185. https://doi.org/10.1016/j.cmpb.2019.105151 Mohammed, R. T., Yaakob, R., Zaidan, A. A., Sharef, N. M., Abdullah, R. H., Zaidan, B. B., & Dawood, K. A. (2020). Review of the Research Landscape of Multi-Criteria Evaluation and Benchmarking Processes for Many-Objective Optimization Methods: Coherent Taxonomy, Challenges and Recommended Solution. International Journal of Information Technology and Decision Making, 19(6), 1619–1693. https://doi.org/10.1142/S0219622020300049 Mohammed, R. T., Zaidan, A. A., Yaakob, R., Sharef, N. M., Abdullah, R. H., Zaidan, B. B., Albahri, O. S., & Abdulkareem, K. H. (2022). Determining Importance of Many-Objective Optimisation Competitive Algorithms Evaluation Criteria Based on a Novel Fuzzy-Weighted Zero-Inconsistency Method. International Journal of Information Technology and Decision Making, 21(1), 195–241. https://doi.org/10.1142/S0219622021500140 Mothukuri, V., Khare, P., Parizi, R. M., Pouriyeh, S., Dehghantanha, A., & Srivastava, G. (2022). Federated-Learning-Based Anomaly Detection for IoT Security Attacks. IEEE Internet of Things Journal, 9(4), 2545–2554. https://doi.org/10.1109/JIOT.2021.3077803 Murray, T. J., Pipino, L. L., & van Gigch, J. P. (1985). A pilot study of fuzzy set modification of delphi. Human Systems Management, 5(1), 76–80. https://doi.org/10.3233/HSM-1985-5111 Ogami, K., Kula, R. G., Hata, H., Ishio, T., & Matsumoto, K. (2017). Using High-Rising Cities to Visualize Performance in Real-Time. Proceedings - 2017 IEEE Working Conference on Software Visualization, VISSOFT 2017, 2017-Octob, 33–42. https://doi.org/10.1109/VISSOFT.2017.25 Panigrahi, R., & Borah, S. (2018). Rank Allocation to J48 Group of Decision Tree Classifiers using Binary and Multiclass Intrusion Detection Datasets. Procedia Computer Science, 132, 323–332. https://doi.org/10.1016/j.procs.2018.05.186 Papadopoulos, P., Abramson, W., Hall, A. J., Pitropakis, N., & Buchanan, W. J. (2021). Privacy and Trust Redefined in Federated Machine Learning. Machine Learning and Knowledge Extraction, 3(2), 333–356. https://doi.org/10.3390/make3020017 Patsariya, P., & Singh, R. R. (2019). Classifier rank identification using multi-criteria decision making method for intrusion detection dataset. International Journal of Innovative Technology and Exploring Engineering, 9(1), 1732–1738. https://doi.org/10.35940/ijitee.A5223.119119 Peng, Y., Kou, G., Wang, G., & Shi, Y. (2011). FAMCDM: A fusion approach of MCDM methods to rank multiclass classification algorithms. Omega, 39(6), 677–689. https://doi.org/10.1016/j.omega.2011.01.009 Qureshi, F., & Krishnan, S. (2018). Wearable hardware design for the internet of medical things (IoMT). Sensors (Switzerland), 18(11). https://doi.org/10.3390/s18113812 Robinson, R. R. R., & Thomas, C. (2016). Ranking of machine learning algorithms based on the performance in classifying DDoS attacks. 2015 IEEE Recent Advances in Intelligent Computational Systems, RAICS 2015, 185–190. https://doi.org/10.1109/RAICS.2015.7488411 Song, J., Wang, W., Gadekallu, T. R., Cao, J., & Liu, Y. (2023). EPPDA: An Efficient Privacy-Preserving Data Aggregation Federated Learning Scheme. IEEE Transactions on Network Science and Engineering, 10(5), 3047–3057. https://doi.org/10.1109/TNSE.2022.3153519 Swarna Priya, R. M., Maddikunta, P. K. R., Parimala, M., Koppu, S., Gadekallu, T. R., Chowdhary, C. L., & Alazab, M. (2020). An effective feature engineering for DNN using hybrid PCA-GWO for intrusion detection in IoMT architecture. Computer Communications, 160, 139–149. https://doi.org/10.1016/j.comcom.2020.05.048 Taheri, R., Shojafar, M., Alazab, M., & Tafazolli, R. (2021). Fed-IIoT: A Robust Federated Malware Detection Architecture in Industrial IoT. IEEE Transactions on Industrial Informatics, 17(12), 8442–8452. https://doi.org/10.1109/TII.2020.3043458 Tang, C.-W., & Wu, C.-T. (2010). Obtaining a picture of undergraduate education quality: A voice from inside the university. Higher Education, 60(3), 269–286. https://doi.org/10.1007/s10734-009-9299-5 Tang, Z., Chen, Y., Ye, S., Hu, R., Wang, H., He, J., Huang, Q., & Chang, S. (2020). Fully memristive spiking-neuron learning framework and its applications on pattern recognition and edge detection. Neurocomputing, 403, 80–87. https://doi.org/10.1016/j.neucom.2020.04.012 Tang, Z., Sun, Z.-H., Wu, E. Q., Wei, C.-F., Ming, D., & Chen, S.-D. (2023). MRCG: A MRI Retrieval Framework With Convolutional and Graph Neural Networks for Secure and Private IoMT. IEEE Journal of Biomedical and Health Informatics, 27(2), 814–822. https://doi.org/10.1109/JBHI.2021.3130028 Tang, Z., Zhu, R., Hu, R., Chen, Y., Wu, E. Q., Wang, H., He, J., Huang, Q., & Chang, S. (2021). A Multilayer Neural Network Merging Image Preprocessing and Pattern Recognition by Integrating Diffusion and Drift Memristors. IEEE Transactions on Cognitive and Developmental Systems, 13(3), 645–656. https://doi.org/10.1109/TCDS.2020.3003377 Tang, Z., Zhu, R., Lin, P., He, J., Wang, H., Huang, Q., Chang, S., & Ma, Q. (2019). A hardware friendly unsupervised memristive neural network with weight sharing mechanism. Neurocomputing, 332, 193–202. https://doi.org/10.1016/j.neucom.2018.12.049 Tavallaee, M., Bagheri, E., Lu, W., & Ghorbani, A. A. (2009). A detailed analysis of the KDD CUP 99 data set. IEEE Symposium on Computational Intelligence for Security and Defense Applications, CISDA 2009. https://doi.org/10.1109/CISDA.2009.5356528 Wang, W., Chen, Q., Yin, Z., Srivastava, G., Gadekallu, T. R., Alsolami, F., & Su, C. (2022). Blockchain and PUF-Based Lightweight Authentication Protocol for Wireless Medical Sensor Networks. IEEE Internet of Things Journal, 9(11), 8883–8891. https://doi.org/10.1109/JIOT.2021.3117762 Wang, W., Memon, F. H., Lian, Z., Yin, Z., Gadekallu, T. R., Pham, Q.-V., Dev, K., & Su, C. (2023). Secure-Enhanced Federated Learning for AI-Empowered Electric Vehicle Energy Prediction. IEEE Consumer Electronics Magazine, 12(2), 27–34. https://doi.org/10.1109/MCE.2021.3116917 Wang, W., Xu, H., Alazab, M., Gadekallu, T. R., Han, Z., & Su, C. (2022). Blockchain-Based Reliable and Efficient Certificateless Signature for IIoT Devices. IEEE Transactions on Industrial Informatics, 18(10), 7059–7067. https://doi.org/10.1109/TII.2021.3084753 Wang, X., Garg, S., Lin, H., Hu, J., Kaddoum, G., Piran, M. J., & Shamim Hossain, M. (2022). Toward Accurate Anomaly Detection in Industrial Internet of Things Using Hierarchical Federated Learning. IEEE Internet of Things Journal, 9(10), 7110–7119. https://doi.org/10.1109/JIOT.2021.3074382 Welagedara, L., Harischandra, J., & Jayawardene, N. (2021). A Review on Edge Intelligence based Collaborative Learning Approaches. 2021 IEEE 11th Annual Computing and Communication Workshop and Conference, CCWC 2021, 572–577. https://doi.org/10.1109/CCWC51732.2021.9376119 Zhang, C., Xie, Y., Bai, H., Yu, B., Li, W., & Gao, Y. (2021). A survey on federated learning. Knowledge-Based Systems, 216. https://doi.org/10.1016/j.knosys.2021.106775 Zhou, X., Xu, X., Liang, W., Zeng, Z., Shimizu, S., Yang, L. T., & Jin, Q. (2022). Intelligent Small Object Detection for Digital Twin in Smart Manufacturing with Industrial Cyber-Physical Systems. IEEE Transactions on Industrial Informatics, 18(2), 1377–1386. https://doi.org/10.1109/TII.2021.3061419 Zhou, X., Yang, X., Ma, J., & Wang, K. I.-K. (2022). Energy-Efficient Smart Routing Based on Link Correlation Mining for Wireless Edge Computing in IoT. IEEE Internet of Things Journal, 9(16), 14988–14997. https://doi.org/10.1109/JIOT.2021.3077937 |
| This material may be protected under Copyright Act which governs the making of photocopies or reproductions of copyrighted materials. You may use the digitized material for private study, scholarship, or research. |