Novel multi-perspective usability evaluation framework for selection of open source software based on BWM and group VIKOR techniques

Zaidan, Bilal Bahaa

Novel multi-perspective usability evaluation framework for selection of open source software based on BWM and group VIKOR techniques

AVAILABLE Article 1947 Views

Back to search

Scan QR Code to share or view item link:

Digital Assets & Download Files

General Information Cached

Title

Novel multi-perspective usability evaluation framework for selection of open source software based on BWM and group VIKOR techniques

Type

Article

Main Author

Zaidan, Bilal Bahaa

Additional Authors

Zaidan, A. A.

Subject

QA75 Electronic computers. Computer science

ISSN

0219-6220

Language Code

eng

Bibliographic Details

Place of Production

Tanjung Malim

Publisher

Fakulti Seni, Komputeran Dan Industri Kreatif

Year of publication

2021

Notes

International Journal of Information Technology and Decision Making

Degree Type

Others

Corporate Name

Universiti Pendidikan Sultan Idris

HTTP Link

Access Web Resource

Abstract

Increasing demand for open-source software (OSS) has raised the value of efficient selection in terms of quality; usability is an essential quality factor that significantly affects system acceptability and sustainability. Most large and complex software packages partitioned across multiple portals and involve many users - each with their role in the software package; those users have different perspectives on the software package, defined by their knowledge, responsibilities, and commitments. Thus, a multi-perspective approach has been used in usability evaluation to overcome the challenge of inconsistency between users' perspectives; the inconsistency challenge would lead to an ill-advised decision on the selection of a suitable OSS. This study aimed to assist the public and private organizations in evaluating and selecting the most suitable OSS. The evaluation of the OSS software packages to choose the best one is a challenging task owing to (a) multiple evaluation criteria, (b) criteria importance, and (c) data variation; thus, it is considered a sophisticated multi-criteria decision making (MCDM) problem; moreover, the multi-perspective usability evaluation framework for OSS selection lacks in the current literature. Hence, this study proposes a novel multi-perspective usability evaluation framework for the selection of OSS based on the multi-criteria analysis. Integration of best-worst method (BWM) and VIKOR MCDM techniques has been used for weighting and ranking OSS alternatives. BWM is utilized for weighting of evaluation criteria, whereas VIKOR is applied to rank OSS-LMS alternatives. Individual and group decision-making contexts, and the internal and external groups aggregation were used to demonstrate the efficiency of the proposed framework. A well-organized algorithmic procedure is presented in detail, and a case study was examined to illustrate the validity and feasibility of the proposed framework. The results demonstrated that BWM and VIKOR integration works effectively to solve the OSS software package benchmarking/selection problems. Furthermore, the ranks of OSS software packages obtained from the VIKOR internal and external group decision making were similar; the best OSS-LMS based on the two ways was 'Moodle' software package. Among the scores of groups in the objective validation, significant differences were identified; this indicated that the ranking results of internal and external VIKOR group decision making were valid, which pointed to the validation of the framework. ? 2021 World Scientific Publishing Company.

References

Abdulkareem, K. H., Arbaiy, N., Zaidan, A. A., Zaidan, B. B., Albahri, O. S., Alsalem, M. A., & Salih, M. M. (2021). A new standardisation and selection framework for real-time image dehazing algorithms from multi-foggy scenes based on fuzzy delphi and hybrid multi-criteria decision analysis methods. Neural Computing and Applications, 33(4), 1029-1054. doi:10.1007/s00521-020-05020-4

Abdullateef, B. N., Elias, N. F., Mohamed, H., Zaidan, A. A., & Zaidan, B. B. (2016). An evaluation and selection problems of OSS-LMS packages. SpringerPlus, 5(1), 1-35. doi:10.1186/s40064-016-1828-y

Adewumi, A., Misra, S., Omoregbe, N., & Sanz, L. F. (2019). FOSSES: Framework for open-source software evaluation and selection. Software - Practice and Experience, 49(5), 780-812. doi:10.1002/spe.2682

Alaa, M., Albakri, I. S. M. A., Singh, C. K. S., Hammed, H., Zaidan, A. A., Zaidan, B. B., . . . Jasim, A. N. (2019). Assessment and ranking framework for the english skills of pre-service teachers based on fuzzy delphi and TOPSIS methods. IEEE Access, 7, 126201-126223. doi:10.1109/ACCESS.2019.2936898

Albahri, A. S., Zaidan, A. A., Albahri, O. S., Zaidan, B. B., & Alsalem, M. A. (2018). Real-time fault-tolerant mHealth system: Comprehensive review of healthcare services, opens issues, challenges and methodological aspects. Journal of Medical Systems, 42(8) doi:10.1007/s10916-018-0983-9

Albahri, O. S., Albahri, A. S., Mohammed, K. I., Zaidan, A. A., Zaidan, B. B., Hashim, M., & Salman, O. H. (2018). Systematic review of real-time remote health monitoring system in triage and priority-based sensor technology: Taxonomy, open challenges, motivation and recommendations. Journal of Medical Systems, 42(5) doi:10.1007/s10916-018-0943-4

Albahri, O. S., Albahri, A. S., Zaidan, A. A., Zaidan, B. B., Alsalem, M. A., Mohsin, A. H., . . . Shareef, A. H. (2019). Fault-tolerant mHealth framework in the context of IoT-based real-time wearable health data sensors. IEEE Access, 7, 50052-50080. doi:10.1109/ACCESS.2019.2910411

Albahri, O. S., Zaidan, A. A., Zaidan, B. B., Hashim, M., Albahri, A. S., & Alsalem, M. A. (2018). Real-time remote health-monitoring systems in a medical centre: A review of the provision of healthcare services-based body sensor information, open challenges and methodological aspects. Journal of Medical Systems, 42(9) doi:10.1007/s10916-018-1006-6

Almahdi, E. M., Zaidan, A. A., Zaidan, B. B., Alsalem, M. A., Albahri, O. S., & Albahri, A. S. (2019). Mobile patient monitoring systems from a benchmarking aspect: Challenges, open issues and recommended solutions. Journal of Medical Systems, 43(7) doi:10.1007/s10916-019-1336-z

Al-Rawashdeh, T. A. (2015). Evaluating open source software usability using a multistage fuzzy model approach. International Review on Computers and Software, 10(10), 1018-1026. doi:10.15866/irecos.v10i10.7668

Bahamdain, S. S. (2015). Open source software (OSS) quality assurance: A survey paper. Paper presented at the Procedia Computer Science, , 56(1) 459-464. doi:10.1016/j.procs.2015.07.236 Retrieved from www.scopus.com

Ćetin, G., & Mehmet, G. (2008). A measurement based framework for assessment of usability-centricness of open source software projects. Paper presented at the SITIS 2008 - Proceedings of the 4th International Conference on Signal Image Technology and Internet Based Systems, 585-592. doi:10.1109/SITIS.2008.106 Retrieved from www.scopus.com

Côté, M. -., Suryn, W., & Georgiadou, E. (2007). In search for a widely applicable and accepted software quality model for software quality engineering. Software Quality Journal, 15(4), 401-416. doi:10.1007/s11219-007-9029-0

Dawood, K. A., Sharif, K. Y., Zaidan, A. A., Abd Ghani, A. A., Zulzalil, H. B., & Zaidan, B. B. (2019). Mapping and analysis of open source software (OSS) usability for sustainable OSS product. IEEE Access, 7, 65913-65933. doi:10.1109/ACCESS.2019.2914368

Fenton, N. E., & Neil, M. (2000). Software metrics: Roadmap. Paper presented at the Proceedings of the Conference on the Future of Software Engineering, ICSE 2000, 357-370. doi:10.1145/336512.336588 Retrieved from www.scopus.com

Fernández-Pérez, Y., Febles-Estrada, A., Cruz, C., & Verdegay, J. L. (2018). Fuzzy multi-criteria decision making methods applied to usability software assessment: An annotated bibliography doi:10.1007/978-3-319-69989-9_11 Retrieved from www.scopus.com

Hauge, O. (2007). Open source software in software intensive industry - A survey. Technical Report, Retrieved from www.scopus.com

Hedberg, H., Iivari, N., Rajanen, M., & Harjumaa, L. (2007). Assuring quality and usability in open source software development. Paper presented at the First International Workshop on Emerging Trends in FLOSS Research and Development, FLOSS'07, 2-6. doi:10.1109/FLOSS.2007.2 Retrieved from www.scopus.com

Iivari, N. (2008). Usability in open source software development - an interpretive case study. Paper presented at the 16th European Conference on Information Systems, ECIS 2008, Retrieved from www.scopus.com

Iivari, N., Rajanen, M., & Hedberg, H. (2014). Encouraging for enculturation - an enquiry on the effort of usability specialists entering OSS projects. Paper presented at the Proceedings of the 25th Australasian Conference on Information Systems, ACIS 2014, Retrieved from www.scopus.com

Joia, L. A., & dos Santos Vinhais, J. C. (2017). From closed source to open source software: Analysis of the migration process to open office. Journal of High Technology Management Research, 28(2), 261-272. doi:10.1016/j.hitech.2017.10.008

Jusoh, Y., Chamili, K., Yahaya, J. H., & Pa, N. C. (2012). The selection criteria of open source software adoption in malaysia. International Journal of Advancements in Computing Technology, 4(21), 278-287. doi:10.4156/ijact.vol4.issue21.33

Jusoh, Y. Y., Chamili, K., Pa, N. C., & Yahaya, J. H. (2014). Open source software selection using an analytical hierarchy process (ahp). American Journal of Software Engineering and Applications, 3(6), 83-89. Retrieved from www.scopus.com

Kannan, A. S. K., Balamurugan, S. A. A., & Sasikala, S. (2021). A novel software package selection method using teaching-learning based optimization and multiple criteria decision making. IEEE Transactions on Engineering Management, 68(4), 941-954. doi:10.1109/TEM.2019.2918050

Keeney, R. L., & Raiffa, H. (1976). Decisions with Multiple Objectives: Preferences and Value Tradeoffs, Retrieved from www.scopus.com

Khatari, M., Zaidan, A. A., Zaidan, B. B., Albahri, O. S., & Alsalem, M. A. (2019). Multi-criteria evaluation and benchmarking for active queue management methods: Open issues, challenges and recommended pathway solutions. International Journal of Information Technology and Decision Making, 18(4), 1187-1242. doi:10.1142/S0219622019300039

Khosravi, K., & Guéhéneuc, Y. -. (2005). Open issues with quality models. Proceedings of the 9^th QAOOSE Workshop(ECOOP), Retrieved from www.scopus.com

Kiah, M. L. M., Haiqi, A., Zaidan, B. B., & Zaidan, A. A. (2014). Open source EMR software: Profiling, insights and hands-on analysis. Computer Methods and Programs in Biomedicine, 117(2), 360-382. doi:10.1016/j.cmpb.2014.07.002

Kou, G., Ergu, D., & Shang, J. (2014). Enhancing data consistency in decision matrix: Adapting hadamard model to mitigate judgment contradiction. European Journal of Operational Research, 236(1), 261-271. doi:10.1016/j.ejor.2013.11.035

Kou, G., & Lin, C. (2014). A cosine maximization method for the priority vector derivation in AHP. European Journal of Operational Research, 235(1), 225-232. doi:10.1016/j.ejor.2013.10.019

Lee, Y. M., Kim, J. B., Choi, I. W., & Rhew, S. Y. (2007). A study on selection process of open source software. Paper presented at the Proceedings - ALPIT 2007 6th International Conference on Advanced Language Processing and Web Information Technology, 568-571. doi:10.1109/ALPIT.2007.83 Retrieved from www.scopus.com

Llerena, L., Rodriguez, N., Castro, J. W., & Acuña, S. T. (2019). Adapting usability techniques for application in open source software: A multiple case study. Information and Software Technology, 107, 48-64. doi:10.1016/j.infsof.2018.10.011

Malczewski, J. (1999). GIS and Multicriteria Decision Analysis, Retrieved from www.scopus.com

Marichal, J. -. (1998). Aggregation operators for multicriteria decision aid. Aggregation Operators for Multicriteria Decision Aid, Retrieved from www.scopus.com

Masson, A. L., Lalanne, D., & Amstutz, T. (2017). A usability refactoring process for large-scale open source projects: The ILIAS case study. Paper presented at the Conference on Human Factors in Computing Systems - Proceedings, , Part F127655 1135-1143. doi:10.1145/3027063.3053345 Retrieved from www.scopus.com

Mohammed, K. I., Jaafar, J., Zaidan, A. A., Albahri, O. S., Zaidan, B. B., Abdulkareem, K. H., . . . Alamoodi, A. H. (2020). A uniform intelligent prioritisation for solving diverse and big data generated from multiple chronic diseases patients based on hybrid decision-making and voting method. IEEE Access, 8, 91521-91530. doi:10.1109/ACCESS.2020.2994746

Mohammed, K. I., Zaidan, A. A., Zaidan, B. B., Albahri, O. S., Albahri, A. S., Alsalem, M. A., & Mohsin, A. H. (2020). Novel technique for reorganisation of opinion order to interval levels for solving several instances representing prioritisation in patients with multiple chronic diseases. Computer Methods and Programs in Biomedicine, 185 doi:10.1016/j.cmpb.2019.105151

Napi, N. M., Zaidan, A. A., Zaidan, B. B., Albahri, O. S., Alsalem, M. A., & Albahri, A. S. (2019). Medical emergency triage and patient prioritisation in a telemedicine environment: A systematic review. Health and Technology, 9(5), 679-700. doi:10.1007/s12553-019-00357-w

Nichols, D. M., & Twidale, M. B. (2006). The usability of open source software: Analysis and prospects. Open Source Softw.Business: Issues Perspectives, , 167-188. Retrieved from www.scopus.com

Nielsen, J. (1993). Usability Engineering, Retrieved from www.scopus.com

Padayachee, I., Kotze, P., & van Der Merwe, A. (2010). ISO 9126 external systems quality characteristics, sub-characteristics and domain specific criteria for evaluating e-learning systems. ISO 9126 External Systems Quality Characteristics, Sub-Characteristics and Domain Specific Criteria for Evaluating E-Learning Systems, Retrieved from www.scopus.com

Paul, C. L. (2009). A survey of usability practices in free/libre/open source software doi:10.1007/978-3-642-02032-2_23 Retrieved from www.scopus.com

Petrovic-Lazarevic, S., & Abraham, A. (2004). Hybrid Fuzzy-Linear Programming Approach for Multi Criteria Decision Making Problems, Retrieved from www.scopus.com

Rahmatullah, B., Zaidan, A. A., Mohamed, F., & Sali, A. (2017). Multi-complex attributes analysis for optimum GPS baseband receiver tracking channels selection. Paper presented at the 2017 4th International Conference on Control, Decision and Information Technologies, CoDIT 2017, , 2017-January 1084-1088. doi:10.1109/CoDIT.2017.8102743 Retrieved from www.scopus.com

Rajanen, M., & Iivari, N. (2010). Traditional usability costs and benefits - fitting them into open source software development. Paper presented at the 18th European Conference on Information Systems, ECIS 2010, Retrieved from www.scopus.com

Rajanen, M., Iivari, N., & Keskitalo, E. (2012). Introducing usability activities into open source software development projects - A participative approach. Paper presented at the NordiCHI 2012: Making Sense through Design - Proceedings of the 7th Nordic Conference on Human-Computer Interaction, 683-692. doi:10.1145/2399016.2399120 Retrieved from www.scopus.com

Raza, A., & Capretz, L. F. (2012). Do open source software developers listen to their users? First Monday, 17(3) doi:10.5210/fm.v17i3.3640

Raza, A., Capretz, L. F., & Ahmed, F. (2012). An open source usability maturity model (OS-UMM). Computers in Human Behavior, 28(4), 1109-1121. doi:10.1016/j.chb.2012.01.018

Raza, A., Capretz, L. F., & Ahmed, F. (2012). Users' perception of open source usability: An empirical study. Engineering with Computers, 28(2), 109-121. doi:10.1007/s00366-011-0222-1

Salih, M. M., Zaidan, B. B., Zaidan, A. A., & Ahmed, M. A. (2019). Survey on fuzzy TOPSIS state-of-the-art between 2007 and 2017. Computers and Operations Research, 104, 207-227. doi:10.1016/j.cor.2018.12.019

Sanga, C. (2010). A Technique for the Evaluation of Free and Open Source e-Learning Systems, Retrieved from www.scopus.com

Sarrab, M., & Rehman, O. M. H. (2014). Empirical study of open source software selection for adoption, based on software quality characteristics. Advances in Engineering Software, 69, 1-11. doi:10.1016/j.advengsoft.2013.12.001

Sarrab, M., & Rehman, O. M. H. (2013). Selection criteria of open source software: First stage for adoption. International Journal of Information Processing and Management, 4(4), 51-58. doi:10.4156/ijipm.vol4.issue4.6

Sbai, N., Lenarduzzi, V., Taibi, D., Sassi, S. B., & Ghezala, H. H. B. (2018). Exploring information from OSS repositories and platforms to support OSS selection decisions. Information and Software Technology, 104, 104-108. doi:10.1016/j.infsof.2018.07.009

Sethanandha, B. D., Massey, B., & Jones, W. (2010). Managing open source contributions for software project sustainability. Paper presented at the PICMET '10 - Portland International Center for Management of Engineering and Technology, Proceedings - Technology Management for Global Economic Growth, 2329-2337. Retrieved from www.scopus.com

Talal, M., Zaidan, A. A., Zaidan, B. B., Albahri, O. S., Alsalem, M. A., Albahri, A. S., . . . Alaa, M. (2019). Comprehensive review and analysis of anti-malware apps for smartphones. Telecommunication Systems, 72(2), 285-337. doi:10.1007/s11235-019-00575-7

Tractinsky, N. (2018). The usability construct: A dead end? Human-Computer Interaction, 33(2), 131-177. doi:10.1080/07370024.2017.1298038

Tzeng, G. -., & Huang, J. -. (2011). Multiple attribute decision making: Methods and applications. Multiple attribute decision making: Methods and applications (pp. 1-333) Retrieved from www.scopus.com

Viorres, N., Xenofon, P., Stavrakis, M., Vlachogiannis, E., Koutsabasis, P., & Darzentas, J. (2007). Major HCI challenges for open source software adoption and development doi:10.1007/978-3-540-73257-0_50 Retrieved from www.scopus.com

Wesson, J. (2003). Usability issues for e-commerce and e-learning: A developing country perspective. Proc.Int.Workshop on Utility, Usability and Complexity of e-Information Systems, , 43-55. Retrieved from www.scopus.com

Yas, Q. M., Zadain, A. A., Zaidan, B. B., Lakulu, M. B., & Rahmatullah, B. (2017). Towards on develop a framework for the evaluation and benchmarking of skin detectors based on artificial intelligent models using multi-criteria decision-making techniques. International Journal of Pattern Recognition and Artificial Intelligence, 31(3) doi:10.1142/S0218001417590029

Yas, Q. M., Zaidan, A. A., Zaidan, B. B., Rahmatullah, B., & Abdul Karim, H. (2018). Comprehensive insights into evaluation and benchmarking of real-time skin detectors: Review, open issues & challenges, and recommended solutions. Measurement: Journal of the International Measurement Confederation, 114, 243-260. doi:10.1016/j.measurement.2017.09.027

Yusop, N. S. M., Grundy, J., Schneider, J. -., & Vasa, R. (2020). How usability defects defer from non-usability defects?: A case study on open source projects. International Journal on Advanced Science, Engineering and Information Technology, (1), 98-105. doi:10.18517/ijaseit.10.1.10225

Zaidan, A. A., Zaidan, B. B., Al-Haiqi, A., Kiah, M. L. M., Hussain, M., & Abdulnabi, M. (2015). Evaluation and selection of open-source EMR software packages based on integrated AHP and TOPSIS. Journal of Biomedical Informatics, 53, 390-404. doi:10.1016/j.jbi.2014.11.012

Zaidan, A. A., Zaidan, B. B., Alsalem, M. A., Albahri, O. S., Albahri, A. S., & Qahtan, M. Y. (2020). Multi-agent learning neural network and bayesian model for real-time IoT skin detectors: A new evaluation and benchmarking methodology. Neural Computing and Applications, 32(12), 8315-8366. doi:10.1007/s00521-019-04325-3

Zaidan, A. A., Zaidan, B. B., Alsalem, M. A., Momani, F., & Zughoul, O. (2020). Novel multiperspective hiring framework for the selection of software programmer applicants based on AHP and group TOPSIS techniques. International Journal of Information Technology and Decision Making, 19(3), 775-847. doi:10.1142/S0219622020500121

Zaidan, A. A., Zaidan, B. B., Hussain, M., Al-Haiqi, A. M., Mat Kiah, M. L., & Abdulnabi, M. (2015). Multi-criteria analysis for OS-EMR software selection problem: A comparative study. Decision Support Systems, 78, 15-27. doi:10.1016/j.dss.2015.07.002

Zaidan, B. B., & Zaidan, A. A. (2017). Software and hardware FPGA-based digital watermarking and steganography approaches: Toward new methodology for evaluation and benchmarking using multi-criteria decision-making techniques. Journal of Circuits, Systems and Computers, 26(7) doi:10.1142/S021812661750116X

Zaidan, B. B., Zaidan, A. A., Karim, H. A., & Ahmad, N. N. (2017). A new digital watermarking evaluation and benchmarking methodology using an external group of evaluators and multi-criteria analysis based on ‘large-scale data’. Software - Practice and Experience, 47(10), 1365-1392. doi:10.1002/spe.2465

Zhao, L., Deek, F. P., & McHugh, J. A. (2010). Exploratory inspection-a user-based learning method for improving open source software usability. Journal of Software Maintenance and Evolution, 22(8), 653-675. doi:10.1002/smr.455

Zionts, S. (1979). MCDM-if not a roman numeral, then what? Interfaces, 9(4), 94-101. Retrieved from www.scopus.com

Zughoul, O., Momani, F., Almasri, O. H., Zaidan, A. A., Zaidan, B. B., Alsalem, M. A., . . . Hashim, M. (2018). Comprehensive insights into the criteria of student performance in various educational domains. IEEE Access, 6, 73245-73264. doi:10.1109/ACCESS.2018.2881282