|
UPSI Digital Repository (UDRep)
|
|
| ||||||||||||||||||||||||||
| Abstract : Universiti Pendidikan Sultan Idris |
| Increasing demand for open-source software (OSS) has raised the value of efficient selection in terms of quality; usability is an essential quality factor that significantly affects system acceptability and sustainability. Most large and complex software packages partitioned across multiple portals and involve many users - each with their role in the software package; those users have different perspectives on the software package, defined by their knowledge, responsibilities, and commitments. Thus, a multi-perspective approach has been used in usability evaluation to overcome the challenge of inconsistency between users' perspectives; the inconsistency challenge would lead to an ill-advised decision on the selection of a suitable OSS. This study aimed to assist the public and private organizations in evaluating and selecting the most suitable OSS. The evaluation of the OSS software packages to choose the best one is a challenging task owing to (a) multiple evaluation criteria, (b) criteria importance, and (c) data variation; thus, it is considered a sophisticated multi-criteria decision making (MCDM) problem; moreover, the multi-perspective usability evaluation framework for OSS selection lacks in the current literature. Hence, this study proposes a novel multi-perspective usability evaluation framework for the selection of OSS based on the multi-criteria analysis. Integration of best-worst method (BWM) and VIKOR MCDM techniques has been used for weighting and ranking OSS alternatives. BWM is utilized for weighting of evaluation criteria, whereas VIKOR is applied to rank OSS-LMS alternatives. Individual and group decision-making contexts, and the internal and external groups aggregation were used to demonstrate the efficiency of the proposed framework. A well-organized algorithmic procedure is presented in detail, and a case study was examined to illustrate the validity and feasibility of the proposed framework. The results demonstrated that BWM and VIKOR integration works effectively to solve the OSS software package benchmarking/selection problems. Furthermore, the ranks of OSS software packages obtained from the VIKOR internal and external group decision making were similar; the best OSS-LMS based on the two ways was 'Moodle' software package. Among the scores of groups in the objective validation, significant differences were identified; this indicated that the ranking results of internal and external VIKOR group decision making were valid, which pointed to the validation of the framework. ? 2021 World Scientific Publishing Company. |
| References |
Abdulkareem, K. H., Arbaiy, N., Zaidan, A. A., Zaidan, B. B., Albahri, O. S., Alsalem, M. A., & Salih, M. M. (2021). A new standardisation and selection framework for real-time image dehazing algorithms from multi-foggy scenes based on fuzzy delphi and hybrid multi-criteria decision analysis methods. Neural Computing and Applications, 33(4), 1029-1054. doi:10.1007/s00521-020-05020-4 Abdullateef, B. N., Elias, N. F., Mohamed, H., Zaidan, A. A., & Zaidan, B. B. (2016). An evaluation and selection problems of OSS-LMS packages. SpringerPlus, 5(1), 1-35. doi:10.1186/s40064-016-1828-y Abdullateef, B. N., Elias, N. F., Mohamed, H., Zaidan, A. A., & Zaidan, B. B. (2016). An evaluation and selection problems of OSS-LMS packages. SpringerPlus, 5(1), 1-35. doi:10.1186/s40064-016-1828-y Adewumi, A., Misra, S., Omoregbe, N., & Sanz, L. F. (2019). FOSSES: Framework for open-source software evaluation and selection. Software - Practice and Experience, 49(5), 780-812. doi:10.1002/spe.2682 Alaa, M., Albakri, I. S. M. A., Singh, C. K. S., Hammed, H., Zaidan, A. A., Zaidan, B. B., . . . Jasim, A. N. (2019). Assessment and ranking framework for the english skills of pre-service teachers based on fuzzy delphi and TOPSIS methods. IEEE Access, 7, 126201-126223. doi:10.1109/ACCESS.2019.2936898 Albahri, A. S., Zaidan, A. A., Albahri, O. S., Zaidan, B. B., & Alsalem, M. A. (2018). Real-time fault-tolerant mHealth system: Comprehensive review of healthcare services, opens issues, challenges and methodological aspects. Journal of Medical Systems, 42(8) doi:10.1007/s10916-018-0983-9 Albahri, O. S., Albahri, A. S., Mohammed, K. I., Zaidan, A. A., Zaidan, B. B., Hashim, M., & Salman, O. H. (2018). Systematic review of real-time remote health monitoring system in triage and priority-based sensor technology: Taxonomy, open challenges, motivation and recommendations. Journal of Medical Systems, 42(5) doi:10.1007/s10916-018-0943-4 Albahri, O. S., Albahri, A. S., Zaidan, A. A., Zaidan, B. B., Alsalem, M. A., Mohsin, A. H., . . . Shareef, A. H. (2019). Fault-tolerant mHealth framework in the context of IoT-based real-time wearable health data sensors. IEEE Access, 7, 50052-50080. doi:10.1109/ACCESS.2019.2910411 Albahri, O. S., Zaidan, A. A., Zaidan, B. B., Hashim, M., Albahri, A. S., & Alsalem, M. A. (2018). Real-time remote health-monitoring systems in a medical centre: A review of the provision of healthcare services-based body sensor information, open challenges and methodological aspects. Journal of Medical Systems, 42(9) doi:10.1007/s10916-018-1006-6 Almahdi, E. M., Zaidan, A. A., Zaidan, B. B., Alsalem, M. A., Albahri, O. S., & Albahri, A. S. (2019). Mobile patient monitoring systems from a benchmarking aspect: Challenges, open issues and recommended solutions. Journal of Medical Systems, 43(7) doi:10.1007/s10916-019-1336-z Al-Rawashdeh, T. A. (2015). Evaluating open source software usability using a multistage fuzzy model approach. International Review on Computers and Software, 10(10), 1018-1026. doi:10.15866/irecos.v10i10.7668 Bahamdain, S. S. (2015). Open source software (OSS) quality assurance: A survey paper. Paper presented at the Procedia Computer Science, , 56(1) 459-464. doi:10.1016/j.procs.2015.07.236 Retrieved from www.scopus.com Ćetin, G., & Mehmet, G. (2008). A measurement based framework for assessment of usability-centricness of open source software projects. Paper presented at the SITIS 2008 - Proceedings of the 4th International Conference on Signal Image Technology and Internet Based Systems, 585-592. doi:10.1109/SITIS.2008.106 Retrieved from www.scopus.com Côté, M. -., Suryn, W., & Georgiadou, E. (2007). In search for a widely applicable and accepted software quality model for software quality engineering. Software Quality Journal, 15(4), 401-416. doi:10.1007/s11219-007-9029-0 Dawood, K. A., Sharif, K. Y., Zaidan, A. A., Abd Ghani, A. A., Zulzalil, H. B., & Zaidan, B. B. (2019). Mapping and analysis of open source software (OSS) usability for sustainable OSS product. IEEE Access, 7, 65913-65933. doi:10.1109/ACCESS.2019.2914368 Dawood, K. A., Sharif, K. Y., Zaidan, A. A., Abd Ghani, A. A., Zulzalil, H. B., & Zaidan, B. B. (2019). Mapping and analysis of open source software (OSS) usability for sustainable OSS product. IEEE Access, 7, 65913-65933. doi:10.1109/ACCESS.2019.2914368 Fenton, N. E., & Neil, M. (2000). Software metrics: Roadmap. Paper presented at the Proceedings of the Conference on the Future of Software Engineering, ICSE 2000, 357-370. doi:10.1145/336512.336588 Retrieved from www.scopus.com Fernández-Pérez, Y., Febles-Estrada, A., Cruz, C., & Verdegay, J. L. (2018). Fuzzy multi-criteria decision making methods applied to usability software assessment: An annotated bibliography doi:10.1007/978-3-319-69989-9_11 Retrieved from www.scopus.com Hauge, O. (2007). Open source software in software intensive industry - A survey. Technical Report, Retrieved from www.scopus.com Hedberg, H., Iivari, N., Rajanen, M., & Harjumaa, L. (2007). Assuring quality and usability in open source software development. Paper presented at the First International Workshop on Emerging Trends in FLOSS Research and Development, FLOSS'07, 2-6. doi:10.1109/FLOSS.2007.2 Retrieved from www.scopus.com Iivari, N. (2008). Usability in open source software development - an interpretive case study. Paper presented at the 16th European Conference on Information Systems, ECIS 2008, Retrieved from www.scopus.com Iivari, N., Rajanen, M., & Hedberg, H. (2014). Encouraging for enculturation - an enquiry on the effort of usability specialists entering OSS projects. Paper presented at the Proceedings of the 25th Australasian Conference on Information Systems, ACIS 2014, Retrieved from www.scopus.com Joia, L. A., & dos Santos Vinhais, J. C. (2017). From closed source to open source software: Analysis of the migration process to open office. Journal of High Technology Management Research, 28(2), 261-272. doi:10.1016/j.hitech.2017.10.008 Jusoh, Y., Chamili, K., Yahaya, J. H., & Pa, N. C. (2012). The selection criteria of open source software adoption in malaysia. International Journal of Advancements in Computing Technology, 4(21), 278-287. doi:10.4156/ijact.vol4.issue21.33 Jusoh, Y., Chamili, K., Yahaya, J. H., & Pa, N. C. (2012). The selection criteria of open source software adoption in malaysia. International Journal of Advancements in Computing Technology, 4(21), 278-287. doi:10.4156/ijact.vol4.issue21.33 Jusoh, Y. Y., Chamili, K., Pa, N. C., & Yahaya, J. H. (2014). Open source software selection using an analytical hierarchy process (ahp). American Journal of Software Engineering and Applications, 3(6), 83-89. Retrieved from www.scopus.com Jusoh, Y. Y., Chamili, K., Pa, N. C., & Yahaya, J. H. (2014). Open source software selection using an analytical hierarchy process (ahp). American Journal of Software Engineering and Applications, 3(6), 83-89. Retrieved from www.scopus.com Kannan, A. S. K., Balamurugan, S. A. A., & Sasikala, S. (2021). A novel software package selection method using teaching-learning based optimization and multiple criteria decision making. IEEE Transactions on Engineering Management, 68(4), 941-954. doi:10.1109/TEM.2019.2918050 Keeney, R. L., & Raiffa, H. (1976). Decisions with Multiple Objectives: Preferences and Value Tradeoffs, Retrieved from www.scopus.com Khatari, M., Zaidan, A. A., Zaidan, B. B., Albahri, O. S., & Alsalem, M. A. (2019). Multi-criteria evaluation and benchmarking for active queue management methods: Open issues, challenges and recommended pathway solutions. International Journal of Information Technology and Decision Making, 18(4), 1187-1242. doi:10.1142/S0219622019300039 Khatari, M., Zaidan, A. A., Zaidan, B. B., Albahri, O. S., & Alsalem, M. A. (2019). Multi-criteria evaluation and benchmarking for active queue management methods: Open issues, challenges and recommended pathway solutions. International Journal of Information Technology and Decision Making, 18(4), 1187-1242. doi:10.1142/S0219622019300039 Khosravi, K., & Guéhéneuc, Y. -. (2005). Open issues with quality models. Proceedings of the 9th QAOOSE Workshop(ECOOP), Retrieved from www.scopus.com Kiah, M. L. M., Haiqi, A., Zaidan, B. B., & Zaidan, A. A. (2014). Open source EMR software: Profiling, insights and hands-on analysis. Computer Methods and Programs in Biomedicine, 117(2), 360-382. doi:10.1016/j.cmpb.2014.07.002 Kou, G., Ergu, D., & Shang, J. (2014). Enhancing data consistency in decision matrix: Adapting hadamard model to mitigate judgment contradiction. European Journal of Operational Research, 236(1), 261-271. doi:10.1016/j.ejor.2013.11.035 Kou, G., & Lin, C. (2014). A cosine maximization method for the priority vector derivation in AHP. European Journal of Operational Research, 235(1), 225-232. doi:10.1016/j.ejor.2013.10.019 Lee, Y. M., Kim, J. B., Choi, I. W., & Rhew, S. Y. (2007). A study on selection process of open source software. Paper presented at the Proceedings - ALPIT 2007 6th International Conference on Advanced Language Processing and Web Information Technology, 568-571. doi:10.1109/ALPIT.2007.83 Retrieved from www.scopus.com Llerena, L., Rodriguez, N., Castro, J. W., & Acuña, S. T. (2019). Adapting usability techniques for application in open source software: A multiple case study. Information and Software Technology, 107, 48-64. doi:10.1016/j.infsof.2018.10.011 Malczewski, J. (1999). GIS and Multicriteria Decision Analysis, Retrieved from www.scopus.com Marichal, J. -. (1998). Aggregation operators for multicriteria decision aid. Aggregation Operators for Multicriteria Decision Aid, Retrieved from www.scopus.com Masson, A. L., Lalanne, D., & Amstutz, T. (2017). A usability refactoring process for large-scale open source projects: The ILIAS case study. Paper presented at the Conference on Human Factors in Computing Systems - Proceedings, , Part F127655 1135-1143. doi:10.1145/3027063.3053345 Retrieved from www.scopus.com Mohammed, K. I., Jaafar, J., Zaidan, A. A., Albahri, O. S., Zaidan, B. B., Abdulkareem, K. H., . . . Alamoodi, A. H. (2020). A uniform intelligent prioritisation for solving diverse and big data generated from multiple chronic diseases patients based on hybrid decision-making and voting method. IEEE Access, 8, 91521-91530. doi:10.1109/ACCESS.2020.2994746 Mohammed, K. I., Zaidan, A. A., Zaidan, B. B., Albahri, O. S., Albahri, A. S., Alsalem, M. A., & Mohsin, A. H. (2020). Novel technique for reorganisation of opinion order to interval levels for solving several instances representing prioritisation in patients with multiple chronic diseases. Computer Methods and Programs in Biomedicine, 185 doi:10.1016/j.cmpb.2019.105151 Mohammed, K. I., Zaidan, A. A., Zaidan, B. B., Albahri, O. S., Albahri, A. S., Alsalem, M. A., & Mohsin, A. H. (2020). Novel technique for reorganisation of opinion order to interval levels for solving several instances representing prioritisation in patients with multiple chronic diseases. Computer Methods and Programs in Biomedicine, 185 doi:10.1016/j.cmpb.2019.105151 Napi, N. M., Zaidan, A. A., Zaidan, B. B., Albahri, O. S., Alsalem, M. A., & Albahri, A. S. (2019). Medical emergency triage and patient prioritisation in a telemedicine environment: A systematic review. Health and Technology, 9(5), 679-700. doi:10.1007/s12553-019-00357-w Nichols, D. M., & Twidale, M. B. (2006). The usability of open source software: Analysis and prospects. Open Source Softw.Business: Issues Perspectives, , 167-188. Retrieved from www.scopus.com Nielsen, J. (1993). Usability Engineering, Retrieved from www.scopus.com Padayachee, I., Kotze, P., & van Der Merwe, A. (2010). ISO 9126 external systems quality characteristics, sub-characteristics and domain specific criteria for evaluating e-learning systems. ISO 9126 External Systems Quality Characteristics, Sub-Characteristics and Domain Specific Criteria for Evaluating E-Learning Systems, Retrieved from www.scopus.com Paul, C. L. (2009). A survey of usability practices in free/libre/open source software doi:10.1007/978-3-642-02032-2_23 Retrieved from www.scopus.com Petrovic-Lazarevic, S., & Abraham, A. (2004). Hybrid Fuzzy-Linear Programming Approach for Multi Criteria Decision Making Problems, Retrieved from www.scopus.com Rahmatullah, B., Zaidan, A. A., Mohamed, F., & Sali, A. (2017). Multi-complex attributes analysis for optimum GPS baseband receiver tracking channels selection. Paper presented at the 2017 4th International Conference on Control, Decision and Information Technologies, CoDIT 2017, , 2017-January 1084-1088. doi:10.1109/CoDIT.2017.8102743 Retrieved from www.scopus.com Rajanen, M., & Iivari, N. (2010). Traditional usability costs and benefits - fitting them into open source software development. Paper presented at the 18th European Conference on Information Systems, ECIS 2010, Retrieved from www.scopus.com Rajanen, M., Iivari, N., & Keskitalo, E. (2012). Introducing usability activities into open source software development projects - A participative approach. Paper presented at the NordiCHI 2012: Making Sense through Design - Proceedings of the 7th Nordic Conference on Human-Computer Interaction, 683-692. doi:10.1145/2399016.2399120 Retrieved from www.scopus.com Raza, A., & Capretz, L. F. (2012). Do open source software developers listen to their users? First Monday, 17(3) doi:10.5210/fm.v17i3.3640 Raza, A., Capretz, L. F., & Ahmed, F. (2012). An open source usability maturity model (OS-UMM). Computers in Human Behavior, 28(4), 1109-1121. doi:10.1016/j.chb.2012.01.018 Raza, A., Capretz, L. F., & Ahmed, F. (2012). Users' perception of open source usability: An empirical study. Engineering with Computers, 28(2), 109-121. doi:10.1007/s00366-011-0222-1 Salih, M. M., Zaidan, B. B., Zaidan, A. A., & Ahmed, M. A. (2019). Survey on fuzzy TOPSIS state-of-the-art between 2007 and 2017. Computers and Operations Research, 104, 207-227. doi:10.1016/j.cor.2018.12.019 Sanga, C. (2010). A Technique for the Evaluation of Free and Open Source e-Learning Systems, Retrieved from www.scopus.com Sarrab, M., & Rehman, O. M. H. (2014). Empirical study of open source software selection for adoption, based on software quality characteristics. Advances in Engineering Software, 69, 1-11. doi:10.1016/j.advengsoft.2013.12.001 Sarrab, M., & Rehman, O. M. H. (2014). Empirical study of open source software selection for adoption, based on software quality characteristics. Advances in Engineering Software, 69, 1-11. doi:10.1016/j.advengsoft.2013.12.001 Sarrab, M., & Rehman, O. M. H. (2013). Selection criteria of open source software: First stage for adoption. International Journal of Information Processing and Management, 4(4), 51-58. doi:10.4156/ijipm.vol4.issue4.6 Sbai, N., Lenarduzzi, V., Taibi, D., Sassi, S. B., & Ghezala, H. H. B. (2018). Exploring information from OSS repositories and platforms to support OSS selection decisions. Information and Software Technology, 104, 104-108. doi:10.1016/j.infsof.2018.07.009 Sethanandha, B. D., Massey, B., & Jones, W. (2010). Managing open source contributions for software project sustainability. Paper presented at the PICMET '10 - Portland International Center for Management of Engineering and Technology, Proceedings - Technology Management for Global Economic Growth, 2329-2337. Retrieved from www.scopus.com Talal, M., Zaidan, A. A., Zaidan, B. B., Albahri, O. S., Alsalem, M. A., Albahri, A. S., . . . Alaa, M. (2019). Comprehensive review and analysis of anti-malware apps for smartphones. Telecommunication Systems, 72(2), 285-337. doi:10.1007/s11235-019-00575-7 Tractinsky, N. (2018). The usability construct: A dead end? Human-Computer Interaction, 33(2), 131-177. doi:10.1080/07370024.2017.1298038 Tzeng, G. -., & Huang, J. -. (2011). Multiple attribute decision making: Methods and applications. Multiple attribute decision making: Methods and applications (pp. 1-333) Retrieved from www.scopus.com Viorres, N., Xenofon, P., Stavrakis, M., Vlachogiannis, E., Koutsabasis, P., & Darzentas, J. (2007). Major HCI challenges for open source software adoption and development doi:10.1007/978-3-540-73257-0_50 Retrieved from www.scopus.com Wesson, J. (2003). Usability issues for e-commerce and e-learning: A developing country perspective. Proc.Int.Workshop on Utility, Usability and Complexity of e-Information Systems, , 43-55. Retrieved from www.scopus.com Yas, Q. M., Zadain, A. A., Zaidan, B. B., Lakulu, M. B., & Rahmatullah, B. (2017). Towards on develop a framework for the evaluation and benchmarking of skin detectors based on artificial intelligent models using multi-criteria decision-making techniques. International Journal of Pattern Recognition and Artificial Intelligence, 31(3) doi:10.1142/S0218001417590029 Yas, Q. M., Zaidan, A. A., Zaidan, B. B., Rahmatullah, B., & Abdul Karim, H. (2018). Comprehensive insights into evaluation and benchmarking of real-time skin detectors: Review, open issues & challenges, and recommended solutions. Measurement: Journal of the International Measurement Confederation, 114, 243-260. doi:10.1016/j.measurement.2017.09.027 Yusop, N. S. M., Grundy, J., Schneider, J. -., & Vasa, R. (2020). How usability defects defer from non-usability defects?: A case study on open source projects. International Journal on Advanced Science, Engineering and Information Technology, (1), 98-105. doi:10.18517/ijaseit.10.1.10225 Zaidan, A. A., Zaidan, B. B., Al-Haiqi, A., Kiah, M. L. M., Hussain, M., & Abdulnabi, M. (2015). Evaluation and selection of open-source EMR software packages based on integrated AHP and TOPSIS. Journal of Biomedical Informatics, 53, 390-404. doi:10.1016/j.jbi.2014.11.012 Zaidan, A. A., Zaidan, B. B., Alsalem, M. A., Albahri, O. S., Albahri, A. S., & Qahtan, M. Y. (2020). Multi-agent learning neural network and bayesian model for real-time IoT skin detectors: A new evaluation and benchmarking methodology. Neural Computing and Applications, 32(12), 8315-8366. doi:10.1007/s00521-019-04325-3 Zaidan, A. A., Zaidan, B. B., Alsalem, M. A., Momani, F., & Zughoul, O. (2020). Novel multiperspective hiring framework for the selection of software programmer applicants based on AHP and group TOPSIS techniques. International Journal of Information Technology and Decision Making, 19(3), 775-847. doi:10.1142/S0219622020500121 Zaidan, A. A., Zaidan, B. B., Hussain, M., Al-Haiqi, A. M., Mat Kiah, M. L., & Abdulnabi, M. (2015). Multi-criteria analysis for OS-EMR software selection problem: A comparative study. Decision Support Systems, 78, 15-27. doi:10.1016/j.dss.2015.07.002 Zaidan, B. B., & Zaidan, A. A. (2017). Software and hardware FPGA-based digital watermarking and steganography approaches: Toward new methodology for evaluation and benchmarking using multi-criteria decision-making techniques. Journal of Circuits, Systems and Computers, 26(7) doi:10.1142/S021812661750116X Zaidan, B. B., Zaidan, A. A., Karim, H. A., & Ahmad, N. N. (2017). A new digital watermarking evaluation and benchmarking methodology using an external group of evaluators and multi-criteria analysis based on ‘large-scale data’. Software - Practice and Experience, 47(10), 1365-1392. doi:10.1002/spe.2465 Zaidan, B. B., Zaidan, A. A., Karim, H. A., & Ahmad, N. N. (2017). A new digital watermarking evaluation and benchmarking methodology using an external group of evaluators and multi-criteria analysis based on ‘large-scale data’. Software - Practice and Experience, 47(10), 1365-1392. doi:10.1002/spe.2465 Zhao, L., Deek, F. P., & McHugh, J. A. (2010). Exploratory inspection-a user-based learning method for improving open source software usability. Journal of Software Maintenance and Evolution, 22(8), 653-675. doi:10.1002/smr.455 Zionts, S. (1979). MCDM-if not a roman numeral, then what? Interfaces, 9(4), 94-101. Retrieved from www.scopus.com Zughoul, O., Momani, F., Almasri, O. H., Zaidan, A. A., Zaidan, B. B., Alsalem, M. A., . . . Hashim, M. (2018). Comprehensive insights into the criteria of student performance in various educational domains. IEEE Access, 6, 73245-73264. doi:10.1109/ACCESS.2018.2881282 |
| This material may be protected under Copyright Act which governs the making of photocopies or reproductions of copyrighted materials. You may use the digitized material for private study, scholarship, or research. |