Optimal reliability and validity of measurement model in confirmatory factor analysis: different likert point scale experiment

Mohd Azry Abdul Malik

Abstract

Designing a questionnaire is one of the most difficult challenges in research design, particularly when deciding which level of Likert point scale is appropriate for the instrumentation. Suitable Likert point scale used in the instrumentation able to reduce the risk of facing potential problems of not achieving reliability (indicator and internal consistency reliability) and validity (convergent validity, discriminating validity, and construct validity) and simultaneously preventing the occurrence of multicollinearity. This study compares the performance of reliability and validity of measurement construct in Confirmatory Factor Analysis (CFA) by using Likert 5-point Likert scale, 6-point Likert scale, 7-point Likert scale, 9-point Likert scale, and 10-point Likert scale. The study uses primary data based on a questionnaire data collection method which involves 100 samples from similar population characteristics for each Likert. The data were analysed using Smart-PLS software. The results suggest that expanding the range of the Likert point scale optimizes the performance of reliability and validity of the measurement model. This study offers an insight to researchers in deciding the best choice of Likert point scale to adapt in instrumentation for a better result in the quantitative analysis process.

References

Adelson, J. L., & McCoach, D. B. (2010). Measuring The Mathematical Attitudes of Elementary Students: The Effects of A 4-Point or 5-Point Likert-Type Scale. Educational and Psychological measurement, 70(5), 796-807. https://doi.org/10.1177/0013164410366694

Awang, Z., Afthanorhan, A., & Mamat, M. (2016). The Likert scale analysis using parametric based Structural Equation Modeling (SEM). Computational Methods in Social Sciences, 4(1), 13. https://doi.org/10.5539/mas.v9n9p58

Chachamovich, E., Fleck, M. P., & Power, M. (2009). Literacy Affected Ability to Adequately Discriminate Among Categories in Multipoint Likert Scales. Journal of Clinical Epidemiology, 62(1), 37-46. https://doi.org/10.1016/j.jclinepi.2008.03.002

Chomeya, R. (2010). Quality of psychology test between Likert scale 5 and 6 points. Journal of Social Sciences, 6(3), 399-403. https://doi.org/10.3844/jssp.2010.399.403

Cummins, R. A., & Gullone, E. (2000, March). Why We Should Not Use 5-point likert scales: The Case for Subjective Quality of Life Measurement. In Proceedings, second international conference on quality of life in cities, 74-93.

Dawes, J. (2008). Do Data Characteristics Change According to The Number of Scale Points Used. International journal of market research, 50(1), 61-77. https://doi.org/10.1177/147078530805000106

Hair, J.F., Hult G.T.M, Ringle, C.M., Sarstedt, M. (2017). A Primer on Partial Least Square Structural Equation Modelling (PLS-SEM) (2 ed.). Thousand Oaks, CA: Sage. ISBN 9781483377445.

Hamid, M. R. A., Sami, W., & Sidek, M. H. M. (2017). Discriminant Validity Assessment: Use of Fornell & Larcker criterion versus HTMT Criterion. Journal of Physics: Conference Series, 890, 012163. doi: 10.1088/1742-6596/890/1/012163

Joshi, A., Kale, S., Chandel, S., & Pal, D. K. (2015). Likert scale: Explored and explained. British Journal of Applied Science & Technology, 7(4), 396. https://doi.org/10.9734/bjast/2015/14975

McDonald, R.P. and Ho, M.-H.R. (2002). Principles and Practice in Reporting Statistical Equation Analyses. Psychological Methods, 7 (1), 64-82. https://doi.org/10.1037/1082-989x.7.1.64

Norman, G. (2010). Likert scales, levels of measurement and the “laws” of statistics. Advances in Health Sciences Education, 15(5), 625-632. https://doi.org/10.1007/s10459-010-9222-y

Preston, C. C., & Colman, A. M. (2000). Optimal Number of Response Categories in Rating Scales: Reliability, Validity, Discriminating Power, And Respondent Preferences. Acta psychologica, 104(1), 1-15. https://doi.org/10.1016/s0001-6918(99)00050-5

Sekaran, U., & Bougie, R. (2010). Research methods for business: A skill building approach (5th ed). Chichester: John Wiley & Sons Ltd.

Valerie, F. (2012). Re-discovering the PLS approach in management science. Management, 15(1), 101-123. https://doi.org/10.3917/mana.151.0102

Wuensch, Karl L. (2005). "What is a Likert Scale? and How Do You Pronounce 'Likert?'". East Carolina University.