UPSI Digital Repository (UDRep)

A CEFR-informed English test: how Content Validity Index and Content Validity Ratio are used for content validation

AVAILABLE Article 100 Views
Scan QR Code to share or view item link:

Digital Assets & Download Files
Login required to access this item.
General Information Cached
Title
A CEFR-informed English test: how Content Validity Index and Content Validity Ratio are used for content validation
Type
Article
Main Author
Kamarul Ariffin Ahmad
Additional Authors
  • Izazol Idris
Subject
PE English
ISSN
2289-9391
Language Code
eng
Bibliographic Details
Place of Production
Tanjong Malim
Publisher
UPSI Press
Year of publication
2025
Notes
Vol. 11 No. 1 (2025): EDUCATUM Journal of Social Sciences (EJoSS)
Degree Type
Others
Corporate Name
Perpustakaan Tuanku Bainun

Abstract

Current education roadmap in Malaysia has seen enormous initiatives taken by the government to ensure the education system is at par with the international standard where the English syllabus and assessments are tailored according to the Common European Framework of Reference (CEFR). This includes higher learning institutions, thus this research aimed to develop the English entrance test that is CEFR-informed. 11 CEFR experts were involved in the study where they reviewed 60 multiple choice questions for reading, grammar and reading comprehension. The results showed that all items were accepted and valid based on the computation of Content Validity Index (CVI) and Content Validity Ratio (CVR). The lowest CVI recorded was .82 with S-CVI of .95 and the lowest CVR was .64 which were above the suggested cut-off value by experts and research. The study concluded that all the items were carefully developed based on the framework and the needs of the test. Keywords: CEFR, Content Validity Index (CVI), Content Validity Ratio (CVR), English Entrance Examination

References

Adom, D., Adu-Mensah, J., & Dake, D. A. (2020). Test, measurement, and evaluation: Understanding and use of the concepts in education. International Journal of Evaluation and Research in Education (IJERE), 9(1), 109. https://doi.org/10.11591/ijere.v9i1.20457 

 

Ahmad Tamizi Abu, Rabiatul Adawiah Ahmad Rashid, & Salmiza Saleh. (2021). Al-Alaq Teaching Module: The Process for Evaluating Content Validity and Its Feasibility. Journal of Contemporary Social Science and Education Studies (JOCSSES), 1(2), 175–184. Retrieved from https://jocss.com/index.php/multidiscipline/article/view/7 

 

Ahorsu, D. K., Lin, C.-Y., Imani, V., Saffari, M., Griffiths, M. D., & Pakpour, A. H. (2020). The fear of covid-19 scale: Development and initial validation. International Journal of Mental Health and Addiction, 20(3), 1537–1545. https://doi.org/10.1007/s11469-020-00270-8 Almanasreh, E., Moles, R., & Chen, T. F. (2019). Evaluation of methods used for estimating content validity. Research in Social and Administrative Pharmacy, 15(2), 214–221. https://doi.org/10.1016/j.sapharm.2018.03.066

 

Aryadoust, V., Ng, L. Y., & Sayama, H. (2020). A comprehensive review of rasch measurement in Language assessment: Recommendations and guidelines for research. Language Testing, 38(1), 6–40. https://doi.org/10.1177/0265532220927487 

 

Babo, R., V. Babo, L., T Suhonen, J., & Tukiainen, M. (2020). E- assessment with multiple-choice questions: A 5 year study of students’ opinions and experience. Journal of Information Technology Education: Innovations in Practice, 19, 001–029. https://doi.org/10.28945/4491 

 

Cohen-Swerdlik (2009). Psychological Testing and Assessment: An Introduction to Tests and Measurement (7th ed.). The United States of America: McGraw-Hill. 

 

Haladyna, T.M., (2004). Developing and Validating Multiple-Choice Test Items (3rd ed.). London: Lawrence Erlbaum. 

 

Hasimah Ja’afar. (2020, February). Discussion on the Current Instrument for Entrance Examination. Personal Interview. 

 

Kamarul Ariffin Ahmad, (2018). Item Analysis and Score Dependability of English Proficiency 1 Examination in a Public University (Master’s Dissertation). University of Malaya, Kuala Lumpur. 

 

Kamarul Ariffin Ahmad, & Zahari Suppian. (2022). International and National Teacher Education Conference (INTEC 2022). In Teacher Education in The Next Normal (pp. 117–126). Hat Yai; Prince of Songkla University. 

 

Kementerian Pendidikan Malaysia, Malaysia Education Blueprint, 2013-2025: Preschool to post-secondary education (2013). Putrajaya. 

 

Lynn, M. R. (1986). Determination and quantification of content validity. Nursing research, 35(6), 382-386. 

 

Massitah Kipli, & Ahmad Zamri Khairani. (2020). Content Validity Index: An application of validating CIPP instrument for programme evaluation. Int Multidiscip Res J, 2(4), 31-40. 

 

Mohd Effendi, Ahmad Zamri, Hisyamsani Idris & Normawati Rahman. (2017). Kesahan Kandungan Pakar Instrumen IKBAR Bagi Pengukuran AQ Menggunakan Nisbah Kesahan Kandungan. 

 

Nor Ashiquin C. Alih, Masdinah Alauyah Md Yusoff, & Abdul Halim Abdul Raof. (2020). Teachers’ Knowledge and Belief on the CEFR Implementation in Malaysian ESL Classroom. International Journal of Multidisciplinary and Current Educational Research (IJMCER), 2(5), 126–134. 

 

Norshahira Osman, Norashida, S. R., & Lukman, Z. M. (2021). Content validity determination of the suicidal ideation behaviour assessment instruments using content validity ratio (CVR) formula. Journal of Drug Delivery and Therapeutics, 11(4-S), 81–85. https://doi.org/10.22270/jddt.v11i4-s.4968 

 

Rachmat, N. A., & Arfiandhani, P. (2019). “I use multiple-choice question in most assessment I prepared”: EFL teachers’ voice on summative assessment. ETERNAL (English, Teaching, Learning, and Research Journal), 5(1), 163. https://doi.org/10.24252/eternal.v51.2019.a13 

 

Shrotryia, V. K., & Dhanda, U. (2019). Content validity of assessment instrument for employee engagement. SAGE Open, 9(1), 215824401882175. https://doi.org/10.1177/2158244018821751 

 

Zeraati, M., & Alavi, N. M. (2014). Designing and validity evaluation of quality of nursing care scale in Intensive Care Units. Journal of Nursing Measurement, 22(3), 461–471. https://doi.org/10.1891/1061- 3749.22.3.461 

 

Zuraidah Mohd Don, & Mardziah Hayati Abdullah. (2019). What the CEFR is and isn’t. Kementerian Pendidikan Malaysia. https://www.moe.gov.my/en/menumedia/printed-media/newspaper-clippings/what-the-cefr-is-and-isn-t-free-malaysia-today-27-mei-2019

 


Installed and configured by Bahagian Automasi, Perpustakaan Tuanku Bainun, Universiti Pendidikan Sultan Idris
If you have enquiries, kindly contact us at pustakasys@upsi.edu.my or 016-3630263. Office hours only.