Analysis of validity and reliability of economic achievement test based on Rasch measurement model

Khoo, Yin Yin

QR Code Link :
Type :	article
Subject :	LB Theory and practice of education
ISSN :	2226-3624
Main Author :	Khoo, Yin Yin
Additional Authors :	Noornadiah Md. Sari
Title :	Analysis of validity and reliability of economic achievement test based on Rasch measurement model

Place of Production :	Tanjong Malim
Publisher :	Fakulti Pengurusan dan Ekonomi
Year of Publication :	2021
Corporate Name :	Universiti Pendidikan Sultan Idris

Abstract : Universiti Pendidikan Sultan Idris

Teachers regularly attend assessments on students to identify students’ levels of mastery. Achievement tests as a quality measurement tool are quintessential so that the conclusions obtained are reliable and significant. Therefore, high-quality achievement tests need to satisfy specific criteria by going through standard procedures. Nonetheless, time and competency constraints lead teachers to utilise economic test questions that do not reach specific standards. Therefore, this research intended to develop and test the validity and reliability of economic achievement tests. An economic achievement test instrument was developed, consisting of 30 objective questions based on Bloom’s taxonomy. The testing of the instrument involved 40 respondents of Form Six economics students. The researchers appointed five experts to evaluate the validity of the content of the achievement test questions. At the same time, the construct validity and instrument reliability test analysis involved item-respondent reliability analysis, item-respondent separation index, Cronbach’s alpha, item polarity, item fit, standardised residual item correlation and respondent itemability difficulty level distribution using Rasch measurement approach through Winsteps software 3.72.3. The data of the tests conducted determined that the achievement test confirmed good content validity and reliability values. The analysis also established that six questions needed to be modified. The development of the economic achievement test offers an alternative measurement design over future performance test testing. The researchers proposed that the implementation of this measurement on other subjects too. Keywords: Rasch Measurement Model, Validity and Reliability, Economic Achievement Test, Academic Achievement.

References

Abu Bakar, N., & Bhasah, A. B. (2008). Penaksiran dalam pendidikan & sains sosial. Penerbit Universiti Pendidikan Sultan Idris.

Adom, D., Mensah, J. A., & Dake, D. A. (2020). Test, measurement and evaluation: Understanding and use of the concepts in education. International Journal of Evaluation and Research in Education, 9(1), 109-119. https://doi.org/10.11591/ijere.v9i1.20457

Amua-Sekyi, E. T. (2016). Assessment, student learning and classroom practice: A review. Journal of Education and Practice, 7 (21).

Arumugham, K. S. (2020). Curriculum, teaching and assessment in the perspective of classroom assessment. Asian People Journal, 3(1), 152-161.

Azarilah, A. A., Saidfudin, M., & Azami, Z. (2013). Asas model pengukuran rasch: Pembentukan skala & struktur pengukuran. Penerbit Universiti Kebangsaan Malaysia.

Bambang, S. (2017). Rasch Model Measurement as Tools in Assessment for Learning. International Conference on Educational Innovation (ICEI 2017), Wyndham Hotel, Surabaya, Indonesia. https://doi.org/10.2991/icei-17.2018.11

Bambang, S., & Wahyu, W. (2014). Aplikasi model rasch untuk penelitian ilmu-ilmu sosial. Trim Komunikata Publishing House.

S., & Wahyu, W. (2015). Aplikasi pemodelan rasch pada assessment pendidikan. Trim Komunikata Publishing House.

Black, P., & Wiliam, D. (1998). Assessment and classroom learning. Assessment in Education: Principles, Policy & Practice, 5(1), 7–74. https://doi.org/10.1080/0969595980050102

Bloom, B. S., Hastings, J. T., & Madaus, G. F. (Eds) (1971). Handbook on the formative and summative evaluation of student learning. McGraw-Hill.

Bond, T. G., & Fox, C. M. (2015). Applying the rasch model: Fundamental measurement in the human sciences (3rd ed.). Lawrence Erlbaum Associates.

Boone, W. J. (2016). Rasch analysis for instrument development: Why, when, and how? CBE—Life Sciences Education, 15(4). https://doi.org/10.1187/cbe.16-04-0148

Broadfoot, P., & Black, P. (2004). Redefining assessment? The first ten years of assessment in education. Assessment in Education: Principles, Policy & Practice, 11(1), 7-26, 10.1080/0969594042000208976

Browne, R.H. (1995). On the use of a pilot study for sample size determination. Statistics in Medicine, 14, 1933-1940.

Cecilio-Fernandes, D., Cohen-Schotanus, J., & Tio, R. A. (2018). Assessment programs to enhance learning. Physical Therapy Reviews, 23(1), 17-20. https://doi.org/10.1080/10833196.2017.1341143

DeLuca, C., & Volante, L. (2016). Assessment for learning in teacher education programs: Navigating the juxtaposition of theory and praxis. Journal of the International Society for Teacher Education, 20 (1), 19-31.

Ebel, R. L., & Frisbie, D. A. (1991). Essentials of educational measurement (5th edition), Prentice-Hall, Englewood Cliffs.

Ellyza, K., & Kamisah, O. (2018). Kesahan dan kebolehpercayaan ujian kemahiran proses sains untuk murid sekolah rendah berdasarkan model pengukuran rasch. Jurnal Pendidikan Malaysia,1-9. http://dx.doi.org/10.17576/JPEN-2018-43.03-01

Fisher Jr., W.P. (2007). Rating scale instrument quality criteria. Rasch Measurement Transaction, 21, 1095. http://www.rasch.org/rmt/rmt211a.htm

Gordanier, J., Hauk, W., & Sankaran, C. (2019). Early intervention in college classes and improved student outcomes. Economics of Education Review, 72, 23–29. https://doi.org/10.1016/j.econedurev.2019.05.003

Huei, O. K., Rus, R. C., & Kamis, A. (2020). Knowledge of design and technology subject: A rasch measurement model approaches for pilot study. International Journal of Academic Research Business and Social Sciences, 10(3), 599–613.

Jimaa, S. (2011). The impact of assessment on students learning. Procedia - Social and Behavioral Sciences, 28, 718–721. https://doi.org/10.1016/j.sbspro.2011.11.133

Kieser, M., & Wassmer, G. (1996). On the use of the upper confidence limit for the variance from a pilot sample for sample size determination. Biometrical Journal, 8, 941-949.

Koretz, D. M. (2002). Limitations in the use of achievement tests as measures of educators’ productivity. The Journal of Human Resources, 37(4), 752. https://doi.org/10.2307/3069616

Linacre, J. M. (2007). A user’s guide to WINSTEPS Rasch-model computer programs. MESA Press.

Linacre, J.M. (2012). User's guide and program manual to WINSTEPS: Rasch model computer programs. MESA Press.

Lopes, J. C., Graça, J. C., & Correia, R. G. (2015). Effects of economic education on social and political values, beliefs and attitudes: Results from a survey in Portugal. Procedia Economics and Finance, 30, 468–475. https://doi.org/10.1016/S2212-5671(15)01314-3

Lynn, M. R. (1986). Determination and quantification of content validity. Nursing research, 35, 378-382.

Majlis Peperiksaan Malaysia (MPM). (2012). Huraian sukatan pelajaran ekonomi. Majlis Peperiksaan Malaysia.

Mclellan, E. (2007). What is a competent competence standard? Quality Assurance in Education, 15(4), 437–448. https://doi.org/10.1108/09684880710829992

McMillan, J. H., & Schumacher, S. (1984). Research In Education. Little, Brown & Company Limited.

Moore, C. G., Carter, R. E., Nietert, P. J., & Stewart, P. W. (2011). Recommendations for planning pilot studies in clinical and translational research. Clinical and Translational Science, 4(5), 332–337. https://doi.org/10.1111/j.1752-8062.2011.00347.x

Mousazadeh, S., Rakhshan, M., & Mohammadi, F. (2017). Investigation of content and face validity and reliability of sociocultural attitude towards appearance questionnaire-3 (SATAQ-3) among female adolescents. Iranian Journal of Psychiatry, 12(1), 15–20.

Nordin, A. R., Zamri, A. K., & Lei, M. T. (2012). Examining quality of mathematics test item using rasch model: Preminarily analysis. Procedia-Social and Behavioral Sciences, 69, 2205-2214.

Okolie, U. C., Igwe, P. A., Nwajiuba, C. A., Mlanga, S., Binuomote, M. O., Nwosu, H. E., & Ogbaekirigwe, C. O. (2020). Does PhD qualification improve pedagogical competence? A study on teaching and training in higher education. Journal of Applied Research in Higher Education, 12(5), 1233–1250. https://doi.org/10.1108/JARHE-02-2019-0049

Osadebe, P. U. (2015). Construction of valid and reliable test for assessment of students. Journal of Education and Practice, 6(1).

Osadebe, P. U. (2018). Assessment of test items with rasch measurement model. Journal of Applied Measurement, 19(1), 106–112.

Owi, K. H., Ridzwan, C. H., & Arasinah, K. (2020). Knowledge of design and technology subject: A rasch measurement model approaches for pilot study. International Journal of Academic Research Business and Social Sciences, 10(3), 599–613. http://dx.doi.org/10.6007/IJARBSS/v10-i3/7075

Polit, D. F., Beck, C. T., & Owen, S. V. (2007). Is the CVI an acceptable indicator of content validity? Appraisal and recommendations. Research in Nursing & Health, 30(4), 459– 467. https://doi.org/10.1002/nur.20199

Rosmawati, M. (2008). Pengesanan dan penggunaan ujian matematik tahun empat sekolah rendah: Analisis rasch [Unpublished doctoral dissertation]. University of Science Malaysia.

Shiel, T. (2017). Chapter 2 building the base: begin with the end in mind. In Designing and Using Performance Tasks: Enhancing Student Learning and Assessment, 25-40. Corwin. https://www-doi-org.ezplib.ukm.my/10.4135/9781506343402.n3

Siti Mistima, M. (2015). Psychometric evaluation on mathematics beliefs instrument using rasch model. Creative Education, 6, 1797-1801.

Stewart, J., & Haswell, K. (2013). Assessing readiness to work in primary health care: The content validity of a self-check tool for physiotherapists and other health professionals. Journal of Primary Health Care, 5(1), 70–73.

Sumaryanta, Mardapi, D., Sugiman, & Herawan, T. (2018). Assessing teacher competence and its follow-up to support professional development sustainability. Journal of Teacher Education for Sustainability, 20 (1), 106-123.

Torabizadeh, C., Yousefinya, A., Zand, F., Rakhshan, M., & Fararooei, M. (2016). A nurses’ alarm fatigue questionnaire: development and psychometric properties. Journal of Clinical Monitoring and Computing, 31(6), 1305–1312. https://doi.org/10.1007/s10877-016-9958-x

Wright, B. D., & Linacre, J. M. (1994). Reasonable mean-square fit values. Rasch Measurement Transactions, 8(3), 370.

Wu, X. V., Enskär, K., Lee, C. C. S., & Wang, W. (2015). A systematic review of clinical assessment for undergraduate nursing students. Nurse Education Today, 35(2), 347–359. https://doi.org/10.1016/j.nedt.2014.11.016

Yan, Z., Li, Z., Panadero, E., Yang, M., Yang, L., & Lao, H. (2021). A systematic review on factors influencing teachers’ intentions and implementations regarding formative assessment. Assessment in Education: Principles, Policy & Practice, 28(3), 228-260. https://doi.org/10.1080/0969594X.2021.1884042

Zaharah, C. I., & Nurulwahida, A. (2021). Analisis statistik kesahan dan kebolehpercayaan ujian pencapaian reka bentuk elektrik. Malaysian Journal of Social Sciences and Humanities, 6(8), 196-206.

This material may be protected under Copyright Act which governs the making of photocopies or reproductions of copyrighted materials.
You may use the digitized material for private study, scholarship, or research.

Back to previous page