Exploring the conceptual and psychometric properties of alternative assessment using the rasch measurement model

Che Md Ghazali Nor Hasnida,

Full Text

The aim of this study is to develop and provide the evidence of psychometric evaluation of an instrument to measure the implementation of an alternative assessment in the context of Islamic education teachings at primary school level. In order to conceptualize alternative assessment implementation, a 32-item questionnaire was designed and administered to a group of 105 primary school teachers teaching Islamic education subject. The respondents were selected through purposive sampling. Content validity is conducted by two experts in the field from an education university. Then, the construct validity and reliability are assessed using the Rasch Measurement Model by identifying the scale rating, uni-dimensionality, item polarity, item fit, item difficulty index, item reliability, person reliability and separation index. The findings revealed that the structure calibration value with the difference of threshold is between 1.4 and 5.0. The values of PMC range from 0.46 to 0.72. All items are in the range of 0.6 to 1.4. Item C17 is removed, and it is the most difficult item. Item reliability is 0.88, person reliability is 0.97 and the separation index is 8.82 for items and 3.68 for person. The overall item quality is good. Obviously, this study does look into the implementation of AA from the sources available, the teaching strategies used by teachers and also the challenges they face in the Malaysian educational context. This validated instrument could be used for real study and also as a self-assessment tools for teachers. Teachers could then determine their strength and weaknesses.

References

1. Aydin, F. (2005). Teacher’s opinions and practises about alternative assessment-evaluation. National Congress on Education Sciences, 20(1), 775-779. 2. Azrilah A. A., Mohd Saifuddin M. & Azami Z. (2013). Asas Model Pengukuran Rasch. Bangi:Penerbit UKM. 3. Bailey, K. M. (2015). Learning about language assessment: dilemmas, decisions, and directions. Boston: National Geographic Learning. 4. Black, P. & Wiliam, D. (1998). Assessment and Classroom Learning. Assessment in Education,Vol. 5, 7-71. 5. Black, P. (1998). Testing, Friend or Foe? The Theory and Practice of Assessment and Testing. US:Falmer Press. 6. Bond, T. G. & Fox, C. M. (2012). Applying the Rasch Model: Fundamental Measurement in the Human Sciences (2nd.) Now York: Routledge. 7. Bond, T. & Fox, C. (2001). Applying the Rasch model: Fundamental measurement in the human sciences. Mahwah, NJ: Lawrence Erlbaum Associates. 8. Broadfoot, P. M. (1996) Education, Assessment and Society: a sociological analysis. British Journal of Sociology of Education, 18(3). 9. Buldur, S. & Tatar, N. (2011). Development of self-efficacy towards using Alternative Assessment Scale. Asia Pacific Education Review, 12, 485-495. 10. Burke, K. (2005). How to Assess Authentic learning. Thousand Oaks, CA: Corwin Press. 11. Cohen, K. M. (1995). Achieving positive attitudes toward science through alternative assessments. USA: Newport University. 12. Danielson, C. (2001). New Trends in Teacher Evaluation, Educational Leadership, 58 (5), 12-15 13. Dietel, R. J., Herman, J. L., & Knuth, R. A. (1991). What does research say about assessment? NCREL, Oak Brook. Available online: http://www.ncrel.org/sdrs/areas/stw_esys/ 4assess.htm (Accessed 16 October 2017). 14. Fensham, P. J. (2004). Beyond Knowledge: Other Outcome Qualities for Science Education. Keynote address delivered at the XIth International Organisation for Science and Technology Education Symposium, July 2004, Lublin, Poland. 15. Fischer, J. (2007). Rasch Measurement Transaction. Rasch Measurement Transaction, 21(1),1095. 16. Fox, C.M. & Jones, J. (1998). Use of Rasch Modelling in counselling psychology research. Journal of Counselling Psychology, 76(4), 569-582. 17. Fullan, M. (2011). Choosing the wrong drivers for whole system reform. Available at: http://www.michaelfullan.ca/media/13501655630.pdf (Accessed: 3 December 2013). 18. Gibbs, G. & Simpson, C. (2004). Conditions under which assessment supports students’ learning. Learning and Teaching in Higher Education , 1, 1–31 19. Gipps, C. V. (2004). Beyond Testing: towards a theory of educational. London: The Falmer Press. 20. Hancock, C.R. (1994). Alternative assessment and second language study: What and why?Available at: https://gse.gmu.edu/assets/docs/forms/mirs/assessment_brochure.pdf(Accessed 16 October 2017). 21. Green, K. E. & Frantom, C. G. (2002). Survey development and validation with the Rasch Model. Paper presented at the International Conference on Questionnaire Development, Evaluation and Testing, Charleston, SC, November, 14-17. 22. Hancock, C. R. (1994). Alternative Assessment and Second Language Study. Heinle: US. 23. Lambert, D. & Lines, D. (2000). Understanding assessment: purposes, perceptions, practice. London and New York: RoutledgeFalmer. 24. Jackson, S. L. (2003). Research Methods and Statistics, A Critical Thinking Approach. USA: Thomson Wadsworth. 25. Kelvin, T. (2007). The case for qualitative approaches to assessment. In Kelvin Tan (Ed.) Alternative Assessment in Schools: A Qualitative Approach. Jurong, Singapore: Prentice Hall. 26. Lambert, D. & David, L. (2000). Understanding assessment: purposes, perceptions, practice. London and New York: RoutledgeFalmer. 27. Linacre, J. M. (2007). A user's guide to WINDTEPS Rasch-model computer programs. Chicago, Illinois: MESA Press. 28. Maslovaty, N. & Kuzi, E. (2002). Promoting motivational goals through alternative or traditional assessment. Studies in Educational Evaluation, 28(3), 199-222. 29. Metin, M. (2013). Teachers’ difficulties in preparation and implementation of Performance task. Educational Services: Theory and Practice, 13(3), 1664-1674. 30. Muijs, D. (2011). Doing Quantitative Research in Education with SPSS. London: SAGE Publications Ltd. 31. Murphy, K. R. & Davidshofer, C. O. (1998). Psychological Testing: Principles and Applications. University of Michigan: Prentice Hall. 32. Nor Hasnida, C. M. G. (2016). An evaluation of the Implementation of the SBA System in Malaysia. (Doctoral Dissertation). Available at : https://eprints.soton.ac.uk/381724/ 33. PMO (2015). Eleventh Malaysian plan. 2016-2020. Available at: https://www.pmo.gov.my/ dokumenattached/speech/files/RMK11_Speech.pdf 34. Rohaya, T. & Mohd Najib, A. G. (2006). Pembinaan dan Pengesahan Instrumen bagi mengukur tahap literasi pentaksiran guru sekolah menengah di Malaysia, 109-125. 35. Russell, M. K. & Airasian, P. W. (2012). Classroom assessment: Concepts and applications (7thed.) New York: McGraw-Hill 36. Saidfudin, M., Azrilah, A. A., Rodzo’an, N. A., Omar, M. Z., Zaharim, A. & Basri, H. (2010). Easier Learning Outcomes Analysis using Rasch Model in Engineering Education Research. EDUCATION'10 Proceedings of the 7th WSEAS International Conference on Engineering Education. 442-447. 37. Salmiah, J. (2013) ‘Acceptance towards SBA among agricultural integrated living skills teachers: challenges in implementing a holistic assessment’, Journal of Technical Education andTraining, 5(1), 44-51. 38. Sasmaz, O. (2014). The alternative assessment evaluation approaches preferred by pre-service teachers and their self-efficacy towards these approaches, Education and Science, 39 (173). 39. Sazmaz, O. & Evrekli, T. (2011). The Science and Technology Pre-Service Teachers’ Self-Efficacy Levels and Opinions about Alternative Assessment and Evaluation Approaches. Educational Services: Theory and Practice, 11(3), 1690-1698. 40. Shepard, L. A. (2000). The role of assessment in a learning culture, Educational Researcher,29(7), 4-14. 41. Slaalvik, E. (2010). Teacher self-efficacy and teacher burnout: A study of relations, Teaching and Teacher Education, 26, 1059-1069. 42. Stiggins, R. J. (2002) ‘Assessment Crisis: The Absence of Assessment For Learning’, Phi Delta Kappan, 83(10), 758-765. 43. Wikstrom, N. (2007). Alternative Assessment in Primary Years of IBE, Available at: http://www. diva-portal.org/smash/get/diva2:199424/FULLTEXT01.pdf 44. Young, S. & Giebelhaus, C. (2005) Formative Assessment and Its Uses for Improving Student Achievement. Education Data Management Solutions, STI. Available at:www.cbohm.com/news/STI/STI_White_Paper.pdf (Accessed Nov 2011).