Measuring the Academic Success of Students with ASICS using Polytomous Item Response Theory

Hishamuddin Ahmad

Abstract

Fully Residential School (FRS) under the Ministry of Education in Malaysia is a type of school, which produces high performance human capital. Important factors in the academic success of high performing students should be focused on to produce more high-performing students to fulfill the needs of the nation. This study aims to apply the Academic Success Inventory for College Students (ASICS) in the context of FRS in Malaysia and measure the instrument quality as well as identify the most important factor in the academic success. The sample comprised 305 students from three FRS. The data was collected using the ASICS instrument, which contained 49 items. Later, the data was analyzed based on the Polytomous Item Response Theory using the Xcalibre software. Based on the Chi-square, p-value, and the -2 LogLikelihood, Samejima's Graded Rating Model was found to be the fit model with the data. Unidimensionality assumption and local independence were tested using the exploratory factor analysis and were fulfilled. The instrument’s reliability was overall very satisfactory (α=0.89) and the construct validity was also fulfilled with the value of 0.86. It was found that most items (91.8%) showed good discrimination. The findings also showed that (i) ASICS is a good instrument to measure academic success among the students as previous studies and (ii) high performing students found to prioritize in aspects such ‘Perceiving Instructor Efficacy’ and ‘Personal Adjustment’ to achieve their academic success. It means, to develop an excellent future generation, we require an efficient education sector, especially one which focuses on efficient teachers for the 4.0 Industrial Age.

References

1. Baker FB (2001). The basics of item response theory. Education Resources Information Center: ERIC, College Park, USA.

2. Cohen RJ, Swerdlik ME, and Smith DK (1992). Psychological testing and assessment: An introduction to tests and measurement. 2nd Edition, Mayfield Publishing Company, Mountain View, USA.

3. de Ayala RJ (2009). The theory and practice of item response theory. Guilford Press, New York, USA. PMid:19590082

4. DeMars C (2010). Item response theory. Oxford University Press, Oxford, UK. https://doi.org/10.1093/acprof:oso/9780195377033.001.00 01

5. Demirtaşl RN, Yalçin S, and Cansu AYAN (2016). The development of IRT based attitude scale towards educational measurement course. Journal of Measurement and Evaluation in Education and Psychology, 7(1): 133-144. https://doi.org/10.21031/epod.43804

6. Ghani AFM, Siraj S, Kassim R, Kenayathulla HB, Marzuki S, and Elham F (2013). Amalan sekolah cemerlang di sekolah berasrama penuh dan sekolah menengah kebangsaan agama: Satu perbandingan. Online Journal of Islamic Education, 1(2): 31-50.

7. Griethuijsen RA, van Eijck MW, Haste H, den Brok PJ, Skinner NC, Mansour N, and BouJaoude S (2015). Global patterns in students’ views of science and interest in science. Research in Science Education, 45(4): 581-603. https://doi.org/10.1007/s11165-014-9438-6

8. Guyer R and Thompson NA (2013). User’s manual for Xcalibre™ item response theory calibration software, version 4.2. Assessment Systems Corporation, Woodbury, Burbank, USA: 44-76.

9. Hair JF, Black WC, Babin BJ, and Anderson RE (2010). Multivariate data analysis. Pearson Education, London, UK.

10. Hambleton RK, Swaminathan H, and Rogers HJ (1991). Fundamentals of item response theory. Sage Publications, Newbury Park, USA.

11. Hishamuddin A and Siti Eshah M (2016). Is 3PL IRT an appropriate model for dichotomous item analysis of AandP final exam?. Malaysian Science and Mathematics Education Journal, 6(1):13-23.

12. KPM (2017a). Bahagian Pengurusan Sekolah Berasrama Penuh dan Sekolah Kecemerlangan. Kementerian Pendidikan Malaysia, Kuala Lumpur, Malaysia. Available online at: https://www.moe.gov.my/

13. KPM (2017b). Perangkaan Pendidikan Malaysia. Bahagian Perancangan dan Penyelidikan Dasar Pendidikan, Putrajaya, Malaysia: 26-56. Available online at: https://www.moe.gov.my/

14. KPM (2018). Portal Rasmi Jabatan Pendidikan Negeri Sembilan. Kementerian Pendidikan Malaysia, Kuala Lumpur, Malaysia. Available online at: https://www.jpnns.moe.gov.my

15. Krejcie RV and Morgan DW (1970). Determining sample size for research activities. Educational and Psychological Measurement, 30(3): 607-610. https://doi.org/10.1177/001316447003000308

16. MOE (2018). Bahagian Perancangan dan Penyelidikan Dasar pendidikan. Ministry of Education Malaysia, Kuala Lumpur, Malaysia. Available online at: https://www.moe.gov.my/

17. Mokshein SE (2018). Aplikasi teori respons item dalam penyelidikan. Universiti Pendidikan Sultan Idris Publisher, Malaysia.

18. Ostini R and Nering ML (2006). Polytomous item response theory models. Sage, Thousand Oaks, USA. https://doi.org/10.4135/9781412985413

19. Prevatt F, Li H, Welles T, Festa-Dreher D, Yelland S, and Lee J (2011). The academic success inventory for college students: Scale development and practical implications for use with students. Journal of College Admission, 211: 26-31.

20. Rivers DC, Meade AW, and Lou Fuller W (2009). Examining question and context effects in organization survey data using item response theory. Organizational Research Methods, 12(3): 529-553. https://doi.org/10.1177/1094428108315864

21. Taber KS (2018). The use of Cronbach’s alpha when developing and reporting research instruments in science education. Research in Science Education, 48(6): 1273-1296. https://doi.org/10.1007/s11165-016-9602-2

22. Thompson NA (2009). Ability estimation with IRT. White Paper, Assessment System Corporation, Minnesota, USA.