Are they literate on ChatGPT? university language students' perceptions, benefits and challenges in higher education learning

Md Monjurul Islam

Abstract

Despite the promising potential of artificial intelligence (AI) tools like ChatGPT for enhancing the learning experience, its integration as an educational tool in higher education poses disruptions that could affect students. This study explored university language students’ usage, awareness, readiness, concerns, and perceived benefits and challenges of using ChatGPT in higher education learning. The study used an explanatory sequential mixedmethod design, combining online surveys and semi-structured interviews. A total of 355 language students from five private higher education institutions in Indonesia were recruited using purposive sampling. These participants were asked to complete the closed-ended questionnaire, and nine participants were selected for interview sessions. The data was analyzed using descriptive analysis and thematic analysis. The findings reveal that a majority of students had been using ChatGPT for their learning purposes, especially for writing assignments. The findings also highlight that university language students had a high awareness, readiness, and concerns about using ChatGPT for their academic activities. Additionally, university language students acknowledged the potential benefits of ChatGPT in terms of writing support, personalized learning, increased productivity, brainstorming for generating ideas, and additional resources. However, they also identified the challenges of using ChatGPT, including inaccuracy, lack of critical thinking, plagiarism and AI detection, and technical problems. These findings provide a nuanced understanding of how university language students use and perceive ChatGPT for learning purposes, highlighting the need for support and guidance from higher education providers in the responsible use of AI in academic settings.

References

Acosta-Enriquez, B. G., Arbulú Ballesteros, M. A., Huamaní Jordan, O., López Roca, C., & Saavedra Tirado, K. (2024). Analysis of college students' attitudes toward the use of ChatGPT in their academic activities: effect of intent to use, verification of information and responsible use. BMC psychology, 12(1), 255–273. https://doi.org/10.1186/s40359-024-01764-z.

Anderson, N., Belavy, D. L., Perle, S. M., Hendricks, S., Hespanhol, L., Verhagen, E., & Memon, A. R. (2023). AI did not write this manuscript, or did it? Can we trick the AI text detector into generated texts? The potential future of ChatGPT and AI in sports & exercise medicine manuscript generation. BMJ Open Sport & Exercise Medicine, 9(1), e001568. https://doi.org/10.1136/bmjsem-2023-001568.

Aruleba, K., Sanusi, I. T., Obaido, G., & Ogbuokiri, B. (2023). Integrating ChatGPT in a computer science course: Students perceptions and suggestions. arXiv preprint arXiv:2402.01640, 1–13. https://doi.org/10.48550/arXiv.2402.01640.

Assad, A. (2024). Exploring the use of ChatGPT among media students in Egypt: Opportunities and challenges. Online Journal of Communication and Media Technologies, 14(2), e202424. https://doi.org/10.30935/ojcmt/14416.

Braun, V., & Clarke, V. (2006). Using thematic analysis in psychology. Qualitative Research in Psychology, 3(2), 77–101. https://doi.org/10.1191/1478088706qp063oa.

Chan, C. K. Y. (2023). A comprehensive AI policy education framework for university teaching and learning. International Journal of Educational Technology in Higher Education, 20(1), 38–62. https://doi.org/10.1186/s41239-023-00408-3.

Chan, C. K. Y., & Hu, W. (2023). Students’ voices on generative AI: Perceptions, benefits, and challenges in higher education. International Journal of Educational Technology in Higher Education, 20(1), 43–60. https://doi.org/10.1186/s41239-023-00411-8.

Chan, C. K. Y., & Lee, K. K. (2023). The AI generation gap: Are Gen Z students more interested in adopting generative AI such as ChatGPT in teaching and learning than their Gen X and millennial generation teachers? Smart Learning Environments, 10(1), 60–83. https://doi.org/10.1186/s40561-023-00269-3.

Creswell, J. W. (2014). Research design: Qualitative, quantitative and mixed method approaches (4th ed.). Sage Publications.

Creswell, J. W., & Clark, V. L. P. (2017). Designing and conducting mixed methods research (3rd ed.). Sage Publications.

Das, S. R., & Madhusudan, J. V. (2024). Perceptions of higher education students towards ChatGPT usage. International Journal of Technology in Education, 7(1), 86–106.

Elbanna, S., & Armstrong, L. (2024). Exploring the integration of ChatGPT in education: Adapting for the future. Management & Sustainability: An Arab Review, 3(1), 16–29. https://doi.org/10.1108/MSAR-03-2023-0016.

Fuchs, K. (2023). Exploring the opportunities and challenges of NLP models in higher education: Is ChatGPT a blessing or a curse? Frontiers in Education, 8, 1166682. https://doi.org/10.3389/feduc.2023.1166682.

Fuchs, K., & Aguilos, V. (2023). Integrating artificial intelligence in higher education: Empirical insights from students about using ChatGPT. International Journal of Information and Education Technology, 13(9), 1365–1371. http://dx.doi.org/10.18178/ijiet.

Graf, A., & Bernardi, R. E. (2023). ChatGPT in research: balancing ethics, transparency and advancement. Neuroscience, 515, 71–73. https://doi.org/10.1016/j.neuroscience.2023.02.008.

Güven, G. Ö., Yilmaz, Ş., & Inceoğlu, F. (2024). Determining medical students' anxiety and readiness levels about artificial intelligence. Heliyon, 10(4), e25894 https://doi.org/10.1016/j.heliyon.2024.e25894.

Hartanto, A. Y., & Rohmah, F. N. (2024). Makin marak siswa pakai AI untuk mengerjakan tugas [The rise of students using AI to do assignments]. tirto.id. https://tirto.id/penggunaan-ai-di-dunia-pendidikan-makin-marak-dan-merata-gZax.

Huallpa, J. J. (2023). Exploring the ethical considerations of using Chat GPT in university education. Periodicals of Engineering and Natural Sciences, 11(4), 105–115. http://dx.doi.org/10.21533/pen.v11i4.3770.

Iku-Silan, A., Hwang, G. J., & Chen, C. H. (2023). Decision-guided chatbots and cognitive styles in interdisciplinary learning. Computers & Education, 201, 104812. https://doi.org/10.1016/j.compedu.2023.104812.