Generative AI learning outcomes and skill acquisition technologies in vocational ICT education are explicitly incorporated in the teaching practice and assessment design. This study used a mixed-methods approach, combining questionnaires with in-depth interviews (n = 20), to investigate pedagogical issues in generative AI and its potential learning and evaluation capabilities. 323 ICT undergraduates from a university of vocational technology in Sri Lanka participated in the research. The outcomes of the experiment showed that generative AI is not a leveler but a competence amplifier; thus, very competent students in ICT and those with a high level of awareness of generative AI are much more likely to succeed, whereas undergraduate use of generative AI denies individuals the opportunity to acquire basic skills. The mediator of learning value is student engagement, which implies that students' use of generative AI is more important than the generative AI itself. These findings, when applied to vocational teachers, imply systematic curriculum development: an initial foundation in basic programming, followed by the introduction of generative AI, open pedagogy that builds metacognition, and process-based testing that assesses actual skills rather than generative AI-related tasks. The research provides empirical guidance for educators who need to determine how to structure their instruction, ensure test integrity, and use generative AI in professional training.
Citation: Wijendra Gunathilake, Sulochana Liyanage. Generative AI in vocational ICT education: Implications for teaching, learning, and assessment[J]. STEM Education, 2026, 6(6): 1019-1035. doi: 10.3934/steme.2026041
Generative AI learning outcomes and skill acquisition technologies in vocational ICT education are explicitly incorporated in the teaching practice and assessment design. This study used a mixed-methods approach, combining questionnaires with in-depth interviews (n = 20), to investigate pedagogical issues in generative AI and its potential learning and evaluation capabilities. 323 ICT undergraduates from a university of vocational technology in Sri Lanka participated in the research. The outcomes of the experiment showed that generative AI is not a leveler but a competence amplifier; thus, very competent students in ICT and those with a high level of awareness of generative AI are much more likely to succeed, whereas undergraduate use of generative AI denies individuals the opportunity to acquire basic skills. The mediator of learning value is student engagement, which implies that students' use of generative AI is more important than the generative AI itself. These findings, when applied to vocational teachers, imply systematic curriculum development: an initial foundation in basic programming, followed by the introduction of generative AI, open pedagogy that builds metacognition, and process-based testing that assesses actual skills rather than generative AI-related tasks. The research provides empirical guidance for educators who need to determine how to structure their instruction, ensure test integrity, and use generative AI in professional training.
| [1] |
Crompton, H. and Burke, D., Artificial intelligence in higher education: The state of the field. Int. J. Educ. Technol. High. Educ., 2023, 20: Article 22. https://doi.org/10.1186/s41239-023-00392-8 doi: 10.1186/s41239-023-00392-8
|
| [2] |
Baidoo-Anu, D. and Owusu Ansah, L., Education in the era of generative artificial intelligence (AI): Understanding the potential benefits of ChatGPT in promoting teaching and learning. J. AI, 2023, 7(1): 52–62. https://doi.org/10.61969/jai.1337500 doi: 10.61969/jai.1337500
|
| [3] | Billett, S., Vocational Education and Learning: Purposes, Traditions and Prospects, Springer, 2020. https://doi.org/10.1007/978-3-030-42449-9 |
| [4] |
Wheelahan, L. and Moodie, G., Vocational education qualifications' roles in pathways to work in liberal market economies. J. Vocat. Educ. Train, 2021, 73(1): 10–28. https://doi.org/10.1080/13636820.2020.1861072 doi: 10.1080/13636820.2020.1861072
|
| [5] | Mulder, M., Competence-based vocational and professional education: Bridging the worlds of work and education. In Competence-Based Vocational and Professional Education (ed. M. Mulder), 2017, 1–43. Springer. https://doi.org/10.1007/978-3-319-41713-4_1 |
| [6] | Achiam, J., Adler, S., Agarwal, S., Ahmad, L., Akkaya, I., Aleman, F.L., et al., GPT-4 Technical Report. arXiv preprint arXiv: 2303.08774. https://doi.org/10.48550/arXiv.2303.08774 |
| [7] |
Sullivan, M., Kelly, A. and McLaughlan, P., ChatGPT in higher education: Considerations for academic integrity and student learning. J. Appl. Learn. Teach., 2023, 6(1): 31–40. https://doi.org/10.26522/jalt.v6i1.3238 doi: 10.26522/jalt.v6i1.3238
|
| [8] | Cedefop, Vocational Education and Training in Europe: 1995–2035. Publications Office of the European Union, 2020. |
| [9] |
Chan, C.K.Y. and Hu, W., Students' voices on generative AI: Perceptions, benefits, and challenges in higher education. Int. J. Educ. Technol. High. Educ., 2023, 20: Article 43. https://doi.org/10.1186/s41239-023-00411-8 doi: 10.1186/s41239-023-00411-8
|
| [10] | UNESCO-UNEVOC, Building Resilience: Empowering Youth through Skills Development. UNESCO-UNEVOC International Centre for TVET, 2021. |
| [11] |
Tlili, A., Shehata, B., Adarkwah, M.A., Bozkurt, A., Hickey, D.T., Huang, R., et al., What if the devil is my guardian angel: ChatGPT as a case study of using chatbots in education. Smart Learn. Environ. , 2023, 10: Article 15. https://doi.org/10.1186/s40561-023-00237-4 doi: 10.1186/s40561-023-00237-4
|
| [12] | Sok, S. and Heng, K., ChatGPT for education and research: A review of benefits and risks. Cambodia J. Educ. Res., 2023, 3(1): 110–121. |
| [13] |
Cotton, D.R.E., Cotton, P.A. and Shipway, J.R., Chatting and cheating: Ensuring academic integrity in the era of ChatGPT. Innov. Educ. Teach. Int., 2023, 61(2): 228–239. https://doi.org/10.1080/14703297.2023.2190148 doi: 10.1080/14703297.2023.2190148
|
| [14] |
Lund, B.D. and Wang, T., Chatting about ChatGPT: How may AI and GPT impact academia and libraries? Libr. Hi Tech News, 2023, 40(3): 26–29. https://doi.org/10.1108/LHTN-01-2023-0009 doi: 10.1108/LHTN-01-2023-0009
|
| [15] | OECD, OECD Skills Outlook 2021: Learning for Life. OECD Publishing, 2021. Available from: https://doi.org/10.1787/0ae365b4-en |
| [16] |
Rudolph, J., Tan, S. and Tan, S., ChatGPT: Bullshit spewer or the end of traditional assessments in higher education? J. Appl. Learn. Teach., 2023, 6(1): 342–363. https://doi.org/10.26522/jalt.v6i1.3321 doi: 10.26522/jalt.v6i1.3321
|
| [17] |
Grassini, S., Shaping the future of education: Exploring the potential and consequences of AI and ChatGPT in educational settings. Educ. Sci., 2023, 13(7): Article 692. https://doi.org/10.3390/educsci13070692 doi: 10.3390/educsci13070692
|
| [18] |
Davis, F.D., Perceived usefulness, perceived ease of use, and user acceptance of information technology. MIS Q., 1989, 13(3): 319–340. https://doi.org/10.2307/249008 doi: 10.2307/249008
|
| [19] |
Venkatesh, V. and Davis, F.D., A theoretical extension of the technology acceptance model: Four longitudinal field studies. Manag. Sci., 2000, 46(2): 186–204. https://doi.org/10.1287/mnsc.46.2.186.11926 doi: 10.1287/mnsc.46.2.186.11926
|
| [20] |
Venkatesh, V., Morris, M.G., Davis, G.B. and Davis, F.D., User acceptance of information technology: Toward a unified view. MIS Q., 2003, 27(3): 425–478. https://doi.org/10.2307/30036540 doi: 10.2307/30036540
|
| [21] |
Sweller, J., Cognitive load during problem solving: Effects on learning. Cogn. Sci., 1988, 12(2): 257–285. https://doi.org/10.1207/s15516709cog1202_4 doi: 10.1207/s15516709cog1202_4
|
| [22] |
Sweller, J., van Merriënboer, J.J.G. and Paas, F., Cognitive architecture and instructional design: 20 years later. Educ. Psychol. Rev., 2019, 31: 261–292. https://doi.org/10.1007/s10648-019-09465-5 doi: 10.1007/s10648-019-09465-5
|
| [23] | Bjork, R.A., Memory and metamemory considerations in the training of human beings, in Metacognition: Knowing about Knowing (eds. J. Metcalfe and A.P. Shimamura), 1994,185–205. MIT Press. |
| [24] | Vygotsky, L.S., Mind in Society: The Development of Higher Psychological Processes, Harvard University Press, 1978. |
| [25] | Denny, P., Kumar, V. and Giacaman, N., Conversing with Copilot: Exploring prompt engineering for solving CS1 problems using natural language. In Proc. 54th ACM Tech. Symp. Comput. Sci. Educ., 2023, 1: 1136–1142. https://doi.org/10.1145/3545945.3569823 |
| [26] | Finnie-Ansley, J., Denny, P., Becker, B.A., Luxton-Reilly, A. and Prather, J., The robots are coming: Exploring the implications of OpenAI Codex on introductory programming. In Proc. 25th Australasian Computing Education Conf., 2023, 10–19. https://doi.org/10.1145/3576123.3576134 |
| [27] | Prather, J., Becker, B.A., Craig, M., Denny, P., Loksa, D. and Margulieux, L., The robots are here: Navigating the generative AI revolution in computing education. In Proc. 2023 Working Group Reports on Innovation and Technology in Comput. Sci. Educ., 2023,108–159. https://doi.org/10.1145/3623762.3633499 |
| [28] |
Kasneci, E., Sessler, K., Küchemann, S., Bannert, M., Dementieva, D., Fischer, F., et al., ChatGPT for good? On opportunities and challenges of large language models for education. Learn. Individ. Differ., 2023,103: Article 102274. https://doi.org/10.1016/j.lindif.2023.102274 doi: 10.1016/j.lindif.2023.102274
|
| [29] | Creswell, J.W. and Clark, V.L.P., Designing and Conducting Mixed Methods Research, 3rd ed. SAGE Publications, 2017. |
| [30] | Creswell, J.W. and Creswell, J.D., Research Design: Qualitative, Quantitative, and Mixed Methods Approaches, 5th ed. SAGE Publications, 2018. |
| [31] |
Krejcie, R.V. and Morgan, D.W., Determining sample size for research activities. Educ. Psychol. Meas., 1970, 30(3): 607–610. https://doi.org/10.1177/001316447003000308 doi: 10.1177/001316447003000308
|
| [32] | Patton, M.Q., Qualitative Research and Evaluation Methods, 4th ed. SAGE Publications, 2015. |
| [33] |
Braun, V. and Clarke, V., Using thematic analysis in psychology. Qual. Res. Psychol., 2006, 3(2): 77–101. https://doi.org/10.1191/1478088706qp063oa doi: 10.1191/1478088706qp063oa
|
| [34] | Lincoln, Y.S. and Guba, E.G., Naturalistic Inquiry, SAGE Publications, 1985. |
| [35] |
i, Z., Lee, N., Frieske, R., Yu, T., Su, D., Xu, Y., et al., Survey of hallucination in natural language generation. ACM Comput. Surv., 2023, 55(12): Article 248. https://doi.org/10.1145/3571730 doi: 10.1145/3571730
|
| [36] | Bommasani, R., Hudson, D.A., Adeli, E., Altman, R., Arora, S., von Arx, S., et al., On the opportunities and risks of foundation models. arXiv preprint arXiv: 2108.07258, 2021. https://doi.org/10.48550/arXiv.2108.07258 |
| [37] | Chen, M., Tworek, J., Jun, H., Yuan, Q., Pinto, H.P.D.O., Kaplan, J., et al., Evaluating large language models trained on code. arXiv preprint arXiv: 2107.03374, 2021. https://doi.org/10.48550/arXiv.2107.03374 |
| [38] |
Alasadi, E.A. and Baiz, C.R., Generative AI in education and research: Opportunities, concerns, and solutions. J. Chem. Educ., 2023,100(8): 2965–2971. https://doi.org/10.1021/acs.jchemed.3c00323 doi: 10.1021/acs.jchemed.3c00323
|
| [39] |
Shanto, S.S., Ahmed, Z. and Jony, A.I., PAIGE: A generative AI-based framework for promoting assignment integrity in higher education. STEM Education, 2023, 3(4): 288–305. https://doi.org/10.3934/steme.2023018 doi: 10.3934/steme.2023018
|
| [40] |
Ahmed, Z., Shanto, S.S. and Jony, A.I., Potentiality of generative AI tools in higher education: Evaluating ChatGPT's viability as a teaching assistant for introductory programming courses. STEM Education, 2024, 4(3): 165–182. https://doi.org/10.3934/steme.2024011 doi: 10.3934/steme.2024011
|