Potentiality of generative AI tools in higher education: Evaluating ChatGPT's viability as a teaching assistant for introductory programming courses

1.   Introduction

With recent breakthroughs in artificial intelligence (AI) and natural language processing, a new class of large language models has emerged, exemplified by systems like GPT-3 and ChatGPT¹. Meanwhile, the generative AI tools like GPT-3 and ChatGPT based on GPT-3.5 already demonstrate quality in human-like dialogues and text generation that give hope for further increase in their use for educational purposes. Some propose that conversational agents like ChatGPT could take on teaching assistant (TA) roles to provide supplemental instructional support alongside human teachers ^[1].

¹ ChatGPT is an AI chatbot developed by OpenAI, based on the GPT-3.5 language model. It is important to note that GPT-3 and ChatGPT are not competing systems, but rather ChatGPT is an application built on top of the GPT-3.5 architecture.

However, the questions pertaining to the appropriateness, effectiveness, and limitations of deploying AI systems in such educational settings give rise to quite crucial ones that would need answering. More so, questions pertinent to such deployment are in technical subjects like computer programming, which demands from learners detailed, problem-oriented, logical thinking, and actual coding skills. The very nature of programming education—requiring the student to develop computational thinking, understand abstract ideas, and apply them in problem-solving—puts this in a unique place within the challenges presented to an AI-based teaching assistant. Hence, the effectivity with which AI systems provide targeted feedback, adjust with individual learning requirements, and instill practical coding skills is relevant to the success of its integration. Thus, a critical review of the strengths and weaknesses of using AI teaching assistants in the context of computer science education becomes central to informing the appropriate deployment and effective support for student learning.

The "appropriateness" of an AI system refers to its suitability for use in a specific educational setting, considering factors such as its ability to effectively support learning outcomes, align with pedagogical goals, and adhere to ethical and societal norms. This involves appropriateness of the impacts that might be caused by the AI system on the learning experiences of students, impacts on teachers, and also the educational environment as a whole.

The advent of large language models like ChatGPT has prompted extensive public debate on the potential societal implications of increasingly capable AI systems ^[2]. While acknowledging valid concerns around issues like misinformation, bias, and accountability, it is evident that AI also presents opportunities for human-centric symbiosis and collaborative intelligence if responsibly implemented ^[3]. Education is one domain primed for beneficial partnerships between human teachers and AI assistants that enhance rather than replace human roles ^[4]. There has been so much hype around this issue, often to the point of assuming that AI will replace teachers, but some researchers now suggest that real progress in AI-assisted education may be developing AI as a supplementary aid that complements human strengths and compensates for human cognitive limits ^[5]. However, more empirical evidence is needed to support this claim and to guide the effective integration of AI in educational contexts.

Programming pedagogy could stand to gain from AI systems that provide scalable, personalized support as students build computational thinking abilities. However, successfully integrating AI in education requires evidence-based insights on the appropriate scope and contributions of AI tools relative to human instruction ^[6]. Before presuming ChatGPT can serve as a virtual teaching assistant, rigorous investigation is needed to map its capabilities and limitations for adaptive programming instruction compared to human tutors. While large language models can democratize access to knowledge, upholding standards of learning quality requires understanding the unique value of human teachers in mentoring computational competencies ^[7].

This work aims to inform this understanding for responsible AI adoption in computer science education and to rigorously investigate the viability of ChatGPT as a virtual TA for introductory programming courses in undergraduate computer science curricula. Programming pedagogy relies heavily on constructionist learning, active problem-solving, and iterative coding—areas where current AI systems may lack human teachers' abilities to scaffold understanding through interactive dialogue, provide adaptive guidance, and properly analyze student code ^[8]. Hence, while ChatGPT offers some advantages like 24/7 availability and scalability, it remains unclear if the tool can adequately explain complex programming concepts, resolve student misconceptions, and encourage systematic coding skills².

² In this context, "systematic coding skills" refer to the ability to approach programming problems in a structured and logical manner, breaking them down into smaller sub-problems, designing algorithms, and writing clean, efficient, and well-documented code. These skills also encompass the ability to test, debug, and maintain code effectively.

In this study, an experimental study eliciting ChatGPT's responses to common introductory programming questions and misconceptions was conducted. The AI's explanations were evaluated by programming faculty along dimensions of accuracy, completeness, pedagogical soundness, and resolving confusion. Additionally, student perceptions of ChatGPT's utility are surveyed after interacting with the tool. The contributions of this study are multifold. First, novel insights were provided on the effectiveness of ChatGPT specifically for TA roles in programming pedagogy based on faculty and student assessments. Second, gaps were identified between human and AI capabilities for adaptive programming instruction key to the guidelines for appropriate integration.

Overall, this work offers data-driven perspectives on both the promises and limitations of large language models for enhancing computer science education. While they may not fully replicate human teaching, with prudent design AI could still augment programming instruction and student learning. The findings will inform the best practices for incorporating generative AI in higher education in an ethical, equitable manner.

The paper is structured as follows: the literature review in section 2 discusses the growing interest in AI tools in education and highlights the need for assessing their pedagogical effectiveness. The methodology in section 3 details the participant recruitment, the questions posed to ChatGPT, and the survey questionnaires administered to both faculty and students. The results in section 4 provide the means from the faculty and student surveys, presenting where ChatGPT, as an AI TA, stands as a strength or weakness. This section looks into the analysis of the survey results on areas where ChatGPT did well and areas it failed, along with the results from the open-ended question. In section 5, the implications of using generative AI in education are inferred, and recommendations for further research based on the results obtained are drawn. Its structure is systematic and exhaustive, evaluating the potential that ChatGPT holds as a digital teaching assistant and, therefore, contains valuable insights in the integration of such AI tools at universities and colleges. Finally, the study is concluded in section 6.

2.   Literature review

Generative artificial intelligence (AI) tools like large language models have sparked growing interest in their potential applications in education ^[9]. Recent advances in natural language processing have enabled systems like ChatGPT to demonstrate human-like conversational abilities and language generation capabilities ^[10]. When AI technologies are created, sold, and used, it is important to weigh the advantages and disadvantages ^[11]. This has prompted questions on whether AI agents could take on teaching and tutoring roles to support or augment human instructors.

Several studies have explored the use of conversational agents and dialogue systems in education ^[12]. For instance, an AI teaching assistant focused on primary education was proposed that could answer student's questions, generate problems, and work examples, with an adaptive algorithm ^[13]. Their evaluations found the algorithm's performance proved to be positive. The ways to implement voice driven AI into classrooms were also investigated ^[14].

While promising, researchers have also highlighted the risks and limitations of relying on AI agents for teaching. For example, conversational agents tend to lack the empathy, social awareness, and theory of mind exhibited by human teachers and hence may struggle to establish rapport or effectively motivate student ^[15]. There are evident challenges regarding the use of AI in educational settings, such as the lack of adaptability, limited ability to provide personalized feedback, and potential biases in AI systems ^[16]. These challenges are particularly relevant for conversational agents and AI tutors developed before the advent of large language models, which may have more limited natural language processing capabilities compared to current models like ChatGPT. The extent of AI constraints in education depends on the form of AI framework and the technical basis. For instance, for small to medium conversational agents based on pre-large language models, it might be hard for them to understand the context, produce logical answers, or give feedback that is adaptive and has sense ^[17]. The current large language models like GPT-3.5 and ChatGPT have demonstrated improved performance in these areas ^[18]. However, even these advanced models may have limitations in terms of reasoning, common sense understanding, and alignment with educational goals, which require further investigation ^[19]. Moreover, AI systems trained on limited data can propagate biases and misinformation when used for instruction ^[20].

Recent works have proposed frameworks and guidelines for incorporating generative AI models into educational settings while addressing concerns around academic integrity. The PAIGE framework ^[21] outlines an AI system to detect potential cheating on assignments and promote academic honesty by distinguishing human versus AI-generated text. The authors suggest pairing generative models like ChatGPT with plagiarism detectors to uphold integrity standards. Similarly, Enriching the Learning Process with Generative AI ^[22] puts forth strategies to evaluate ChatGPT's ability to stimulate critical thinking through reflective writing exercises, while having instructors verify originality.

There have been hardly any studies that discuss the role of a ChatGPT-like model in assisting teachers. Therefore, their effectiveness as teaching assistants should be tested further, specially now when they are still new. At first glance, it seems that ChatGPT cannot handle intricate coding exercises involving reasoning; yet it can solve simpler ones related to programming languages ^[23]. This raises uncertainties about its suitability for assisting in technical courses like programming.

As a whole, current literature suggests that it is important to conduct thorough investigations that evaluate how good or bad academic-teaching tools (for various subjects) are in today's age; such work should also assess their efficiency. In this study, authentic data is provided about whether ChatGPT can be used at colleges/universities for teaching these subjects using information provided by teachers and students. The findings are expected to inform guidelines on appropriate integration of generative AI in education.

3.   Methodology

The participants were 20 undergraduate students from the department of computer science (ages 18–22 years old) enrolled in an introductory Python programming course. All participants were recruited through announcements made by the course instructor during class. Interested students were informed that the study involved evaluating an AI teaching assistant for the programming course.

The student participants were given access to ChatGPT for a period of 2 weeks during their programming course. They were instructed to ask at least 15 questions from the pre-compiled set of questions as they arose naturally during coursework. ChatGPT's responses were collected. Five faculty members who commonly teach introductory programming then evaluated the ChatGPT responses.

The five faculty members who evaluated ChatGPT's responses were chosen based on their experience in teaching introductory programming courses. All faculty members had a minimum of three years of teaching experience in computer science, with an average of 5 years. Three of the faculty members held doctoral degrees in computer science or a related field, while the other two held master's degrees. Additionally, all faculty members had completed pedagogical training or workshops focused on effective teaching strategies in computer science education.

Using a 5-point Likert scale, the faculty members assessed accuracy, completeness, pedagogical soundness, and the ability to resolve confusion. After the 2-week period, the students completed the perception survey regarding ChatGPT's responses. They rated dimensions like understandability, accuracy, completeness, and resolving confusion on a 5-point scale. Finally, all of the results were analyzed to determine ChatGPT's ability as a teaching assistant.

Figure 1 shows the overall methodology design of the study conducted. The diagram that follows outlines the method employed in the course of the study.

Figure 1.  Methodology diagram.

DownLoad: Full-Size Img PowerPoint

3.1.   Introductory programming questions

The questions cover a broad range of fundamental Python programming concepts and skills that would typically be encountered in an introductory coding course or tutorial. Topics include Python data types like strings, lists, and dictionaries as well as foundational programming techniques like variables, functions, conditional logic, loops, and arrays. The questions prompt students to demonstrate Python syntax, write basic code snippets that illustrate language features, use built-in Python functions and methods, work with collections like lists and dictionaries, handle strings and arrays, read/write files, import modules, install packages, understand scope rules, and utilize programming best practices like comments and docstrings. The set of questions aims to assess the comprehension of Python from multiple angles through short-answer, fill-in-the-blank, and coding exercises. There is a balance of theoretical conceptual questions and practical programming questions that require writing code. Overall, the set of 30 questions covers a diverse collection of Python topics and skills that provide insight into a student's mastery of introductory level Python programming. The full question set is provided in Figure 2.

Figure 2.  Introductory Python programming question set.

DownLoad: Full-Size Img PowerPoint

3.2.   Survey questionnaire

A 10-item Likert scale questionnaire was developed to assess faculty perceptions of ChatGPT's performance as a teaching assistant for introductory programming concepts. The faculty survey included statements evaluating various dimensions such as accuracy, completeness, pedagogical strategies, language, resolving confusion, providing examples, handling edge cases, encouraging problem solving, appropriate tone, and overall helpfulness. Additionally, an open-ended question allowed faculty to provide any other relevant feedback on ChatGPT's responses. This questionnaire enabled comprehensive quantitative and qualitative data to be collected from faculty on how ChatGPT's teaching assistant abilities compared to human teaching assistants for introductory programming content. Figure 3 shows the faculty survey questionnaire.

Figure 3.  Survey questionnaire on faculty perceptions of ChatGPT's performance as a teaching assistant.

DownLoad: Full-Size Img PowerPoint

A parallel 10-item Likert scale questionnaire was administered to the student participants to gauge their perceptions of ChatGPT's usefulness as a programming teaching assistant. The student survey prompted participants to rate their level of agreement with statements regarding understandability, clarity, helpfulness of examples, providing code snippets, identifying knowledge gaps, assisting with debugging, encouraging systematic thinking, individualized responses, overall utility, and recommendation to peers. An open-ended question also allowed students to elaborate on their experience interacting with ChatGPT. This questionnaire provided crucial insight into student perspectives on the value of ChatGPT as a supplemental teaching aid for introductory programming pedagogy targeted to their comprehension levels and needs. Figure 4 displays the questionnaire used for surveying the students.

Figure 4.  Survey questionnaire on student perceptions of ChatGPT's performance as a teaching assistant.

DownLoad: Full-Size Img PowerPoint

3.3.   Survey measurement scale

A Likert scale is a psychometric measurement technique commonly used in questionnaires and surveys to gauge attitudes, perceptions, and opinions ^[24]. In a Likert scale, respondents are presented with a statement and asked to indicate their level of agreement or disagreement on a symmetric agree-disagree scale. A conventional 5-point or 7-point symmetric scale is used, ranging from "strongly disagree" on one end to "strongly agree" at the other end, with a neutral midpoint. Respondents select the point on the scale that aligns best with their view. Each response is associated with a numerical score, allowing quantitative statistical analysis. Likert scales can efficiently capture gradations of opinion, perspective, and subjectivity from participants in a standardized way ^[25]. They produce interval data that provide insights into the respondent's feelings toward the target item or experience being evaluated. Likert scale data assumes equidistance between scale points, allowing means and summations to be meaningfully computed. The Likert scale questions used in the faculty and student questionnaires for this study reflect a standard use of this measurement technique. The Likert scale used in this study is provided in Table 1.

Table 1.  Likert scale.

Strongly Disagree	Disagree	Neutral	Agree	Strongly Agree
1	2	3	4	5

---

 | Show Table

DownLoad: CSV

4.   Results

4.1.   Faculty survey results

Figure 5.  Average ratings of the survey from the faculty members.

DownLoad: Full-Size Img PowerPoint

The 10-item faculty questionnaire generated the following average ratings of ChatGPT's responses across key dimensions of teaching assistant performance. For accuracy of programming concepts (Q1), completeness of explanations (Q2), use of effective teaching strategies (Q3), and encouraging problem-solving (Q8), ChatGPT received mean ratings between 4–4.6 out of 5, indicating general faculty agreement that the AI assistant demonstrated strengths in these areas. However, lower average scores were obtained for resolving student confusion (Q5, mean 3.2), providing useful examples and code (Q6, mean 3), and handling edge cases (Q7, mean 3.6). The lowest ratings came on the aspects of working through complex problems and limitations. The faculty gave an average score of 4.2 for ChatGPT's overall helpfulness as an introductory programming teaching assistant (Q10).

4.2.   Student survey results

Figure 6.  Average ratings of the survey from the students.

DownLoad: Full-Size Img PowerPoint

The 10-item student questionnaire produced the following average ratings regarding ChatGPT's performance across key dimensions of teaching assistant support. Students gave high mean scores for understandability (Q1, 4.4), providing clear answers (Q2, 4.1), usefulness of examples (Q3, 4.35), debugging assistance (Q6, 4.3), encouraging systematic thinking (Q7, 4.15), and overall usefulness (Q9, 4.5). Slightly lower but still agreeable ratings were given for ChatGPT's ability to identify knowledge gaps (Q5, 4.1), individualize responses (Q8, 4), and provide code snippets on request (Q4, 3.55). The highest average rating came on the willingness to recommend ChatGPT as a programming teaching assistant (Q10, 4.55). Generally, students perceived ChatGPT to be an effective teaching aid, with strengths in understandability, usefulness of examples, debugging support, and overall utility.

4.3.   Open-ended question responses

In addition to the Likert scale questions, both faculty and student surveys included an open-ended question allowing participants to provide further feedback on ChatGPT's performance as a teaching assistant. The responses to these open-ended questions were analyzed using thematic analysis ^[26] to identify common themes and insights.

Faculty responses to the open-ended question highlighted ChatGPT's strengths in providing clear explanations for basic concepts and its potential to support student learning. However, they also emphasized the need for human instruction to address complex problems and provide individualized support. Some faculty members expressed concerns about the limitations of ChatGPT in handling edge cases and adapting to student needs.

Student responses to the open-ended question generally praised ChatGPT's ability to provide helpful explanations, examples, and debugging assistance. Many students appreciated the accessibility and convenience of having an AI-based teaching assistant available 24/7. However, some students also noted that ChatGPT's responses could be inconsistent or lack the depth of human instructors in certain situations.

5.   Discussion

The faculty and student surveys provide valuable insights into the potential strengths and weaknesses of ChatGPT as an AI teaching assistant for introductory programming concepts. Overall, both faculty and students gave reasonably high ratings for ChatGPT's performance across most dimensions evaluated in the questionnaires. The open-ended-question responses from faculty and students further support these findings, providing qualitative evidence for ChatGPT's strengths and limitations as a teaching assistant.

It is important to note that this is an exploratory study with a limited sample size of 20 students and 5 faculty members. While this sample size is appropriate for an initial investigation, the findings should be interpreted with caution, and further research with larger sample sizes is needed to establish the generalizability of the results. The conclusions drawn from this study are preliminary and serve as a foundation for future research with more robust sample sizes and study designs. The exploratory nature of this study should be considered when evaluating the potential of ChatGPT as a teaching assistant in introductory programming courses.

The study did not involve a direct comparison between human teaching assistants and ChatGPT. The emphasis that human instruction still plays an important role is seen from identified weaknesses in ChatGPT, and it does not put it head-to-head with human performance. Future work should attempt to identify the respective strengths and weaknesses of human instructors and AI-based systems within the context of introductory programming courses. Furthermore, the identified strengths and weaknesses of ChatGPT are mapped against specific survey questions, and hence they are not clear in this study. Subsequent research should be clear in establishing more evident relationships between the dimensions of interest and the relevant items on the survey to draw more firm conclusions.

The faculty responses suggested that ChatGPT evidenced good accuracy in the explanations of programming concepts, completeness in the breadth and depth of explanations, and an encouraging problem-solving approach. Ratings by faculty for these dimensions with respect to ChatGPT fell between 4.0 and 4.6 points on a 5-point scale, implying general agreement that ChatGPT showed human-like abilities in those aspects of the introductory programming pedagogy. The faculty was also impressed with the style of ChatGPT in the use of proper jargons and tone in the explanations given to the new students regarding the programming concepts.

Faculty, however, identified clear weaknesses in some of ChatGPT's responses. The lowest average faculty ratings received by ChatGPT included its ability to resolve student confusion, provide useful code snippets and examples, handle edge cases, and exceptions. The mean scores on these dimensions fell between 3.0 and 3.6, which does speak to some issues with the ability of ChatGPT to personalize explanations toward specific student difficulties and limitations. This is in line with the fact that large language models like ChatGPT have known weaknesses in properly responding to novel situations and questions too far from the purview of its training data. The importance of human instruction in programming education extends beyond the current limitations of AI systems. Human instructors possess unique qualities such as adaptability, empathy, and the ability to provide individualized support ^[27]. These qualities are crucial for effectively guiding students through complex problem-solving processes and addressing their unique learning needs ^[28].

The student perceptions provide a complementary perspective. The students found ChatGPT highly understandable, with clear and complete explanations. They gave high marks for ChatGPT's usefulness of examples, debugging assistance, ability to encourage systematic thinking, and overall utility as a programming teaching aid. However, like the faculty, students identified providing individualized responses and useful code snippets on request as relative weaknesses, though still moderately positive in absolute terms.

It is important to note that while students rated ChatGPT's debugging assistance (Q6) relatively high (mean 4.3), this does not necessarily validate ChatGPT's ability to effectively debug complex code. The survey question assesses students' perceptions of ChatGPT's helpfulness in debugging, but does not directly measure its actual performance in resolving complex coding issues. Further research is needed to objectively evaluate ChatGPT's debugging capabilities compared to human teaching assistants.

The high ratings students gave in sections such as understandability, clarity, and overall usefulness point out that ChatGPT's strength is in making the presented programming topics coherent and clear for them. The students were also very receptive to examples and analogies ChatGPT used in explaining coding ideas, resonating with the tool's ability to tap into knowledge synthesized from huge training datasets. Student responses were also positive with respect to the way it helped them to learn systematic problem-solving in programming and, therefore, provided valuable assistance in debugging. This further emphasizes how useful ChatGPT is as a patient, tireless teaching assistant, demystifying complex concepts and giving step-by-step coaching in problem-solving. The students, however, were able to point out a relative weakness: the bot's inability to provide individualized responses and code snippets on request, thereby reflecting an inherent limitation of current AI systems. On the whole, the student perspective at least validates the strengths of ChatGPT in explaining intro programming basics comprehensively while pointing out that there will still be a need for human Tas to fix things, debug code, and provide tailored instruction. According to a student perspective, the paper reflects on promises and pitfalls of AI in education, emphasizing the guideline requirement concerning intelligent integration and human-AI collaboration.

The study revealed that while faculty members rated ChatGPT lower, students gave it higher marks, for example, in its usefulness. The students may view ChatGPT as an additional teaching tool, valuing its examples and ability to improve their learning process, which could account for this discrepancy in perception. Faculty, on the other hand, may have higher standards for the caliber and educational usefulness of ChatGPT's examples because they are seasoned educators.

Taken together, the faculty and student perceptions suggest that ChatGPT exhibits strengths in explaining introductory programming concepts, such as accuracy, completeness, and comprehension. However, further research is needed to validate these findings and to directly compare ChatGPT's performance to that of human teaching assistants. However, both groups surfaced limitations in ChatGPT's capacity to resolve complex confusion, provide examples and code, handle novel cases, and individualize responses. This aligns with the inherent strengths and weaknesses of large language models—a broad competency within the training distribution but a lack of true adaptability and reasoning.

The interpretation of the average rating for the performance of ChatGPT is relatively cautious. Such an average rating, ranging from 3.0 to 3.6, can be regarded as relatively low against other dimensions in this study. Without comparative data for humans performing similar tasks, human TAs or lecturers, a definite conclusion about the significance of these ratings cannot be made. Besides these factors, there are other instructor-related human factors, most likely time pressure and task complexity, that will influence their performance. Future research must aim to conduct comparative AI teaching assistant and human instructor studies which will provide a more holistic view over the relative advantages and disadvantages of these two.

The appropriateness of ChatGPT as an AI teaching assistant in introductory programming courses depends on various factors. While this study highlights ChatGPT's strengths in explaining core concepts and providing general assistance, its limitations in handling complex problems and providing individualized support raise concerns about its overall suitability. The effectiveness of ChatGPT in supporting learning outcomes and aligning with pedagogical goals requires further investigation. Moreover, the ethical implications of using AI systems in educational settings, such as potential biases and the impact on teacher-student interactions, must be carefully considered. Ensuring the appropriate use of ChatGPT and similar AI tools in programming education will require ongoing research, the development of best practices, and collaboration between educators, researchers, and AI developers.

Overall, ChatGPT seems to be of reasonable help in tackling core programming concepts but might fall short in the replacement of human teaching assistants for debugging complex coding challenges and resolving individual confusion by providing individualized coding examples and solutions. Using ChatGPT as an accompaniment to the TA would help to explain programming constructs and how they are put together, with qualified human tutors providing advanced debugging and project assistance. Indeed, faculty and students alike noted the promise of ChatGPT as a new AI teaching tool while surfacing places where human instruction seemed especially valuable, such as when walking the student through complex and open-ended learning tasks.

6.   Conclusions

This study delves into what appears to be one of the most beneficial uses of ChatGPT, the large language model, in an artificial intelligence teaching assistant role in a first-semester course for programming students. These relevant findings were obtained from prompts and surveys through an experimental study, showing the strengths and weaknesses of ChatGPT in this educational context. Furthermore, the appropriateness of ChatGPT and similar AI tools in programming education should be carefully evaluated, considering factors such as their alignment with learning outcomes, pedagogical goals, and ethical considerations.

The results of the present study suggest that ChatGPT has promising potential for the coherent explanation of basic programming concepts. Most faculty and students rated the chatbots high in terms of accuracy, completeness, pedagogical strategies employed, and general help in learning the information. However, the deficiencies would be apparent in the case of the resolution of complex confusion and adapting to individual requirements. Deficiencies include offering help with debugging and tackling new cases.

Therefore, the findings suggest that ChatGPT has merit as a supplemental teaching aid, but also highlight the need for human instruction to address advanced concepts and provide individualized support. The results align with the broad competency but lack of adaptability inherent in large pretrained language models like ChatGPT.

While promising, ChatGPT and similar technologies may be most effective when used as complementary tools focused on core competencies like explaining concepts, rather than as direct replacements for human teaching assistants. With carefully planned integration and a clear understanding of its strengths and limitations, AI has the potential to enhance programming education when used in conjunction with human instruction.

Future research should directly compare the performance and perception of human teaching assistants and AI-based systems like ChatGPT in introductory programming courses. Such studies will provide more concrete evidence regarding the relative strengths and weaknesses of each approach and inform the development of effective strategies for integrating AI tools into programming education.

Several promising ways of working in this area can be used as a starting point for research. Initially, it would be important to increase the number of participants in the experiment on introductory programming which would involve students from different programming courses. Moreover, it is necessary to test different models beyond ChatGPT to establish AI teaching assistants' effectiveness. Another area that requires further exploration is focusing on the development of instructions for integrating ChatGPT efficiently into the study process.

Author contributions

Zishan Ahmed: Conceptualization, Methodology creation, Investigation, Resources, Data Curation, Survey questionnaire preparation, Writing - Original Draft, Writing –Abstract, Writing – Methodology, Writing – Discussion, Writing- Conclusion; Shakib Sadat Shanto: Result Analysis, Data Curation, Survey conduction, Writing – Introduction, Writing – Literature review, Writing - Results; Akinul Islam Jony: Conceptualization, Resources, Survey conduction, Writing - Review & Editing, Supervision. All authors have read and approved the final version of the manuscript for publication.

Use of AI tools declaration

The authors declare that they have not used artificial intelligence (AI) tools in the writing of this article. However, the artificial intelligence (AI) tool, ChatGPT, was used to generate responses for the purpose of the study.

Acknowledgments

The authors would like to thank the reviewers for their constructive feedback.

Conflict of interest

The authors declare that they have no conflicts of interest.

Ethics declaration

The authors declare that the ethics committee approval was waived for the study.