Impact of participation in the World Robot Olympiad on K-12 robotics education from the coach's perspective

1.   Introduction

Science, technology, engineering, and mathematics (STEM) education is receiving increasing attention, and robotics education is becoming popular worldwide. STEM education can inspire children's natural enthusiasm for exploring the unknown and give them opportunities to put their knowledge into practice [1]. Robots are considered an essential element of STEM education because they can convey complex mathematical and scientific thinking [2]. The integration of robotics and STEM education and continual improvement of robotics education become increasingly a focus of research and are conducive to cultivating students' ability to innovate and their scientific literacy. Barnes and other scholars created an extracurricular activity program, the Children's Robot Theater, for rural elementary school children and conducted two iterations of the program within two years [3]. They reported that the integration of robotics with STEM education had great potential and could promote STEM education in an integrated manner. In addition, the combination of STEM education and educational robotics technology provides opportunities for the development of skills and capabilities required in the workplace [4].

In recent years, many countries have held robotic competitions, such as BotBall (http://www.botball.org/), the robotic competition organized by the FIRST organization (http://www.usfirst.org/), RoboCup Junior (http://www.robocupjunior.org), and the World Robot Olympiad (WRO; http://www.wroboto.org). As a mainstream robotic competition, the WRO has been successfully held 16 sessions, attracted students, parents, and coaches from more than 85 countries and regions. Robotic competitions are undoubtedly an effective venue for the application of robotics in STEM education and have helped cultivate students' innovative spirit and practical ability [5]. One study reported that such competitions provided school-age children with unique and essential learning opportunities [6]. The project-based and goal-oriented hands-on experiences provided by robotic competitions can have a long-term impact on students' learning motivation and skill development. In the process of building robots, children are also developing engineering and teamwork skills. The tutoring activities at robotic competitions not only focus on the robotic competition itself but also promote STEM education. This is because students have to integrate tasks in robot design, assembly, coding, operations, and modifications together to achieve the desired goals. Such process not only helps them foster effective collaboration as a team [7], but also improves their motivation and critical thinking [8,9]. Çetin and others collated 23 relevant studies and concluded that educational robots enable students to apply computer science and computational thinking to problem-solving [10].

The previous studies on robotic competitions mainly focused on students [11,12] and parents [13]. As indirect participants in robotic competitions, coaches witness the growth of students during a competition. We believe that coaches' views regarding the impact of robotic competitions on participants are of great research value. This study takes WRO as an example to explore the impact of robotic competitions from the perspective of coaches. We explore how robotic competitions can be used to enhance robotics and STEM education. This study poses the following questions:

Q1. For coaches from different professional sectors and for competition categories, what is the impact of the WRO on students?

Q2. From the coach's perspective, what is the largest gain made by students through participating in the WRO?

Q3. From the coach's perspective, what impact does the WRO have on students' learning and STEM literacy in schools?

2.   Research design

2.1.   Research participants

The research participants were coaches who attended the WRO 2019 finals in Hungary as volunteers. A total of 245 coaches from 41 countries completed the questionnaire, in which 195 of the responses were valid, representing 79.6% of the total. In addition, we interviewed 14 coaches in person.

2.2.   Research methods

This study uses a mixed research method to evaluate and analyze the coaches' views on student developments after the students participated in the WRO through two rounds of questionnaire surveys (each by online and offline) and selected interviews. Questionnaires were distributed online between 23 January and 28 December 2019 whereas paper questionnaires were distributed at the WRO 2019 finals. Through the questionnaire surveys and interviews with selected coaches, we investigated the impact of robotic competitions on students who participated in the competition.

2.3.   Questionnaire and interview

Our team developed a World Robot Impact Scale Questionnaire consisting of two parts, in reference to the previous studies [5,14]. The first part was related to demographics, including gender, professional sector, and student leadership. The second part focused on the coach's views on the students who participated in the WRO, and included 26 multiple-choice questions. These questions were orientated to the following seven dimensions: Learning Skills, Engineering Thinking, Emotional Engagement, Career Choice, Problem Solving, Collaboration Quality, and Global Consciousness. The reliability analysis of the questionnaire is presented in Table 1. The reliability values of all dimensions are higher than 0.7, indicating that the reliability of the scale is acceptable.

Table 1. 

Overall sample reliability analysis

.

Dimension	Number of items	Number of questions	Cronbach's alpha
Learning Skills	195	5	0.828
Engineering Thinking	195	5	0.823
Emotional Engagement	195	2	0.804
Career Choice	195	2	0.712
Problem Solving	195	3	0.820
Collaboration Quality	195	4	0.839
Global Consciousness	195	5	0.798

---

 | Show Table

DownLoad: CSV

The interview adopted a semi-structured approach in accordance with the research purpose. The specific questions asked are as follows:

o What are your student's favorite WRO activities, for example, robot building, programming, or reporting?

o How has the WRO affected your students?

o What is the largest gain that your students have had from the WRO?

o Does the WRO promote your students' school learning? If yes, which disciplines are promoted?

o Will the WRO affect your student's future career and life choices?

2.4.   Data analysis tools

For the chosen reliable datasets, SPSS-20.0 was used to obtain the descriptive statistics and analyze the significance of the questionnaire data. For the interview recordings, iFLYTEK was used to transcribe the recordings and generate verbatim manuscripts. The research team members completed the dictation and proofreading, revised, and annotated the verbatim manuscripts, and then imported them into Nvivo-12.0 for coding according to a three-level coding program based on the grounded theory. Level coding procedures were followed for coding.

3.   Results

3.1.   Descriptive statistics

The 195 valid questionnaires were from 147 male coaches (75.38%) and 48 female coaches (24.62%). The reason for this imbalance is that the questionnaires were randomly distributed, and more male coaches were participating in the competition than female coaches. The coaches included elementary school teachers (23.59%), high school teachers (27.18%), training agency teachers (24.10%), and others (25.13%). The analysis revealed that less than one-third (32.82%) of the coaches were the first-time coach among the 195 coaches surveyed, which also means that more than two-thirds of the coaches had participated in the WRO two or more times, i.e., 16.41% for 2 times, 17.44% for 3 times, 20.00% for 4 times, and 13.33% for 5 or more times. Logically, the coaches who participated in more WROs would be more experienced in coaching their students in the following WROs.

Most of the participated students were between 13 and 19 years old (62.57%). The students registered for the categories of contest were 121 (62.05%) in the Regular Category, 49 (25.13%) in the Open Category, 21 (10.77%) in the WRO Football, and 4 (2.05%) in the Advanced Robotics Challenge (ARC, the highest level in the WRO). In terms of the competition age groups, the numbers of elementary, junior high, and senior high school students were 59, 68, and 62, respectively, roughly the same across these three age groups.

The questionnaire was scored using a 5-point Likert scale. The average score for the dimensions in the questionnaire was 4.29 (the higher the score, the greater the impact), indicating that the coaches believed that participating in the competition could improve students' abilities in all dimensions. To explore the coaches' scores for each dimension of the questionnaire, descriptive statistical analysis of the seven dimensions was conducted and the results are summarized as follows.

● For learning skills, the average score was 4.36 (higher than other dimensions); especially the average scores for 'learning new knowledge' and 'programming ability' were 4.55 and 4.52, respectively, indicating that coaches were more impressed with the improvement on student's learning skills by participating the WRO.

● The engineering thinking dimension had an average score of 4.27 from the five related sub-questions. Compared with the score of 4.39 in 'clarifying goals', 'analyzing problems', 'formulating plans', and 'building', the average score of 4.17 in 'concentration' indicated a need for students to improve during the whole process of the 'robot project'.

● The emotional engagement, career choice, and problem-solving dimensions had similar average scores of 4.33, 4.34, and 4.32, respectively, indicating that the coaches were highly agreed on the extent of the WRO impact on these three 'indicative' dimensions for students.

● For collaboration quality, the average score was 4.28. The coaches seemed not so satisfied with student's attention to listening to each other's opinions or ideas when dealing with a collaborative task as the score for this sub-question was only 4.17.

● The average score was 4.21 for the global consciousness dimension. The coaches agreed that the competition helped students understand and familiarize the foreign environment, communicate with foreigners, become aware of cultures of other countries, and bring honor to their country but might also ranked this dimension the least important among the seven dimensions with the lowest average score among them in general.

3.2.   Differences for coaches from different professional sectors

One-way variance was used to analyze the differences between coaches from different professional sectors in relation to the various dimensions. The corresponding p-values for emotional engagement, career choice, problem solving, collaboration quality, and global consciousness were all greater than 0.05 (Table 2), indicating no significant difference regarding these five dimensions in terms of which sector a coach was from.

Table 2. 

Coach's professional sectors in relation to the seven dimensions

.

	Sector (mean ± standard deviation)						F	p
	Primary School	Junior High School	Senior High School	University	Training Agency	Others
Learning Skills	22.22±2.45	20.75±3.98	21.41±3.82	22.09±2.70	22.79±2.43	19.67±2.34	2.512	0.031*
Engineering Thinking	21.09±3.05	20.00±3.34	21.47±3.60	21.91±2.39	22.47±2.27	19.33±1.63	3.221	0.008**
Emotional Engagement	8.72±1.39	8.59±1.29	8.40±1.86	9.09±1.14	8.94±1.24	7.83±1.47	1.254	0.286
Career Choice	8.61±1.72	8.47±1.65	8.64±1.56	9.00±0.89	9.00±1.29	7.50±2.07	1.347	0.246
Problem-Solving	12.88±1.78	12.49±2.00	12.93±2.33	13.34±1.20	13.44±1.60	12.17±0.49	1.298	0.266
Collaboration Quality	17.48±2.35	16.49±3.08	17.13±3.19	17.64±1.50	17.11±2.90	16.33±1.21	0.638	0.671
Global Consciousness	21.02±3.70	19.78±4.76	20.96±4.12	22.27±2.76	21.79±3.40	19.96±3.02	1.343	0.248
Note: * means significant at the 0.05 level; ** means significant at the 0.01 level.

---

 | Show Table

DownLoad: CSV

For the learning skills dimension, the Levene statistic (F = 3.064, p = 0.011) of the homogeneity test of variance reached a significance level of 0.05, indicating that the items in this dimension were not homogeneous. The Tamhane's T2, Dunnett's T3, Games-Howell and Dunnett's C post-test methods did not reveal significant differences. However, for engineering thinking, coaches from different sectors resulted in a significant difference at the level of 0.01 (F = 3.221, p = 0.008). Using the least essential difference method for multiple comparisons, we identified significant differences in sector average scores: training agency > junior high school, training agency > primary school, training agency > others, senior high school > junior high school.

3.3.   Differences in competition categories

One-way analysis of variance was used to investigate the categories of competition entered by students in relation to the seven dimensions (Table 3). Because the career choice dimension violated the assumption of homogeneity of variance (F = 3.699, p = 0.013) and the post-test method returned a result of non-significance, competition categories had significant effects only on the global consciousness dimension at a level of 0.05. The average score for the creative competition was significantly higher than that for the regular competition in the global consciousness dimension.

Table 3. 

Competition categories in relation to the seven dimensions

.

	Student competition category (mean ± standard deviation)				F	p
	Regular Category	Open Category	WRO Football	Advanced Robotics Challenge (ARC)
Learning Skills	21.54±3.23	22.50±2.94	21.67±3.76	22.25±3.40	1.078	0.36
Engineering Thinking	20.98±3.28	22.06±2.68	21.48±3.01	22.50±3.11	1.606	0.189
Emotional Engagement	8.53±1.38	9.02±1.31	8.48±2.25	9.00±1.15	1.454	0.229
Career Choice	8.74±1.44	8.88±1.41	7.71±2.15	9.50±1.00	3.555	0.015
Problem-Solving	12.70±2.08	13.51±1.52	13.11±1.44	13.75±1.50	2.456	0.064
Collaboration Quality	16.83±2.84	17.49±2.83	17.85±2.20	17.00±3.56	1.208	0.308
Global Consciousness	20.49±4.06	22.29±3.54	20.90±3.36	22.25±3.40	2.671	0.049*
Note: * means significant at the 0.05 level; ** means significant at the 0.01 level.

---

 | Show Table

DownLoad: CSV

4.   Discussion

4.1.   Influence of professional sectors of coaches and competition category

Although professional sectors of coaches did not cause significant differences in relation to most of the seven dimensions investigated in this study, coaches' scores for engineering thinking exhibited substantial differences. In relation to sub-question 9, "Students can build fully functional robots quickly, " the average score was the lowest from the junior high school teachers (3.84), much lower than the score from the elementary school teachers (4.2), senior high school teachers (4.34 points), university teachers (4.64), and training agency teachers (4.62). In addition, the average score from the training agency teachers (4.62) was much higher than that from the elementary school teachers (4.2). The training agency teachers seemed had more systematic knowledge and a stronger understanding of how to quickly guide students solving the challenges they were facing during the robot project. This may be a reason why many parents are willing to send their children to training agencies to expand their horizons and improve their practical and problem-solving skills in China. Teachers at training agencies are therefore more experienced in guiding students to carry out the hands-on practices, enabling the students to complete their assignments quickly.

WRO has different requirements for different competition categories (regular, open, football, and ARC). Teams must use the designated programming software in the regular competition whereas teams in the open competition can choose the controller, programming software, and building structure freely. Teams in the football competition must build a car-shaped robot that is able to defend and attack whilst the ARC competition is an open challenge in engineering design for university students.

The coaches agreed that participating in the open competition played a key role in cultivating students' global awareness. This may be because the open competition offers students more freedom to excel and innovate by using various techniques and methods as much as they can. As a result, the higher the students achieved in a more challenging competition, the more pride the coaches may feel for their inspiring guidance provided to the students. Hence, for the sub-question "Students want to bring honor to their country, " the coaches in ARC scored the highest score of 4.53 whereas the coaches in the regular competition had an average score of 4.12.

Overall, the data analysis showed that the coaches agreed in general that participation in the WRO can positively affect the participant's abilities in all the seven dimensions investigated in this study. This is consistent with the findings of a study on RoboCup Junior [6]. From the coaches' perspective, the WRO enabled students to gain new knowledge actively, improve their programming and problem-solving skills instantly, and open prospective in STEM-related careers in the future. In relativity, the coaches inclined to the perception that participating in the robotic competition had less effect on concentration or team cohesion among students when they were designing and constructing robots. However, this requires confirmation from more research in the future WROs.

4.2.   Coaches' view on student gains, learning adaptation, and STEM literacy

We extracted the relevant data from the 2018 WRO so as to make a comparative analysis with the data from the 2019 WRO. The themes mentioned by coaches in the 2018 and 2019 WROs in relation to effects, gains, school learning, and STEM literacy are displayed in Figure 1. The themes in red are common in both years.

Figure 1. 

The 2018 and 2019 dimension themes mentioned by coaches

.

DownLoad: Full-Size Img PowerPoint

The coaches believed that the greatest gain for the participated students was the improvement on their patience and resilience, and communication in international settings while dealing with a challenging task. For example, a coach said, "Because this is the first time our students have participated in this type of competition, it is a good opportunity to increase their resilience, " and an Italian coach noted that the students learned to "solve the challenge of the problem, compete with others…learn to work with other teams, communicate more, and be exposed to different cultures." In addition, in general, coaches also valued the positive impacts of participating in the WRO on students' learning engagement, self-awareness, and career choice.

Compared with the school-centered learning and STEM education, the WRO was considered by the coaches to accelerate students' development in learning motivation, communication, artistic skills, and career planning. Coaches agreed that what the students experienced and gained in the WRO could be instantly related and adapted to their daily learning in schools across multiple subjects, and more likely to inspire them to create new initiatives in learning. One coach reported, "The engineering knowledge, mathematics knowledge, physics knowledge, etc., gained during the competition promote students' school subject learning." An Indonesia coach wrote, "the WRO helps them learn mathematics and science and helps them learn English to communicate." Another German coach shared that "the influence was not only in technology-related courses and English courses but also in some soft skills, such as teamwork and discussion."

5.   Conclusion

First, participating in WRO can improve students' engineering abilities and global awareness, promote STEM-related career planning, help students develop initiatives in adaptive learning in schools, and eventually achieve their best performance as much as they can. Second, although the coaches were generally positive towards students' teamwork skills during WRO, they believed that there is more space for students to improve and grow in this area in future WROs. Another area needing more attention would be improving students' concentration and cohesion during the entire robot project, instead of patches of concentration at some stages. In addition to the coaches' focuses on communication between students and team members during the competition and how students could challenge themselves to use their initiatives, manage pressure, work as a team, and reflect on competition outcomes, coaches had some concerns with students' emotional control and ability to make further improvement in an internationally competitive environment. Future research should focus on enhancing the team cohesion, the endured concentration, and the emotional control and improvement in future WROs and STEM education in general.

Acknowledgments

This work was supported by World Robot Olympiad Association [Grant Number: GR001/2019] and Shanghai Normal University [Grant Number: A-9103-18-365218/ JC212-307-AC9103-19-365218]. Many thanks research assistants Mei-Chuan Chien, Yulong Sun, Yaoxian Zhuang, Yan-qiu LIU, Xiran Feng, Huaiyu Lu to support these data collection and analysis. The authors would like to thank the Editor and reviewers' constructive feedback on the original manuscript. We are very grateful for the guidance Professor William Guo provided during the course of revision, especially his thorough editing of the revised manuscript.

Author's biography

Yicong Zhang is a research student in the Department of Educational Technology with College of Education at Shanghai Normal University, China.

Yanan Lu is a teaching and research member in the AI education Center at Shanghai AI Laboratory, Shanghai, China

Xianqing Bao is an associate professor in the Department of Educational Technology with College of Education at Shanghai Normal University, China.

Feng-Kuang Chiang is a director and distinguished professor in the Department of Educational Technology at Shanghai Normal University, China. His research focuses include STEM education, learning space, and integration of information technology and curriculum. He is currently a visiting professor with the School of Education at the Shanghai Jiao Tong University, China.