China primary school students' STEM views, attitudes, self-concept, identity and experiences: A pilot study in Shandong province

1.   Introduction

Science, technology, engineering, and mathematics (STEM) have been important drivers of economic development and global competitiveness at the international and local levels ^[3]; therefore, STEM education has gained significant traction in recent years ^[31,60]. In this paper, we refer to STEM as the integration of content, practices, and dispositions of the four disciplinary domains ^[60,65], while STEM education refers to the teaching and learning of foundational knowledge and the development of skills in formal and informal education settings ^[2]. The idea of STEM education was conceived by the U.S. National Science Foundation to support student-centered learning through problem-solving real-world issues by applying knowledge and practices from the respective disciplinary domains, thereby enhancing the authenticity of learning ^[48]. Through such a learning process, students develop meta-knowledge in 21st-century learning comprising critical and creative thinking, communication, and collaboration in school settings ^[22]. This type of education prepares students to view issues through an interdisciplinary lens, as real-world issues are often complex and require knowledge from more than one disciplinary domain. Ultimately, STEM learning will enable students to be equipped with skills that they need now and will need in the future ^[51].

STEM education has gained traction worldwide; its curriculum transcends primary, secondary, and higher settings ^[6,8]. STEM level has been argued to be especially important in K–12 education as it lays the foundation for a nation's future science and engineering workforce, prosperity, and even security ^[4]. While there are diverse views regarding the nature of K–12 STEM education, stakeholders have paid more attention to integrated forms of STEM teaching and learning. The term integrated STEM refers to the deliberate combination of core disciplinary content from STEM disciplines ^[27,28]. STEM is no longer considered as four isolated disciplines implemented individually; the domain has been treated as a single unified discipline. For the purpose of this paper, STEM refers to integrated STEM unless stated otherwise.

While STEM education has gained early traction in the United States and some parts of Asia (e.g., Thailand and Singapore; see ^[66], it is noteworthy that China has only recently begun to explore STEM education, with discussions starting in the past 10 years. The establishment of the UNESCO Category 1 Institute of STEM Education in Shanghai serves as a significant milestone, underscoring China's growing focus on STEM education ^[45]. In what follows, we provide a broad description of STEM education in China, followed by an account of how STEM education is interpreted in the Shandong Province of China—the site where this reported study was conducted.

2.   Background of the study

STEM education in China may be traced back to the early 2010s when significant government and education reform efforts were expended on integrating STEM into the national curriculum. Such attempts reflect the country's national strategic priority to develop STEM talents to drive the future workforce and contribute to China's global competitiveness. In 2017, the STEM Education White Paper was issued by the National Institute of Education Sciences (NIES) ^[49] as a guide for advancing STEM education across the country. Specifically, the paper underscores the importance of integrating the four disciplines of STEM rather than treating them as isolated subjects, so that students will develop a more holistic understanding of STEM concepts, including their applications. In the following year, the NIES launched The China STEM Education 2029 Action Plan ^[49] to streamline the understanding of STEM education. The plan (a) outlines ways for more organizations to participate in more STEM-related programs, (b) makes STEM education more accessible to students from underrepresented and underserved backgrounds, (c) encourages the adjustments of evaluation methods that better reflect the effectiveness and impact of STEM programs, (d) promotes ways to help students think like scientists and practice like engineers when solving complex problems, (e) suggests the establishment of platforms for STEM research and evaluation, and (f) recommends building STEM schools as part of the infrastructure for supporting STEM education.

In 2014, the Shanghai Education Commission proposed "STEM +" education. The "+" is not merely the expansion of subject knowledge but an upgraded notion of providing a holistic education that underscores the importance of having students solve real-world problems. It was only two years later that the term "STEAM education" first appeared in the official national document, the 13th Five-Year Plan for Education, which is a segment of the 13th Five-Year Plan for Economic and Social Development for Economic and Social Development of the People's Republic of China ^[41]. Specifically, the document reads, "Regions with conditions should actively explore the application of information technology in new education models such as 'crowd creation space', interdisciplinary learning (STEAM education), and maker education" ^[41]. The initiative is part of China's broader efforts to integrate modern technologies, such as big data, machine learning, and artificial intelligence, into the Chinese education system. It is inferred that STEAM education will be a potential driver for fostering 21st-century competencies such as creativity, innovative thinking, and practical skills that will "future-proof" ^[41] learners to take on new jobs.

In 2017, there were calls for STEM education to offer a coherent curriculum encompassing diverse disciplines. The Ministry of Education issued the "Compulsory Education Primary School Science Curriculum Standards", advocating an interdisciplinary learning approach and suggesting that teachers try teaching STEM lessons. The NIES announced the White Paper on STEM Education in China. The main goal of STEM education is to cultivate students' ability to apply their knowledge and solve problems creatively ^[49]. At the same time, the "China Vocational Education 2029 Innovation Action Plan" was released. In addition to formal STEM education in schools, the plan emphasizes the participation of non-formal educational institutions and encourages the commitment of additional social resources to STEM education efforts.

In 2021, schools were tasked by the government to introduce a group of outstanding science popularization talents and relevant science popularization machinery in the way of "invite in" and effectively carrying out after-school science popularization service activities ^[43]. The popularization of science and technology (from now on referred to as "science popularization") is an activity that is undertaken by the whole country to popularize scientific and technological knowledge, promote the scientific spirit, disseminate scientific ideas, and advocate scientific methods. Accordingly, it is an essential basic work for achieving innovative development ^[57]. Teachers are encouraged to promote science education based on inquiry practice, implement interdisciplinary theme learning, and bring the spirit of scientific practices into school campuses ^[44]. An example of such efforts is the work of the team of volunteers from Shandon Normal University that designed and enacted activities in primary schools in Jinan City during National Science Popularization Day.

STEM education has taken many forms in Shandong Province, and this includes scientific popularization, STEM clubs, and STEM camps. Science popularization is a common form aimed at primary and middle school students. It is short-term and mainly stimulates students' interest in science. STEM clubs provide after-school service activity in primary schools, usually one to two classes a week, with a small class teaching system. STEM camps are holiday activities in which teachers lead students to visit science centers or science museums, usually for half a day or one day. In our study, the STEM activity was organized in a science popularization program.

3.   Context of the study

This study was an initiative of the first and third authors to pilot STEM curricula in the three primary schools. This was their first attempt at designing and researching students' learning through a short activity. The duration was kept short as the teachers and students were unfamiliar with the activities that required collaboration and engaged knowledge and practices from across disciplines. This was not a regular curriculum since, like most schools in China, these schools' curriculum was typically monodisciplinary and subject-specific.

The design of the activity was informed by the works of the second author ^[60,65], who suggested adopting solution-centric STEM lesson designs as there were existing solutions to the problem (tasks) assigned to students for problem-solving. In contrast to a problem-centric STEM curriculum that focuses on a complex, extended, and persistent ^[60,65] problem, solution-centric STEM focuses on improving existing solutions. Hence, students are tasked with developing iterations of solutions that are better than the previous versions. Further, a solution-centric STEM curriculum is more manageable for younger children as their worldview may be more limited ^[60,65], and hence, they may be unable to tackle the problems from the user's perspective. As such, the solution-centric STEM curriculum was designed for the students of this pilot study to provide a baseline understanding of how students respond to the curriculum; the findings can be used to inform future curriculum design. When validated, the student survey administered in this study could be used to gather students' views and experiences in integrated STEM curriculum in future studies.

Considering that the audience was primary school students and the total activity time for this pilot and exploratory study was only two hours, we had only designed a simple STEM activity that the young children could do. The theme of the STEM activity was the production of space-related straw flyers. The activity was intended to cultivate the primary school students' interest in spaceflight. Figure 1 shows the STEM curriculum map that provides an overview of the ideas and practices engaged during the activity. The topical content addressed in the four separate STEM disciplinary domains is specified in the diagram. The connections between the disciplinary domains are shown in the linkages to show the explicit connections between the two disciplines. Thicker line borders and arrow lines show a stronger presence of discipline and connections, respectively. For instance, the box for science and mathematics has thicker line borders as compared to engineering and technology, as there were more science and mathematics concepts invoked in the activity. The connections between science and engineering are strong and hence, the arrow line is thick. On the other hand, the connection between science and technology is weak and hence represented by the dotted arrow lines. This diagram provides an overview of the disciplinary ideas and connections involved in this activity. More details about the STEM activity design can be found in Appendix A.

Figure 1.  STEM activity map of the task. The boxes show the disciplinary domain ideas addressed in the activity and the connections between the disciplinary domains.

DownLoad: Full-Size Img PowerPoint

4.   Research questions and significance

This research was a pilot study that sought to examine how students viewed and experienced the STEM lessons. The research question addressed in this study was:

What were the students' views about participating in STEM, attitudes toward STEM, self-concept when participating in STEM, STEM identities, and STEM learning experiences?

At the time of this study, few studies have reported STEM activities conducted in schools or classrooms in Shandong Province. This pilot study was significant in two ways. First, it was the first attempt to examine how the primary school students in Shandong Province viewed and experienced an integrated STEM activity for the first time. As such, their views were not affected by previous experiences that may be likened to integrated STEM with a focus on refining existing solutions. This baseline understanding from a small-scale pilot study was informative for designing and implementing pre-service and in-service STEM teacher education programs where the third author worked as a professor. Second, the survey instruments were adapted from previous studies ^[62,64] and revalidated with the current sample. The validation of this instrument in a different curricular context would contribute to the generalizable use of the instrument for other studies and contexts. Having a common set of validated instruments, especially those that were analyzed using inference statistics (e.g., Rasch analysis used in this study), to study a set of constructs across STEM curricula contexts would help STEM educators, academics, and researchers draw inferences about the affordances of diverse STEM curricula and students' views and experiences in engaging with these different STEM curricula.

Based on the literature in STEM education research, STEM learning outcomes may be broadly categorized as students' views about STEM, attitudes toward STEM lessons, student's self-concept while learning STEM, construction of STEM identities, and experiences in STEM lessons ^[62,64]. These categories form the five constructs in the survey instrument that was developed to measure the outcomes of the STEM programs implemented in Singapore schools ^[62,64] and used in this study. Accordingly, we provide a literature review of the five theoretical constructs reported in the paper.

5.   Literature review

5.1.   Students' views toward STEM education

Students' views about STEM lessons affect their motivation and decision to participate in STEM activities ^[40]. For instance, if students think that STEM lessons are too challenging for them, they may not want to participate in the activities when given a choice ^[71]. An individual's ability to cope with difficulties and overcome adversity by becoming stronger each time they encounter an obstacle may enhance their resilience to hardship and allow them to enjoy their field and pursue a STEM career ^[17]. Interest and motivation can help students learn and understand STEM knowledge, being important factors in motivating students to pursue STEM education ^[50]. The 2018 PISA results showed that although Chinese students rank first in math, science, and reading, less than 20% of students are willing to pursue STEM-related jobs ^[42]. An informal learning environment can improve students' interest in STEM ^[46] and increase students' opportunities to pursue STEM careers ^[33]. Fadzil et al. ^[23] provided evidence on how scientist-teacher-student partnership (STSP) programs enhanced the students' interest in STEM-related subjects. Students' positive perceptions of scientists or engineers have been reported to be positively related to their STEM career aspirations ^[12].

5.2.   Students' attitudes toward STEM lessons

Students' attitudes toward STEM will affect their willingness to participate in STEM lessons, STEM identities, and interests in pursuing STEM careers. Students' attitudes toward STEM influence their willingness to study STEM subjects ^[26,27,70]. Interest and motivation shape students' acquisition and internalization of STEM knowledge and motivate them to pursue STEM studies ^[50]. Fadzil et al. ^[23] revealed how the STSP program enhanced students' interest in learning STEM-related subjects. STSP is a platform that uses authentic, inquiry-based learning to give students and teachers access to the scientific community and allow them to participate in real scientific research ^[29]. Attitudes and beliefs are critical for STEM persistence ^[24]. Several studies have been conducted to explore students' attitudes toward STEM education and students of different grades have different attitudes toward STEM education ^[77]. Students' attitudes toward STEM, such as enjoyment of STEM lessons and career interest in engineering and science, can be influenced significantly by grade levels ^[68]. Students' choices of majors in college and their career interests after graduation are affected by their attitudes toward STEM at the primary school, middle school, or secondary school levels ^[38]. Students' attitudes and views about STEM are influenced by their motivation, experience, and self-efficacy ^[7]. Creative teaching strategies contribute to building up students' attitudes toward STEM. Problem-based learning, for instance, helps to improve students' attitudes toward STEM learning and their exploration of future career options ^[37]. The 6E (engage, explore, explain, engineer, enrich, evaluate)-based learning strategy is also effective for students to join STEM activities and enhance their STEM attitudes and career interests ^[76]. The integrated STEM program positively affected student attitudes toward STEM ^[77]. STEM-PBL (Project-Based Learning) is a student-centered teaching method that helps improve Hong Kong high school girls' attitudes toward STEM learning and enhance their STEM career aspirations ^[72].

5.3.   Students' self-concept in STEM learning

Students' self-concept, comprising confidence and perception of self as a STEM learner and STEM readiness, will affect their willingness and successful participation in STEM learning. Self-concept refers to a person's self-perception ^[56]. The STEM self-concept is a person's perception of themself in STEM education. STEM self-concept is an important aspect for students to enter in STEM area. Self-concept is formed via comparing with other students in STEM education; students who have high self-concept may have higher confidence ^[14]. The key to students' continued involvement in STEM is the development of a healthy STEM-related self-concept ^[62]. Career choice behavior is related to domain-specific academic self-concepts ^[20]. Research shows it is important to develop students' STEM self-concept as it positively predicts STEM career aspirations ^[12]. Previous research showed that students who have high science self-concept will be able to choose a science-relevant career ^[61]. Students have lower mathematics self-concept and less interest in mathematics careers ^[25]. Promoting a variety of careers in STEM fields helps students see the multiple pathways that can emerge from the study of STEM-related subjects ^[58]. Another research outcome showed that secondary students had lower self-concept than primary students, and boys had a higher self-concept than girls ^[12]. Denissen et al. ^[15] provided another example of possible uses of the model. They studied intra-individual coupling between academic achievement, interest, and self-concept of ability in approximately 1,000 children between grades 1 and 12. They found that individuals tended to feel competent and interested in areas where they achieved well and that the strongest coupling was between interest and self-concept of ability. They also observed an increase in the coupling across time.

5.4.   Students' STEM identity

STEM identity indicates one's perception of belonging in the STEM community ^[9]. Students identify the different identities in STEM education, such as 'I see myself as a...'. Problem-solving, collaboration, creative, etc., are 21st-century skills; if students think of themselves as problem-solvers, collaborative people, and creative people, they may agree that these abilities are important to improve in STEM education. STEM identity provides a theoretical lens to understand better the mechanisms by which experiences might influence career choice and persistence ^[18]. STEM identity is the key mediator between STEM media exposure and STEM career interest ^[11]. Dou and Cian ^[19] tested an expanded STEM identity model that could support studies contributing to a deeper understanding of factors related to STEM identity. They identified significant effects of interest and recognition on STEM identity and a significant effect of performance-competence mediated through interest and recognition. Dou et al. ^[18] found that having a stronger STEM identity increases students' likelihood of pursuing STEM careers. Their large-scale study with 15,847 students at 27 U.S. colleges and universities located in 20 different states also found that the odds of participants choosing a STEM career in college increased by 85% with every one-point increase on the STEM identity scale. The findings are coherent with arguments that students' STEM identities will shape their learning outcomes, including decisions in STEM careers.

5.5.   Students' STEM learning experiences

Students' actual experience during the STEM activities provides authentic and first-hand experiences that will affect their decision to participate in future activities and their attitude and view toward STEM ^[7]. Students have difficulty connecting the science experience ^[59]. Students can gain STEM experience from formal and informal learning. The informal STEM learning experience can support students' study in a formal learning environment ^[53] and resolve the limitations of formal school experience, building students' awareness and interest in STEM fields ^[50]. In China, STEM education is not a compulsory program or curriculum; hence, the informal STEM learning experience is essential for students to understand STEM education and STEM-related jobs. Increasing students' STEM opportunities and experiences may foster their STEM interest and engagement and help increase STEM demand ^[69]. Informal STEM experiences in early childhood such as storytelling may have a long-term effect on STEM identity measured in young adults ^[18]. Cohen et al. ^[13] used a STEM identity theoretical framework to examine the effects of early STEM experiences; the conclusion was the lasting impact that early experiences can have on future STEM identity and career intentions. Problem-based learning requires students to collaborate on authentic problems and find solutions to authentic problems in real-world practice, thereby gaining meaningful learning experiences ^[32]. Wan et al. ^[72] developed and validated a scale of STEM-PBL experience, which comprises four dimensions: scientific inquiry, technological application, engineering design, and mathematical processing, with significant correlations between the four dimensions of STEM-PBL experience with students' interest in learning STEM, utilitarian motives for learning STEM, and STEM career aspiration.

In sum, this section discusses the literature related to the five constructs examined in the survey to provide insights to China primary schools students' views, experiences, attitudes, self-concept, and identity in STEM. Accordingly, we report on the research methods in this study.

6.   Research methods

6.1.   Research participants

A total of 332 students from three elementary schools in Jinan City of the Shandong Province in China participated in the study. At the elementary school level, there is a lower focus on examination preparations in comparison to higher grade levels. Students have more time to pursue interests in authentic inquiries that promote the development of general STEM literacy. Therefore, abundant STEM popularization activities were offered in elementary schools to develop students' strong interest in science and scientific literacy among young learners. The participants were Grade-5 and Grade-6 students equipped with practical knowledge and cooperative learning ability. They had strong curiosity and imagination about STEM phenomena. The school teachers assisted in recruiting the students as voluntary research participants in this study. Before students participated in the STEM activity, we obtained the students' and their parents' consent to participate in the study. All students participated in the study on a voluntary basis as an extracurricular activity during the school hours. The research study was conducted by postgraduate students from the first and third authors' university who were part of the science popularization team and majored in STEM education. They had more experience and confidence in designing and enacting STEM activities than the school teachers who had relatively less experience and knowledge about STEM education. As such, the STEM activities were organized and enacted by the postgraduate students. Table 1 below shows the sample sizes of school students who participated in the study and their respective grade levels.

Table 1.  Profile of the school students recruited as research participants for the study.

School label	Grade	Sample size	Age
School A	5	66	11
School B	5	99	11
School C	6	167	12

---

 | Show Table

DownLoad: CSV

Notably, the research participants were students from the upper primary grade levels who were similar in age and shared common science and mathematics knowledge covered in the upper primary syllabuses. As compared to the lower grade levels, the students were more mature and had more years of primary education that allowed them to participate meaningfully in the activities by applying relevant ideas or concepts and practices learned in their regular school curriculum in science and mathematics. Schools A, B, and C were similar in terms of student demographics. As such, the survey responses from the three schools were aggregated and analyzed collectively.

6.2.   Research instruments

The survey instrument was designed by the second author ^[62,67] and administered in two other STEM curricula and contexts. Specifically, it examined middle school students participating in two integrated STEM curricula in Singapore. The survey constructs and items (see Appendix B) were examined by the first and third authors and deemed appropriate for this study as it offered comprehensive coverage of constructs about students' views and experiences in STEM learning that are reported in the literature (see e.g., ^{[35,37,40,67,76]}). The constructs and items were also relevant to the context of the study and the interests of the first and third authors. The survey instrument was translated into Mandarin and the accuracy of translation was checked by the second author, fluent in both spoken and written English and Mandarin, before it was administered to the research participants. After the activity, participants had about 10 minutes to complete the printed survey. As we had little knowledge about the students, a few were randomly selected and interviewed after the survey to obtain further insights into their thoughts about the STEM activity and their experiences. The interviews were used to substantiate and complement the survey findings. The interview questions asked are included in Appendix C.

6.3.   Data analysis

Data were analyzed through Rasch analysis ^[73,74,75]. Rasch has received wide attention and in-depth research and is also a relatively mature theoretical model of project response. Rasch model measures potential variables that cannot be directly observable by the individual's performance on the topic (usually expressed as raw scores). According to the principle of the Rasch model, the probability of a particular individual making a response to a specific problem can be expressed as a simple function of the individual's ability and the difficulty of the problem. The accuracy of an individual in answering a question depends entirely on the comparison between the individual's ability and the difficulty of the item. Rasch analysis considers both students' ability parameters and survey item parameters in the process of analysis. It can analyze the test volume more comprehensively. As such, it has been used in scale design, educational testing, and other fields (see e.g., ^[36,63,64]). For each survey item, students were asked to respond according to a five-point Likert-scale rating (Strongly disagree = 1, Disagree = 2, Neutral = 3, Agree = 4, Strongly Agree = 5). Items with asterisk (*) have been reversed before analysis. The item statistic order measure can be seen in Appendix D. The MNSQ of all items in the five constructs is equal to or less than 2.0. The item and person separation reliabilities data are reported in Appendix E. Item separation reliability of all constructs exceeded 0.90, which indicates that each construct has items with a wide range of difficulty. On the other hand, the person separation reliability for each construct fell between 0.70 and 0.90, which indicates a heterogeneous sample of students surveyed.

The interviews were transcribed and analyzed qualitatively ^[55]. Prescriptive coding was used, and the codes comprised the five constructs measured in the survey. Using these constructs as codes helped to focus the coding on the specific constructs and address the research questions directly. Interrater reliability checks were conducted between two of the authors to ensure that the coding was accurate in terms of interpretation and categorization. Any discrepancies were discussed to ensure that complete alignment was attained.

7.   Findings and discussion

7.1.   Construct A: Views about STEM lessons

Figure 2 shows the Wright map of the students' views about the STEM lessons on making the straw flyer. The item mean (i.e., M on the right side of the ruler) was about 2 logits lower than the person mean (i.e., M on the left side of the ruler). This meant that it was generally easy for the students to agree with the items in Construct A.

Figure 2.  Wright map on Construct A: students' views about STEM lessons.

DownLoad: Full-Size Img PowerPoint

Among the 10 items in Construct A, the following five items were relatively more difficult for the students to agree with:

A6*: STEM activities are not difficult to understand.

A2: STEM activities improve my confidence in getting a STEM job.

A8: I like solving problems in STEM activities.

A5: STEM activities are challenging.

A9: STEM activities are good for my overall learning.

STEM is an interdisciplinary curriculum and hence, connecting concepts across disciplines can be challenging for students who are more attuned to learning content in discrete subject areas ^[48]. In the context of this study, science and mathematics were compulsory subjects for primary students, but they were less familiar with the aspects of technology and engineering. As the activity required knowledge and skills from more than one disciplinary domain, the students might think that STEM was difficult to understand (A6*). However, it was interesting to note that the students did not find the STEM lessons challenging (A5). This could be due to the students' ability to construct the straw flyers despite having difficulty understanding the theoretical principles underlying flight. This finding suggested the need to also not neglect content understanding in STEM teaching so that the activity was both mind-on and hands-on. For instance, more curriculum time could be allocated to discuss the relevant theoretical principles guiding the design of the straw flyers before students start building them.

It was interesting to note that while students looked highly engaged and interested in the STEM activity, they found it relatively difficult to agree that they liked solving problems in STEM lessons (A8). This finding could be related to A5 and A6, as studies (see e.g., ^[67]) have found that students with a weak grasp of the concepts find it challenging to problem-solve. Hence, this underscored the earlier point about helping students learn the relevant concepts before participating in the hands-on activity. This could, in part, explain why students found it difficult to agree that the STEM activity was good for their overall learning. According to one student,

I think that participating in STEM activities helps me with my science studies. In my opinion, STEM activities are fun to learn about science. I'm not sure if they help me with other subjects (Student 300B from School 3, Nov 29, 2023)

The above interview excerpt illuminated the student's valuing of content learning of direct relevance to their subjects. As the student was unable to draw a direct connection, s/he was uncertain about the usefulness of the STEM learning experience. This finding contrasted with Toma et al.'s ^[68] study, who argued that STEM education could promote and improve the learning of each of the disciplines. We argue here that, from the view of students, such learning could take place only if the content knowledge required to problem-solve the task was related to the school subjects. Such efforts are possible, as shown by a study by Baran et al. ^[5], who conducted a study in middle school and found that STEM education programs could help middle school students make connections between schoolwork and daily lives.

While students might not like problem-solving, it should be noted that problem-based learning could improve students' STEM attitudes ^[37], and many STEM activities were designed around solving real problems (B7) ^[60]. In this process, students often became problem-solvers (D5, E5), tried their best to complete the tasks assigned by the teacher (C5), and solved the problem (C9). Problem-solving is arguably one of the abilities that must be mastered in the 21st century, and Alatas's study ^[1] showed a positive effect of STEM learning on students' problem-solving skills (B9).

The students did not think the STEM lessons would help them build confidence in finding a job in STEM (A2). One student said,

STEM courses are really interesting, and I'm very interested in them and want to participate more, but I don't know much about STEM-related jobs, so far only scientists, mathematicians and engineers, so I'm not sure I have the confidence to do STEM jobs" (Student 100B from School 1, Nov 20, 2023).

Notably, the two-hour activity was not enough to support students' confidence building in this area. In our early communication with school principals, we learned that a large portion of the STEM teachers were mathematics and science teachers, and they had relatively little understanding of STEM work. Increasing students' awareness and understanding of STEM careers would require more long-term efforts. The "scientist in the classroom" program was a common outreach model that sought to bring to schools the content expertise and enthusiasm of practicing professional scientists to stimulate student learning, interest in science, and consideration of science careers ^[34]. K–12 students were engaged in authentic, hands-on activities that generated interest in science and new views of science and scientists ^[34]. Such efforts were aligned with the China STEM Education 2029 Action Plan.

Among the 10 items in Construct A, the following five items were relatively easier for the students to agree with:

A1: STEM activities increase my interest in STEM jobs.

A7: I enjoy STEM activities.

A4: I want more STEM activities in school.

A10: What I learned during STEM activities has relevance to my life.

A3: I learn interesting things during the STEM activities.

While the students expressed doubts about the overall usefulness of the STEM lessons (A9), they wanted more STEM lessons (A4) possibly because they enjoyed it (A7). The STEM activity also increased their interest in STEM jobs, although they did not have the confidence that they would get one (A2). Students might have wanted more STEM lessons (A4) because they could learn interesting knowledge that could not be learned in other lessons (A3, E4). Studies showed that students enjoyed learning when the activity resonated with them ^[62,67]. In this study, the students thought that the lessons were relevant to their lives (A10). This probably explained why they enjoyed the STEM lessons. Mohr-Schroeder's study showed that after a five-day STEM camp, the middle school students' interest in STEM had improved, and the students wanted more STEM activities ^[46,76]. However, we were also mindful that such interests could be short-term, requiring more long-term studies to examine the lasting effects of interests in STEM learning due to such efforts.

7.2.   Construct B: Attitudes toward STEM

Figure 3 shows the Wright map of the students' attitudes toward STEM in general. The items mean (i.e., M on the right side of the ruler) was approximately two logits lower than the person mean (i.e., M on the left side of the ruler). This meant that it was generally easy for the students to agree with the items in Construct B.

Figure 3.  Wright map on Construct B: students' attitudes toward STEM.

DownLoad: Full-Size Img PowerPoint

Among the 10 items in Construct B, the following five items were relatively more difficult for the students to agree with:

B4: Everyone should learn about STEM.

B6*: Not only highly trained professionals can understand STEM.

B5: STEM can provide solutions to every problem.

B8*: STEM is not too difficult for me.

B10: My overall learning ability improves after participating in STEM activities.

While there is strong advocacy for STEM education around the world ^[60] due to diverse reasons such as STEM literacy for citizens, the students might not think likewise. In this study, the students did not think that everyone should learn about STEM (B4). They also did not think that STEM could provide solutions to every problem (B5). Below are some interview excerpts:

I think STEM is good, but it may not suit everyone. STEM is not so easy to learn and it needs interest to explore. (Student 200B from School 2, Nov 22, 2023)

STEM activity is pretty good, it can provide some great ideas to solve problems in daily life, but unfortunately it doesn't solve all the problems. (Student 200C from School 2, Nov 22, 2023)

Student 200B agreed that learning STEM would be beneficial, but s/he thought that it may not be something that everyone should be required to learn due to different needs and interests. Student 200C was realistic in saying that STEM did not necessarily address all issues. These findings were coherent with another two studies conducted with middle school students in Singapore ^[62,67], where students expressed similar views.

Many studies reported that STEM activities could improve students' learning abilities, such as scientific creativity ^[21]. However, our research found that the students did not think that their overall learning ability improved after participating in STEM activities (B10). This could be related to the earlier views on the lack of relevance of STEM learning to their subject learning (A9). As the students had limited integrated STEM learning experience, they thought that STEM was difficult (B8) and only professionals could understand STEM (B6).

Among the 10 items in Construct B, the following five items were relatively easier for the students to agree with:

B1: I have positive feelings toward STEM.

B3: STEM improves the quality of life.

B7: STEM is useful for solving practical problems.

B9: Participation in STEM activities improves my problem-solving skills.

B2: STEM is important for society.

Yazici's ^[76] study showed that students developed a positive STEM attitude after they participated in STEM activities. Our study also found that students had positive feelings toward STEM programs (B1, A7) and wanted more STEM classes (A4). While the students did not think that everyone should learn STEM, they agreed that STEM was important and beneficial for society (B2, B3). While students might not be acquainted with the political rhetoric of STEM, they were exposed to the daily applications of STEM that drove the cashless society. This utilitarian and pragmatic view seemed aligned with the findings that while students did not like problem-solving (A8), they recognized the usefulness of STEM to solving practical problems (B7) and that participating in STEM could improve their problem-solving skills (B9). These findings are also coherent with previous findings in studies conducted with Singapore middle school students ^[62,67].

7.3.   Construct C: Students' self-concepts on STEM

Figure 4 shows the Wright map of the students' self-concepts on STEM learning. The item mean (i.e., M on the right side of the ruler) was approximately 1.4 logits lower than the person mean (i.e., M on the left side of the ruler). This meant that it was generally easy for the students to agree with the items in Construct C.

Figure 4.  Wright map on Construct C: students' self-concepts in STEM learning.

DownLoad: Full-Size Img PowerPoint

Among the 10 items in Construct C, the following six items were relatively more difficult for the students to agree with:

C6: I am better at STEM subjects than other subjects.

C8: I perform better than others during STEM activities.

C1: I have developed a good understanding of STEM.

C10*: Learning STEM is not difficult for me.

C2: I am confident in my STEM knowledge.

C7: I can learn new STEM ideas quickly.

The students did not think that they were better in STEM subjects than other subjects (C6). In comparison to other students, they also did not think that they could perform better in STEM lessons (C8). The students seemed to express lower confidence in STEM (C2), and this was coherent with their views that they did not think that they had a good understanding of STEM (C1), and that learning STEM was difficult (C10) and not quick to learn (C7). The development of STEM education in China was uneven, with different policies in each province and different exposure opportunities for everyone ^[39]. One student expressed the following:

I participated in some STEM activities and learned some STEM knowledge, but these were far from enough compared to my math classes. I think some of my classmates can learn better than me. STEM covers a wide range of topics. I will work harder to learn STEM in the future (Student 200A from School 2, Nov 22, 2023).

Previous studies have found that having the confidence to learn was important for students to fully participate in learning ^[10,30]. The students in this study needed more support in confidence-building as a precondition for them to learn. Students' STEM confidence may vary by race and gender. African American and Hispanic men reported higher levels of STEM confidence than White men, and women continued to report lower average STEM confidence than White men ^[35].

Among the 10 items in Construct C, the following four items were relatively easy for the students to agree with:

C9: I can solve STEM problems.

C3: I have experienced success in STEM activities.

C4: I can do well during the STEM activities.

C5: I can complete the tasks in STEM activities.

Although students do not like solving problems in STEM activities (A8), they thought that they could solve STEM problems (C9). Even if they were not confident in their STEM knowledge (C2), they thought that they could complete the tasks (C5) and did well in STEM lessons (C4). This could be because they had experienced success in STEM lessons (C3) and were able to complete the STEM tasks (C5). One student commented,

Actually, I don't have enough STEM knowledge, but I think STEM is essential for children to learn and I want to try my best to complete the task in STEM lessons. (Student 200A from School 2, Nov 22, 2023)

7.4.   Construct D: STEM identity

Figure 5 shows the Wright map of the students' STEM identity. The item mean (i.e., M on the right side of the ruler) was approximately 0.5 logits lower than the person mean (i.e. M on the left side of the ruler). This meant that it was generally easy for the students to agree with the items in Construct D.

Figure 5.  Wright map on Construct D: students' STEM identity.

DownLoad: Full-Size Img PowerPoint

Among the 10 items in Construct D, students found it relatively more difficult to agree that a STEM professional was someone who was a/an:

D3: budding STEM professional.

D4: critical thinker.

D8: engineering person.

D9: collaborative person.

D2: member of the STEM community.

Among the 10 items in Construct D, students found it relatively easier to agree that a STEM professional was someone who was a/an:

D10: designer.

D5: problem-solver.

D7: innovative person.

D6: creative person.

D1: STEM student.

Students may not be able to identify themselves with STEM professionals due to their lack of acquaintance with such a person. As a school student, they may not have experience with engineers. For instance, one student said, "I think engineering is a difficult job that needs plenty of professional knowledge. I can draw a draft but not be an engineer" (Student 200C from School 2, Nov 22, 2023). Due to a lack of experience with collaborative STEM learning experience that requires critical thinking, they may not identify with a STEM professional as a collaborative person or member of the STEM community. Many studies have shown that STEM activities contributed to students' ability to collaborate, but in this study, it was difficult for students to identify as collaborators (D9). One student commented, "I think that in this STEM activity, I can understand the task and make the straw flyer by myself without having to cooperate with other students." (Student 100C from School 1, Nov 20, 2023)

On the other hand, students might be exposed to design work that requires them to create or innovate and problem-solve. One student said, "STEM activity can improve my creativity! I want to be a creative person. Because scientists are innovative people, they are so smart and can innovate meaningful things" (Student 100B from School 1, Nov 20, 2023). Many students knew little about STEM education and had taken part in STEM activity just once or twice, so most of them had a self-identity of STEM students (D1) instead of budding STEM professionals (D3). Learning community is not a new concept, but it is rarely practiced in China, especially in connection with STEM education ^[78]. In China, there are some STEM clubs at school, and the STEM community is less often seen. This could be, in part, why students found it difficult to identify themselves as a member of the STEM community (D2). Their views on STEM identity illuminated the types of exposure they had in STEM and the opportunities it provided. An example of this was their exposure to a larger community of STEM learners so that they understood STEM inquiry was collaborative in nature.

7.5.   Construct E: STEM learning experiences

Figure 6 shows the Wright map of the students' STEM learning experiences. The item mean (i.e., M on the right side of the ruler) was approximately 1.8 logits lower than the person mean (i.e., M on the left side of the ruler). This meant that it was generally easy for the students to agree with the items in Construct E.

Figure 6.  Wright map on Construct E: Students' STEM learning experiences.

DownLoad: Full-Size Img PowerPoint

Among the 10 items in Construct E, the following five items were relatively more difficult for the students to agree with:

E7: The STEM-SP# helped me think and decide on my future career.

E9: I learned how to apply my knowledge in STEM during the STEM-SP.

E6: I am interested in joining other events that are similar to the STEM-SP.

E10: The STEM-SP made me more interested in STEM.

E8: I have a better understanding of STEM during the STEM-SP.

(#STEM-SP: STEM-Scientific Popularization)

The items in Construct E were related to this STEM activity (STEM-SP) and their previous STEM experience. The students thought that participating in this STEM activity did not make them interested in STEM (E10). There are two possible reasons for this: (1) The students were already interested in STEM and did not think that their interests in STEM were inspired by this activity. (2) The students thought they had participated in similar activities before and wanted to try new topics. As such, they did not think that they wanted to participate in similar STEM activities (E6). As mentioned earlier, students did not feel confident enough to pursue STEM careers (A2); hence, participating in STEM activities did not make them think too much about their future careers (E7). STEM is complex and relevant to the real world ^[60]; as such, it was not surprising that students could claim to have a better understanding of STEM after this brief opportunity (E8). It was also difficult for them to apply what they had learned in the STEM activity, as the concepts might not have been introduced in the regular lessons (E9). Again, this suggested that more careful planning considering the theoretical concepts involved in problem-solving had to be underscored.

Among the 10 items in Construct E, the following 5 items were relatively easy for the students to agree with:

E3: I learned to collaborate more effectively during the STEM-SP.

E5: I learned to solve problems more efficiently during the STEM-SP.

E1: I enjoyed the STEM-SP.

E2: I learned useful skills during the STEM-SP.

E4: I gained new knowledge on STEM during the STEM-SP.

21st century competencies include critical thinking (D4), problem-solving (D5), communication, collaboration (D9), and creativity (D6) ^[54]. Students acquired useful skills in STEM activity (E2) and they liked the STEM activity (E1). They learned to collaborate more effectively during the STEM activity (E3). Raksapoln ^[52] showed that the increase in students' collaboration skills was due to the activities involved in STEM learning through engineering design. Effective communication among team members is very important for creativity, especially in new product development teams. While the students could not establish direct connections between the activity to the school subjects, they thought that they had gained new knowledge through the STEM activity (E4). As such, the STEM activity offered a complementary curriculum to the regular lessons.

8.   Implications

Based on the findings, several implications could be drawn for STEM educators. Teachers may wish to pay attention to a few main points when designing STEM lessons for students. For instance, they may wish to make explicit connections between the STEM lessons and the regular subject learning outcomes so that students can better appreciate the former and not view it as extra lessons. Before the hands-on STEM inquiry, teachers may wish to identify relevant content knowledge that students would need to apply to their prototyping. Students would be equipped with substantive content knowledge to design and explain their prototypes more meaningfully. Clarity in the role of disciplinary knowledge and the related inquiry would allow for more intentional design of STEM problems for students to learn higher-order knowledge ^[59]. This study adopted a solution-centric design in the STEM activity that was accepted by students. Teachers could try to design similar STEM activities in the future.

The findings of this study contribute to the growing STEM education literature that argues for the importance of early STEM education ^[18]. As mentioned earlier, Dou et al. found a strong predictive power between STEM identity and STEM career. Our study found that the students did not identify with STEM professionals. This suggests that more opportunities have to be provided for students to perceive themselves as a member of a larger STEM community and not simply a classroom learner ^[62,67]. This broad mindset could help students appreciate the purpose of STEM learning that transcends beyond content acquisition to membership assimilation. It could provide students with a head start in thinking about pursuing STEM careers. We need early intervention in the educational system if we wish to influence career choices focused on STEM ^[47]. Dickerson et al. ^[16] investigated the STEMBASE program's impact on students' attitudes and perceptions regarding STEM education and careers. The results showed that students had a positive attitude toward STEM and hoped for more similar STEM activities and an increase in career education opportunities ^[16]. Such opportunities may also inadvertently address confidence issues reported in our study. These are two areas that the teachers in our pilot study could focus on when enacting STEM lessons.

This study also contributes to the literature in terms of research instrumentation. This instrument has been adapted from another study that examined two other integrated STEM programs for middle school students in Singapore ^[62,67]. The instrument is validated for the context of this study and shows potential to be used in other research contexts and grade levels. This is a significant contribution to STEM education research that currently lacks sufficient research instruments for measuring the comprehensive set of constructs discussed in this paper. Teachers who want to evaluate the outcomes of STEM learning could use the reported instrument and revalidate it for use. They could also use it for diverse programs and track changes in students' responses over time using the same Rasch analysis and WinSTEP.

9.   Conclusions

In summary, this is a significant study as it reports the situation of STEM education in China from the student's perspective. The findings show that primary students had positive views, attitudes, self-concept, identity, and experiences toward STEM. The primary school students who participated in STEM activities considered themselves more designers than engineers or critical thinkers. Students may not have gained a deep understanding of some difficult STEM topics, but they learned something useful. This study contributed to the first application of solution-centric STEM curriculum design in China and underscored the importance of STEM education at the primary grade levels. This survey was validated for the study and a future study can be conducted with more schools to draw more generalized findings.

10.   Limitation

The findings cannot be generalized to other schools in or outside of China, as this was a pilot and exploratory study in Shandong, China, which was at its nascent stage of STEM curriculum adoption. The sample size was relatively small compared to the whole student population in the city, province, and country. For this pilot study, only one activity was designed and trialed at the schools, as the teachers and students were new to the idea of an integrated STEM curriculum. There were limitations of this study that could be addressed by additional studies with larger sample sizes drawn from a larger geographic location and a greater variety of STEM curricula. Additionally, we acknowledge that the implementation members from the science popularization team that was set up for about a year at the time of the study may not have sufficient knowledge and experience with connecting the disciplines, which could have affected the findings.

We also acknowledge that students' responses to the survey should always be contextualized to the STEM activities that they participated in. Therefore, these findings could not be generalized to all STEM activities. In this case, the STEM activity had a stronger focus on science and engineering, so students with different interests in the various disciplines may respond differently to the survey items. Also, students' STEM learning experiences should be shaped by the resources available to them. In the reported STEM activity, students had the same and limited amount of resources to work with. The findings could be different if conducted in another context, where students had different quantities and types of materials to build the straw flyer.

Author contributions

Feiyue Wang: Data curation, formal analysis; Tang Wee Teo, Shoubao Gao: Supervision; All authors have contributed to the writing of this paper. All authors have read and approved the final version of the manuscript for publication.

Acknowledgments

We would like to thank the constructive feedback provided by the reviewers. We would also like to thank the teachers and students at the participating schools for their time in this study.

Conflict of interest

We declare that there is no conflict of interest in this paper.

Ethics declaration

Consent was obtained from the school and the students' parents or guardians prior to data collection.

Appendix

Appendix A.   Details about the activity

Table 2.  The details of STEM activity.

Process	Context	Teachers' Activity	Students' Activity	Duration
Identity problem	Introduce the topic	Show flying and pictures of wooden planes built by the Wright brothers to introduce the theme of airplanes	Watch the big screen to answer the teacher's questions	10mins
	Ask a question	Reality question: How do airplanes fly? What principle is applied? Could you use a piece of paper to simulate the flight of an airplane?
Specific solution	The single straw flying machine and design picture	The teacher provided students with a simple design drawing and model of a single straw flyer	Students carefully look at the design drawings and models provided by the teacher to understand each structure	25mins
	Show the setup of materials	The teacher prepares the materials the students need, such as straws, cardboard, paper clips, tape, etc	Observe the spare materials provided by the teacher and think about how to make a straw flyer according to the materials provided
	First production of a single straw flyer	The teacher guides the students to make a single straw flyer	Students draw a single straw flyer on a drawing and select materials to make it
Formulate solution	Explain force analysis and Bernoulli's principle	The teacher explained in detail the force analysis of aircraft flight and the Bernoulli principle involved	Watch the blackboard and multimedia, listen to the teacher's explanation of related science and technology knowledge	10mins
	Production improvement	Lead the students to make the straw plane twice according to the improved design drawing (change straws of different thickness, increase the number of straws, and adjust the size of the ring)	Students work in groups to discuss how to make another straw flyer
Implement new solution	Make a multi-straw flyer	Teachers guide students to come up with creative solutions	After discussion, the students drew a design drawing of the new straw flyer and tried to make it	30mins
Review solution	Reflect on the making of the product	The teacher selects representative works and asks students to show the design drawings and production steps	Students present their work and explain the reasons for the design	10mins
	Evaluation	Teachers give suggestions based on students' work and lead students to complete the mutual assessment	Fill in the personal evaluation scale
New problem	Leave the question for thought	The teacher asks questions: What shape can a cardboard be other than a ring shape?	Students think outside of class	5mins

---

 | Show Table

DownLoad: CSV

Figure 7.  Photograph of first made by students.

DownLoad: Full-Size Img PowerPoint

Figure 8.  Photograph of teacher guidance of making process.

DownLoad: Full-Size Img PowerPoint

Figure 9.  Photography of single straw flyer.

DownLoad: Full-Size Img PowerPoint

Figure 10.  Photograph of the straw flyer second made by students.

DownLoad: Full-Size Img PowerPoint

Figure 11.  Photograph of teacher is showing the production.

DownLoad: Full-Size Img PowerPoint

Figure 12.  Photograph of the prototype.

DownLoad: Full-Size Img PowerPoint

Appendix B.   Survey instrument

(1) General Views About STEM Lessons

(2) My Attitudes Toward STEM

(3) My Self-concept in Learning STEM

(4) Construction of STEM-identities

(5) My Experience in the STEM program

Appendix C.   Interview questions

1. What's the difference between Science class and STEM activity? What is regular class like?

2. Do you like this activity? Why?

3. Do you have the opportunity to be exposed to STEM activities in your life? What topics have you participated in?

4. Do you think STEM activities are helpful to your studies? Why?

5. Do you like working with others? What do you think are the benefits of cooperation?

6. Through this activity, what have you learned and what abilities have been improved?

7. Do you know some STEM relevant careers? Why?

8. Can you problem-solve in STEM activities?

9. Do you think STEM activities are difficult? Why?

Appendix D.   Item statistics measure order

Table 3.  Item statistics measure order for Construct A.

Input: 225 Person 10 Item Reported: 225 Person 10 Item 5 Cats WINSTEPS 3.66.0
Person: Real sep.: 2.30 Rel.:.84... Item: Real sep.: 6.85 Rel.:.98
Item statistics: MEASURE ORDER
ENTRR	TOTAL			MODEL	INFIT		OUFIT		PT-MEASURE		EXACT	MATCH
NUMBER	SCORE	COUNT	MEASYRE	S.E.	MNSQ	ZSTD	MNSQ	ZXTD	CORR.	EXP.	OBS%	EXP%	DISPLACE	ITEM
6	756	225	1.87	.11	1.99	7.8	1.99	7.6	.52	.76	36.2	57.4	.00	A6
8	881	225	.43	.11	.79	-2.4	.75	-2.7	.79	.71	62.4	56.3	.00	A8
2	882	225	.42	.11	.91	-.9	.89	-1.0	.74	.71	60.5	56.4	.00	A2
5	906	224	.09	.11	1.29	2.8	1.33	2.9	.60	.70	53.6	57.3	.00	A5
9	913	225	.05	.11	.89	-1.2	.89	-1.0	.73	.70	58.1	57.3	.00	A9
7	927	225	-.12	.11	.80	-2.3	.73	-2.6	.76	.69	62.9	57.5	.00	A7
1	939	225	-.27	.11	.65	-4.1	.60	-4.0	.76	.68	71.9	59.0	.00	A1
4	948	225	-.38	.11	.85	-1.6	.78	-1.9	.72	.67	60.5	58.9	.00	A4
10	960	225	-.54	.11	.86	-1.5	.86	-1.1	.69	.66	62.4	60.5	.00	A10
3	1028	225	-1.56	.13	.84	-1.5	.71	-1.7	.62	.58	71.4	69.3	-.01	A3
MEAN	914.0	224.9	.00	.11	.99	-.5	.95	-.6			60.0	59.0
S.D.	66.5	.3	.83	.01	.37	3.2	.39	3.2			9.5	3.6

---

 | Show Table

DownLoad: CSV

Table 4.  Item statistics measure order for Construct B.

Input: 297 Person 10 Item Reported: 297 Person 10 Item 5 Cats WINSTEPS 3.66.0
Person: Real sep.: 1.66 Rel.:.73... Item: Real sep.: 6.14 Rel.:.97
Item statistics: MEASURE ORDER
ENTRR	TOTAL			MODEL	INFIT		OUFIT		PT-MEASURE		EXACT	MATCH
NUMBER	SCORE	COUNT	MEASYRE	S.E.	MNSQ	ZSTD	MNSQ	ZXTD	CORR.	EXP.	OBS%	EXP%	ITEM
4	1010	297	.76	.07	.94	-.7	.96	-.5	.61	.63	41.2	42.9	B4
6	1011	297	.75	.07	1.95	9.4	1.97	9.3	.32	.63	31.3	42.9	B6
5	1100	297	.34	.07	.97	-.4	1.02	.3	.58	.60	45.4	43.6	B5
8	1129	297	.20	.07	1.47	5.0	1.42	4.3	.51	.59	42.3	44.5	B8
10	1159	297	.04	.07	.79	-2.7	.83	-2.0	.66	.57	52.2	45.7	B10
1	1193	297	-.14	.07	.88	-1.5	.83	-1.9	.63	.56	50.9	47.4	B1
9	1232	297	-.36	.08	.60	-5.4	.58	-5.0	.69	.53	59.1	49.1	B9
7	1234	297	-.37	.08	.76	-3.1	.79	-2.3	.62	.53	59.5	49.1	B7
3	1244	297	-.43	.08	.71	-3.7	.67	-3.7	.64	.53	61.5	49.4	B3
2	1296	297	-.78	.08	.82	-2.1	.77	-2.1	54	.49	57.0	54.4	B2
MEAN	1160.8	297.0	.00	.07	.99	-.5	.98	-.4			50.0	46.9
S.D.	93.0	.0	.49	.01	.39	4.2	.39	4.0			9.3	3.5

---

 | Show Table

DownLoad: CSV

Table 5.  Item statistics measure order for Construct C.

Input: 301 Person 10 Item Reported: 301 Person 10 Item 5 Cats WINSTEPS 3.66.0
Person: Real sep.: 2.57 Rel.:.87... Item: Real sep.: 5.93 Rel.:.97
Item statistics: MEASURE ORDER
ENTRR	TOTAL			MODEL	INFIT		OUFIT		PT-MEASURE		EXACT	MATCH
NUMBER	SCORE	COUNT	MEASYRE	S.E.	MNSQ	ZSTD	MNSQ	ZXTD	CORR.	EXP.	OBS%	EXP%	ITEM
8	973	301	.89	.08	1.06	.8	1.03	.4	.72	.75	58.6	53.7	C8
6	977	301	.86	.08	1.07	.9	1.08	.9	.73	.75	56.5	53.7	C6
1	1081	301	.15	.08	.79	-2.8	.78	-2.7	.78	.73	61.6	52.4	C1
10	1092	301	.07	.08	1.99	9.7	2.22	9.9	.49	.73	45.2	52.5	C10
2	1103	301	-.01	.08	.89	-1.5	.85	-1.7	.77	.72	60.6	52.7	C2
7	1110	300	-.08	.08	.77	-1.0	.74	-3.1	.78	.72	59.8	51.9	C7
9	1116	301	-.10	.08	1.01	.2	1.00	.0	.72	.72	56.2	51.8	C9
3	1146	301	-.31	.08	.84	-2.1	.81	-2.2	.77	.71	57.9	51.8	C3
4	1178	301	-.54	.08	.69	-4.4	.65	-3.9	.77	.70	62.7	53.2	C4
5	1232	301	-.94	.09	.77	-3.0	.72	-2.8	.74	.68	65.4	56.5	C5
MEAN	1100.8	300.9	.00	.08	.99	-.5	.99	-.5			58.4	53.0
S.D.	75.9	.3	.53	.00	.36	3.8	.43	3.8			5.2	1.3

---

 | Show Table

DownLoad: CSV

Table 6.  Item statistics measure order for Construct D.

Input: 255 Person 10 Item Reported: 255 Person 10 Item 5 Cats WINSTEPS 3.66.0
Person: Real sep.: 2.96 Rel.:.90... Item: Real sep.: 7.46 Rel.:.98
Item statistics: MEASURE ORDER
ENTRR	TOTAL			MODEL	INFIT		OUFIT		PT-MEASURE		EXACT	MATCH
NUMBER	SCORE	COUNT	MEASYRE	S.E.	MNSQ	ZSTD	MNSQ	ZXTD	CORR.	EXP.	OBS%	EXP%	ITEM
4	732	255	.86	.09	1.04	.5	.99	.0	.81	.80	58.7	51.7	D4
3	743	255	.77	.09	1.08	.9	1.04	.4	80	.80	57.8	51.1	D3
8	755	255	.67	.09	.79	-2.4	.77	-2.6	.84	.80	65.2	51.6	D8
9	804	255	.27	.09	.80	-2.4	.80	-2.3	.84	.80	61.7	50.8	D9
2	808	255	.24	.09	.99	-.1	1.01	.2	.80	.80	57.8	50.8	D2
5	850	255	-.09	.09	.66	-4.2	.71	-3.4	.84	.79	62.6	50.4	D5
7	857	255	-.15	.09	.78	-2.7	.75	-2.8	.83	.79	54.8	50.2	D7
10	857	255	-.15	.09	1.08	.9	1.08	.9	.80	.79	57.8	50.2	D10
6	932	255	-.75	.09	.98	-.2	.99	.0	.78	.78	52.6	49.8	D6
1	1037	255	-1.67	.10	1.98	8.0	1.95	5.5	.61	.74	39.6	57.4	D1
MEAN	837.5	255.0	.00	.09	1.02	-.2	1.01	-.4			56.9	51.4
S.D.	88.6	.0	.73	.00	.35	3.2	.34	2.5			6.7	2.1

---

 | Show Table

DownLoad: CSV

Table 7.  Item statistics measure order for Construct E.

Input: 332 Person 10 Item Reported: 332 Person 10 Item 5 Cats WINSTEPS 3.66.0
Person: Real sep.: 2.49 Rel.:.86... Item: Real sep.: 4.59 Rel.:.95
Item statistics: MEASURE ORDER
ENTRR	TOTAL			MODEL	INFIT		OUFIT		PT-MEASURE		EXACT	MATCH
NUMBER	SCORE	COUNT	MEASYRE	S.E.	MNSQ	ZSTD	MNSQ	ZXTD	CORR.	EXP.	OBS%	EXP%	ITEM
7	1271	332	1.11	.10	1.50	4.7	1.56	5.0	.81	.85	36.2	53.2	E7
9	1336	332	.50	.10	1.19	2.0	1.25	2.4	.82	.83	63.4	56.2	E9
6	1355	332	.31	.10	1.12	1.2	1.08	.8	.82	.83	64.0	57.5	E6
8	1387	332	-.02	.10	.99	-.1	1.03	.4	.82	.81	70.2	58.2	E8
10	1387	332	-.02	.10	1.05	.6	1.04	.4	.81	.81	60.0	58.2	E10
3	1400	332	-.16	.10	.73	-3.0	.69	-3.2	.85	.81	69.4	59.2	E3
5	1401	332	-.17	.10	.89	-1.1	.99	-.1	.82	.80	68.9	59.3	E5
1	1409	332	-.26	.11	.99	-.1	1.01	.2	.81	.80	67.2	60.2	E1
2	1430	332	-.50	.11	.69	-3.6	.62	-3.6	.84	.79	71.5	61.8	E2
4	1452	332	-.77	.11	.78	-2.4	.71	-2.4	.81	.77	78.7	62.9	E4
MEAN	1382.8	322.0	.00	.10	.99	-.2	1.00	.0			64.9	58.7
S.D.	48.8	.0	.51	.00	.23	2.4	.27	2.5			10.8	2.6

---

 | Show Table

DownLoad: CSV

Appendix E.   Person and item separation reliabilities

Table 8.  Person and item separation reliabilities for Construct A.

Construct A.xlsx
	225 Input		225 Measure		Infit		Outfit
Person	Total	Count	Measure	Realse	IMSQ	ZSTD	OMNSQ	ZSTD
Mean	40.6	10.0	2.72	.67	1.01	.0	.95	.0
SD	6.2	.1	1.84	.35	.43	1.1	.44	1.0
REAL RMSE	.76	True SD	1.68		SEPARATION		2.22	PERSON
RELIABILITY	.83
	10 Input		10 Measure		Infit		Outfit
Item	Total	Count	Measure	Realse	IMSQ	ZSTD	OMNSQ	ZSTD
Mean	914.0	224.9	.00	.12	.99	-.5	.95	-.6
SD	66.5	.3	.83	.01	.37	3.2	.39	3.2
REAL RMSE	.12	True SD	.82		SEPARATION		6.85	ITEM
RELIABILITY	.98

---

 | Show Table

DownLoad: CSV

Table 9.  Person and item separation reliabilities for Construct B.

Construct B.xlsx
	297 Input		297 Measure		Infit		Outfit
Person	Total	Count	Measure	Realse	IMSQ	ZSTD	OMNSQ	ZSTD
Mean	39.1	10.0	1.25	.50	1.05	.0	.98	-.1
SD	6.0	.0	1.09	.24	.65	1.4	.59	1.4
REAL RMSE	.55	True SD	.94		SEPARATION		1.69	PERSON
RELIABILITY	.74
	10 Input		10 Measure		Infit		Outfit
Item	Total	Count	Measure	Realse	IMSQ	ZSTD	OMNSQ	ZSTD
Mean	1160.8	297.0	.00	.08	.99	-.5	.98	-.4
SD	93.0	.0	.49	.01	.39	4.2	.39	4.0
REAL RMSE	.08	True SD	.48		SEPARATION		6.14	ITEM
RELIABILITY	.97

---

 | Show Table

DownLoad: CSV

Table 10.  Person and item separation reliabilities for Construct C.

Construct C.xlsx
	301 Input		301 Measure		Infit		Outfit
Person	Total	Count	Measure	Realse	IMSQ	ZSTD	OMNSQ	ZSTD
Mean	36.6	10.0	1.40	.56	1.00	-.2	.99	-.2
SD	7.5	.1	1.67	.26	.67	1.5	.67	1.5
REAL RMSE	.62	True SD	1.55		SEPARATION		2.50	PERSON
RELIABILITY	.86
	10 Input		10 Measure		Infit		Outfit
Item	Total	Count	Measure	Realse	IMSQ	ZSTD	OMNSQ	ZSTD
Mean	1100.8	300.9	.00	.09	.99	-.5	.99	-.5
SD	75.9	.3	.53	.01	.36	3.8	.43	3.8
REAL RMSE	.09	True SD	.53		SEPARATION		5.93	ITEM
RELIABILITY	.97

---

 | Show Table

DownLoad: CSV

Table 11.  Person and item separation reliabilities for Construct D.

Construct D.xlsx
	255 Input		255 Measure		Infit		Outfit
Person	Total	Count	Measure	Realse	IMSQ	ZSTD	OMNSQ	ZSTD
Mean	32.8	10.0	.70	.63	1.03	-.2	1.01	-.2
SD	9.9	.0	2.33	.41	.66	1.6	.63	1.5
REAL RMSE	.76	True SD	2.20		SEPARATION		2.91	PERSON
RELIABILITY	.89
	10 Input		10 Measure		Infit		Outfit
Item	Total	Count	Measure	Realse	IMSQ	ZSTD	OMNSQ	ZSTD
Mean	837.5	255.0	.00	.10	1.02	-.2	1.01	-.4
SD	88.6	.0	.73	.01	.35	3.2	.34	2.5
REAL RMSE	.10	True SD	.72		SEPARATION		7.46	ITEM
RELIABILITY	.98

---

 | Show Table

DownLoad: CSV

Table 12.  Person and item separation reliabilities for Construct E.

Construct E.xlsx
	332 Input		332 Measure		Infit		Outfit
Person	Total	Count	Measure	Realse	IMSQ	ZSTD	OMNSQ	ZSTD
Mean	41.7	10.0	3.00	.96	1.04	-.3	1.00	-.4
SD	8.6	.0	2.67	.59	1.06	2.0	1.07	2.0
REAL RMSE	1.13	True SD	2.42		SEPARATION		2.14	PERSON
RELIABILITY	.82
	10 Input		10 Measure		Infit		Outfit
Item	Total	Count	Measure	Realse	IMSQ	ZSTD	OMNSQ	ZSTD
Mean	1382.8	332.0	.00	.11	.99	-.2	1.00	.0
SD	48.8	.0	.51	.00	.23	2.4	.27	2.5
REAL RMSE	.11	True SD	.49		SEPARATION		4.58	ITEM
RELIABILITY	.95

---

 | Show Table

DownLoad: CSV