Research on performance evaluation of higher vocational education informatization based on data envelopment analysis

1.   Introduction

Advanced educational information technology has promoted the improvement of educational quality and optimized the talent training model. In February 2019, the General Office of the State Council of the People's Republic of China issued the "China's Education Modernization 2035" plan. The plan emphasizes that information technology is a powerful support for educational modernization and calls for accelerating educational reforms in the information age. Vocational education is essential to China's education system ^[1]. In 2014 and 2017, the General Office of the State Council of the People's Republic of China and the Ministry of Education of the People's Republic of China issued the "Decision on Accelerating the Development of Modern Vocational Education" ^[2] and the "Opinions on Accelerating the Development of Vocational Education Informatization ^[3], " both emphasizing the need to accelerate the informatization construction of vocational education and apply modern information technology to various aspects of vocational education and teaching.

Although the national government has repeatedly placed educational informatization in a strategic position to support and lead educational modernization, various issues still arise in the informatization process of higher vocational colleges. The allocation of informatization resources is often unreasonable, with many colleges adopting a subjective and fragmented approach in funding, human resources and equipment allocation across departments and units. As a result, the output fails to meet the expected benefits, and the problem of a low input-output ratio becomes increasingly evident ^[4]. The direction of informatization performance evaluation needs to be more balanced, focusing more on resource orientation and quantity, often leading to an overemphasis on input while neglecting output.

Previous research on educational informatization has mostly focused on the eastern and coastal regions, emphasizing the informatization of basic education. Henan Province is located in the central region of China, where there is a lack of theoretical support and performance data support for the informatization of higher vocational education. In order to further promote the development of educational informatization in Henan Province and optimize resource allocation, it is necessary to conduct a systematic evaluation and research on the performance of educational informatization in higher vocational colleges in Henan Province ^[5].

2.   Relevant literature

General higher vocational education refers to vocational learning, including EHEA courses or qualifications ^[6]. Currently, higher vocational education focuses on cultivating innovative and applied technical talents ^[7]. The goals of talent cultivation highlight professionalism, diversity and practicality ^[8].

The objectives of talent cultivation determine that the informatization of higher vocational education should not only focus on improving students' information-based vocational skills but also emphasize the improvement of teachers' information-based teaching ability and literacy ^[9]. Therefore, in order to exert students' initiative and thoroughly reflect their role as the subject of learning, it is necessary to draw on constructivist philosophical theories to establish a new teaching model that empowers students to be active, proactive and creative in the learning process, enabling them to truly become the initiators of information processing and active constructors of knowledge, rather than passive recipients of external stimuli and recipients of knowledge indoctrination. Teachers should be organizers and guides of classroom teaching, helpers and facilitators of student meaning construction rather than knowledge dispensers and dominators of the classroom ^[10,11].

Information technology, with computers at its core, mostly refers to multimedia computers, classroom networks, campus networks and the internet ^[12], which provide the foundation for achieving this goal. By fully utilizing and leveraging the advantages of modern information technology, deep integration of information technology with education and teaching can be achieved, promoting educational reforms and innovations at all levels. The traditional approach integrates information technology with the curriculum ^[13].

Since the mid-1990s, experiments integrating information technology with the curriculum have been increasing across Canada, yielding promising results ^[14]. The U.S. "2061 Plan" proposes the integration of information technology with various disciplines at a higher level.

Li Mushui ^[15] summarized the meaning of information technology, explored its application in higher vocational education, and highlighted its crucial role in constructing autonomous learning platforms for students and diversifying teaching methods for higher vocational teachers, which plays a crucial role in stimulating students' interest in learning.

To promote the informatization of vocational education, the government has launched the digital campus project, formulated construction norms for digital campuses in vocational colleges and further clarified the content and requirements of the informatization of vocational education construction.

Huang Baode ^[16] analyzed the objectives and tasks of informatization construction in higher vocational colleges and expounded on the measures that should be taken in future informatization construction in higher vocational education, such as the construction of information technology equipment, information platforms, information resource libraries and comprehensive educational management platforms.

Li Wenping ^[17] proposed the following measures for the informatization construction in higher vocational colleges: ensuring complete coverage of campus networks and WiFi, developing high-quality resource service platforms and online learning platforms, establishing digital libraries, developing digital textbooks and opening up online learning spaces. Innovative management mechanisms should be established, online education's management and service system should be improved, and the informatization construction of higher vocational education should be promoted.

Therefore, the informatization of higher vocational education refers to the comprehensive application of modern information technology in teaching, scientific research, management and life services, with the construction of a digital campus as the core, integrating and developing information resources and cultivating skilled information professionals.

Performance originated from management studies. It is used to assess the work status of the evaluated individuals by examining the relationship between their input costs and output results ^[18]. Performance evaluation employs specific evaluation methods and provides a comprehensive assessment of the overall operational effectiveness of an organization based on specific evaluation criteria and indicator systems ^[19]. Performance evaluation is carried out around the set performance goals, and the achieved effects and outputs within a certain period are evaluated. Performance indicators are the main content of performance evaluation work, including the quantity, quality and cost of project outputs, as economic benefits, social benefits, ecological benefits, sustainable impacts and satisfaction of service recipients. Performance indicator values are the specific manifestations of performance indicators, usually represented in numerical values, ratios, etc. ^[20].

With the rapid development of educational informatization, performance evaluation has gradually extended to educational informatization. Yin Yazhu and Li Yi ^[21] defined the concept of educational performance evaluation. They believed that educational performance evaluation should be dynamically conducted from goal setting, resource utilization, process arrangement and results demonstration throughout the entire education process.

Zhang Xiyan ^[22] proposed the connotation of educational informatization performance. They believed that practical activities of educational informatization generate the connotation of educational informatization performance and can be measured in specific ways.

Wang Xi ^[23] surveyed and analyzed of 74 higher vocational colleges in Jiangsu Province, concluding that the teaching methods in higher vocational education have been significantly improved through the integration of information technology. The integration of information technology has promoted active learning, improved teaching efficiency and enhanced students' practical abilities.

In summary, the integration of information technology in higher vocational education aims to empower students to become active learners and constructors of knowledge. It involves using of modern information technology, such as multimedia computers, networks and the internet, to enhance teaching and learning processes. Integrating information technology in higher vocational education promotes educational reforms, diversifies teaching methods, stimulates students' interest in learning and improves teaching efficiency. Performance evaluation in educational informatization focuses on evaluating the achievement of performance goals and the effects and outputs of educational activities. It encompasses various performance indicators, such as quantity, quality, cost, economic and social benefits and satisfaction of service recipients.

Therefore, the performance evaluation of informationization in higher vocational education refers to the comprehensive evaluation of information resource's input and output efficiency using quantifiable indicators. It aims to determine what needs to be done and how to do it in the informationization process in higher vocational education, ensuring a high degree of consistency between the results and goals of educational informationization.

As the focus of educational informationization shifts from infrastructure to instructional applications, the performance evaluation indicators of educational informationization have also changed. Huang Qiongzhen ^[24] proposed performance evaluation indicators that reflect the input cost and output of informationized teaching resources.

Hu Shuixing and Zhang Jianping ^[25] analyzed of the cost-effectiveness of educational informationization and its influencing factors, pointing out that cost-effectiveness is the main content of educational informationization evaluation.

3.   Research methodology

3.1.   Sample and Indicator sources

This study is based on data collected from modern higher vocational and technical education websites, student competition data platforms, teacher development data platforms, the Education Department of Henan Province and the official websites of various colleges. In the end, input and output data from 82 higher vocational colleges in Henan Province were collected. Among the 82 colleges, 28 are technical colleges, 5 are industrial colleges, 2 are science and technology colleges and 5 are information technology colleges. More than half of the colleges belong to science and engineering colleges.

Technical colleges belong to higher vocational colleges, primarily focusing on student employment and the cultivation of vocational application skills. Industrial colleges typically provide knowledge and skills related to engineering and manufacturing fields. Science and technology institutes emphasize scientific research and technological innovation, cultivating students' scientific thinking and problem-solving abilities and promoting the transformation and application of scientific and technological achievements. Information technology colleges specialize in providing knowledge related to computer science, software engineering, network technology, data science, artificial intelligence, information security and other information technology fields. STEM is a multidisciplinary approach that gives students a learning environment to use science, technology, engineering and mathematics in their everyday life ^[26]. STEM education aims to cultivate innovative talents by enhancing students' ability to apply interdisciplinary knowledge to solve practical problems ^[27].

The research primarily refers to the evaluation indicators of educational informationization in Henan Province, combined with those from Zhejiang, Anhui, Hebei and other provinces ^[4,28]. Redundant items were merged or removed. A three-level indicator system was established by collecting actual input and output data from higher vocational colleges in Henan Province. The first-level indicators are divided into input and output categories, and the second-level indicators include basic network infrastructure, innovative environment, basic service platforms, digital resource construction, innovative teaching, funding support and enhancement of information literacy as input indicators. The output indicators include research output, teaching output and social service. The third-level indicators include 14 input indicators, such as the maximum bandwidth of the campus network backbone, the number of unified identity authentication platforms, the number of information technology-related training sessions and the number of teaching computers. The output indicators include 10 indicators, such as the number of research papers, the number of authorized invention patents, the number of graduates, the number of competition awards and the teacher development index.

3.2.   Entropy method

After establishing the evaluation indicator system for the performance evaluation of informationization in higher education institutions, it is necessary to assign weights to the indicators. The weights of the indicators directly represent their importance and directly impact on the evaluation results of the informationization performance in colleges. Therefore, choosing a scientific weighting method is crucial.

Generally, there are two types of weighting methods: subjective and objective. The former includes methods such as the Delphi and the Analytic Hierarchy Process (AHP), which rely on expert judgments to determine the weights subjectively. This method is easy to implement but may introduce significant human bias and errors. Objective weighting methods are data-based and typically include the entropy method and factor analysis. Subjective judgments do not influence these methods, are more objective and help prevent errors caused by human factors ^[29].

The entropy method is an objective weighting method not affected by subjective factors. In this study, we use the entropy method to assign weights to the indicators by calculating their entropy values. Smaller entropy values indicate higher weights for the corresponding indicators, indicating greater importance and usefulness within the indicator system ^[30].

In education, the entropy method has been used in recent years to evaluate the development level of higher education. Ma Dan ^[31] studied the application of the entropy method in the evaluation of resource allocation efficiency in higher education. Wang Xiaozhen ^[32] used the AHP and entropy method to study the research performance of colleges in Fujian Province. Luo Weiming ^[33] investigated the application of the entropy method in the input-output evaluation model of higher education. Chen Yingxing ^[34] used the entropy method to objectively evaluate the quality of undergraduate education, using Jiangsu Province as an example.

When conducting DEA analysis, the sample size and the number of input and output variables should comply with the following constraints:

The study is based on a sample of 82 universities; therefore, a maximum of five input variables and 5 output variables with the highest weight values can be selected for DEA analysis.

3.3.   Data envelopment analysis (DEA)

The evaluation results of traditional performance evaluation methods largely depend on the science and effectiveness of the evaluation indicator system. However, in practical applications, due to human design factors, there is often a strong correlation between different indicators, making it difficult to achieve satisfactory reliability in the final evaluation results.

The basic idea of Data Envelopment Analysis (DEA) is to treat each object under study as a Decision Making Unit (DMU) and conduct a comprehensive analysis of input-output ratios to maximize outputs with limited inputs as the primary management objective.

In evaluating the performance of educational informatization, there are many input indicators, such as funding, human resources, technology, equipment, etc. and outputs, such as profit, income, quantity and quality, which can indicate the effectiveness of activities. The DEA method can simultaneously and automatically handle multiple inputs and outputs in the education sector, selecting the optimal input-output scheme ^[35]. Furthermore, DEA does not require the prior setting of indicator weights or the dimensionless processing of indicators, effectively reducing the influence of subjective factors on evaluation results, overcoming the correlation between different indicators and addressing the shortcomings of traditional evaluation methods. Therefore, it can more objectively reflect the information and characteristics of the decision-making units themselves.

Considering the availability of samples, this study chooses the CCR and BCC models in DEA for analysis. The CCR model assumes constant returns to scale and can obtain a comprehensive technical efficiency value. Decision-making units with a comprehensive technical efficiency of 1, i.e., located on the production frontier, are considered "DEA efficient, " indicating higher efficiency in the transformation of informatization performance. Decision-making units with a comprehensive technical efficiency value other than 1, indicating inputs and outputs not on the production frontier, are considered "non-DEA efficient." The BCC model assumes variable returns to scale and, combined with the CCR model, calculates scale efficiency and pure technical efficiency to determine whether the input-output of university informatization has reached an ideal state ^[36]. When the scale of university informatization production is small, the input-output ratio will increase rapidly with the increase in scale, indicating increasing returns to scale (irs). When informatization production reaches its peak, output is proportional to scale, achieving the optimal production scale and indicating constant returns to scale (-). When informatization production becomes too large, output decreases, indicating decreasing returns to scale (drs). At this point, as colleges increase their informatization inputs, the proportion of output increase will be smaller than the proportion of input increase. Finally, based on the projection distance of colleges to the production frontier, the input redundancy and output shortfall are calculated to quantify the optimal resource allocation plan for improving the evaluation of the informatization performance of each college.

Figure 1.  Entropy-DEA hybrid evaluation model.

DownLoad: Full-Size Img PowerPoint

4.   Data results and case analysis

4.1.   Entropy method results

The entropy method was chosen as the weighting method to determine the weights of indicators by calculating their entropy values, making the evaluation results more objective and reasonable.

From Table 1, it can be observed that the highest weight is assigned to the number of teaching computers (0.3439), followed by the number of online courses (0.2896). In contrast, the number of unified identity authentication platforms weights of 0.0000. A weight value 0 indicates that the indicator does not contribute to the informatization process. In the case of the number of unified identity authentication platforms, every vocational college has one. When all colleges have the same input without any differentiation, the importance of this indicator becomes negligible. Therefore, this input indicator is excluded from the DEA analysis.

Table 1.  Summary of input indicator weights.

Input Indicator	Entropy (e)	Information Utility Value (d)	Weight Coefficient (w)
Campus Network Backbone Maximum Bandwidth	0.8979	0.1021	20.23%
Unified Identity Authentication Platforms	1.0000	0.0000	0.00%
Information Technology-related Training	0.9422	0.0578	11.46%
Teaching Computers	0.8265	0.1735	34.39%
Online Courses	0.8539	0.1461	28.96%
Research and Teaching Equipment per Student	0.9750	0.0250	4.96%

---

 | Show Table

DownLoad: CSV

Table 2.  Summary of output indicator weights.

Output Indicator	Entropy (e)	Information Utility Value (d)	Weight Coefficient (w)
Research Papers	0.8631	0.1369	22.96%
Granted Patents	0.8008	0.1992	33.40%
Graduates	0.9690	0.0310	5.19%
Competition Awards	0.7861	0.2139	35.87%
Teacher Development Index	0.9846	0.0154	2.58%

---

 | Show Table

DownLoad: CSV

From the above table, it can be seen that there are differences in the weights of output indicators. The highest weight is assigned to the number of competition awards (0.3587), while the weight of the teacher development index is the lowest (0.0258).

4.2.   DEA analysis

4.2.1.   Comprehensive technical efficiency analysis (CCR)

The value of comprehensive technical efficiency is 1.00, indicating that the informationization performance evaluation of the vocational colleges has reached the optimal level. The input and output are well matched, making them DEA-efficient decision-making units. Suppose the comprehensive technical efficiency is less than 1.00. In that case, the vocational college needs to improve in both information management and scale management, resulting in a lower level of informationization performance evaluation, making it a non-DEA efficient decision-making unit.

Figure 2 shows that 46 vocational colleges are DEA-efficient decision-making units, achieving optimal efficiency. The input and output of informationization in these colleges are pretty coordinated, and the performance evaluation of educational informationization is relatively high. On the other hand, 36 vocational colleges are identified as non-DEA efficient decision-making units, indicating that the input and output of educational informationization in these colleges need to match, resulting in a lower level of informationization.

Figure 2.  DEA type (CCR).

Notes: "Efficient" indicates DEA efficient; "Non" indicates non-DEA efficient.

DownLoad: Full-Size Img PowerPoint

Figure 3 shows that the majority of the 82 vocational colleges in Henan Province have relatively high comprehensive technical efficiency, with an average efficiency exceeding 0.8. The informationization performance evaluation level of vocational colleges in Henan Province is relatively high, with an essential match between input and output. However, there is still room for improvement.

Figure 3.  Comprehensive Technical Efficiency (CCR) analysis chart.

DownLoad: Full-Size Img PowerPoint

Among them, six colleges have comprehensive technical efficiency values exceeding 0.90, indicating that their informationization input and output are relatively reasonable, and their informationization performance evaluation level is high.

Twenty vocational colleges have comprehensive technical efficiency values ranging from 0.6 to 0.9, while ten vocational colleges have comprehensive technical efficiency values below 0.60. It suggests that these colleges' informationization performance evaluation level is relatively low, and some inputs have not been fully utilized.

4.2.2.   Pure technical efficiency analysis (BCC)

Pure technical efficiency refers to the efficiency of decision-making units influenced by management and technology, focusing on the changes in technical efficiency resulting from the management mechanisms and models of the college's information technology department.

A pure technical efficiency value of 1.00 indicates that the college's information management mechanisms and models are relatively reasonable and well-developed, and the application efficiency of educational information resources and facilities is relatively good. A pure technical efficiency value below 1.00 indicates that the information management mechanisms of the colleges need improvement, resulting in a lower level of information performance evaluation.

From Figure 4, it can be seen that 69 colleges have a pure technical efficiency value of 1, indicating that the information management level of these colleges is relatively high, and their level of information development is in line with the planning. They can maintain their current development plans. In addition, 13 colleges have pure technical efficiency values below 1.00. This is because these colleges are mainly liberal arts colleges, such as regular colleges and art colleges. The current management system tends to focus more on reconstruction than application, and there is a tendency to focus more on input rather than output.

Figure 4.  BCC.

DownLoad: Full-Size Img PowerPoint

In Figure 5, among these 13 colleges, three colleges have pure technical efficiency values above 0.80, while the remaining colleges have relatively lower values. This also reflects that the three colleges have pure technical efficiency values close to the production frontier, indicating a high level of information management. They only need to pay slightly more attention to achieve higher pure technical efficiency. However, the other colleges have significant room for improvement and must adopt various strategies to enhance their management level. For example, they can establish annual information development plans, introduce the Chief Information Officer (CIO) mechanism, and promote the planning and development of informationization in colleges ^[37].

Figure 5.  Pure Technical Efficiency (BCC) analysis chart.

DownLoad: Full-Size Img PowerPoint

4.2.3.   Scale efficiency analysis (CCR/BBC)

Scale efficiency is used to measure whether the scale structure of a production sector is reasonable. Here, it is mainly used to measure the scale of college informationization investment. Suppose a university has a scale efficiency value of 1.00. In that case, it indicates that the scale of informationization in that university is reasonable, meaning that the current information input can achieve maximum output and the input-output structure is rational. For colleges with scale efficiency values below 1.00, adjusting the input scale and improving the structural configuration between informationization input and output is necessary.

Figure 6 shows that there are 46 colleges with a scale efficiency of 1.00 and 36 colleges with a scale efficiency below 1.00. The average scale efficiency is 0.9307. There are 14 colleges with a scale efficiency value above 0.9, 19 colleges with a scale efficiency value between 0.6 and 0.9, and 3 colleges with a scale efficiency value below 0.6.

Figure 6.  Scale Efficiency (CCR/BBC) analysis chart.

DownLoad: Full-Size Img PowerPoint

Figure 7.  Scale Returns to Scale (RTS) types.

Notes: irs represents increasing returns to scale, drs represents decreasing returns to scale, -represents constant returns to scale

DownLoad: Full-Size Img PowerPoint

For colleges with scale efficiency values below 1.00, the lower level of informationization performance is caused by either increasing or decreasing returns to scale. Among them, 18 colleges have increasing returns to scale, indicating that the growth rate of information output is higher than that of input. Currently, the scale of informationization construction is relatively small, and there is a need to increase investment to achieve the optimal match between informationization input and output and achieve the optimal scale efficiency of information resources.

Nine colleges have decreasing returns to scale, indicating that the growth rate of information output is lower than of investment. The scale of informationization construction needs to be bigger, and excessive resources and financial resources have been spent on informationization development. Although there has progressed in informationization, it has yet to yield more significant benefits. In the future, it is necessary to reduce informationization investment to achieve the best match between informationization input and output.

Fifty-five colleges have a scale efficiency value of 1.00, indicating that they are currently in a state of constant returns to scale. This means the current investment ratio is moderate and has achieved the optimal match between information input and output. It is only necessary to maintain the current input-output ratio.

4.2.4.   Projection analysis

Projection values directly reflect a decision-making unit's current input quantity and maximum output quantity. On the one hand, it can guide decision-making units in optimizing resource allocation. On the other hand, it can identify the development potential of decision-making units and provide a basis for decision-making.

Out of the 46 higher vocational colleges that are DEA efficient, their projection values are all zero, indicating that the current input and output of these colleges have reached their maximum. However, for the 36 non-DEA efficient higher vocational colleges, the input-output ratio has yet to reach the appropriate level, and further adjustments in input-output resources are needed.

In terms of input, each indicator has some degree of redundancy. These colleges' information technology investment structure is unreasonable, or the resources need to be more effectively utilized. Developing input plans in advance, reducing ineffective investment and improving the application level of information resources is necessary. These colleges have achieved optimal output in indicators such as the number of authorized invention patents and teacher development index, with slight variation among the colleges. However, there need to be more is a significant disparity in output levels regarding the number of graduates, published papers and competition awards, which requires focused attention.

4.3.   Case analysis

In order to provide a personalized decision-making basis for colleges, we select one non-DEA efficient higher vocational college, DMU8, for specific analysis.

Table 3.  DMU8.

	Entropy (e)	0.8274	Weight Coefficient (w)
Efficiency Analysis	0.8631	1
	0.8008	0.8274
Projection Analysis	Maximum Campus Network Bandwidth	1726	Input Redundancy Rate (%)	Maximum Campus Network Bandwidth	17.261
	Information Technology Training Sessions	0		Information Technology Training Sessions	17.261
	Teaching Computers	3029		Teaching Computers	48.941
	Online Courses	568		Online Courses	54.239
	Per Capita Value of Teaching and Research Equipment	2468.2		Per Capita Value of Teaching and Research Equipment	17.261
	Papers	0	output Insufficient Rate (%)	Papers	0
	Authorized Invention Patents	1		Authorized Invention Patents	26.843
	Graduates	854		Graduates	14.208
	Competition Awards	0		Competition Awards	0
	Teacher Development Index	0		Teacher Development Index	0

---

 | Show Table

DownLoad: CSV

The college relies on the Henan Province Defense Science and Technology Industry Vocational Education Group, a national demonstrative vocational education group, with solid educational strength. Regarding information technology investment, indicators such as per capita value of teaching and research equipment, number of teaching computers per hundred students and per capita teaching space area are higher than the standards set in the "Undergraduate Vocational College Setting Standards".

The input redundancy rate indicates that the college has an excessive investment in specific indicators, mainly in the number of teaching computers and online course quantity, with redundancy rates of 54.239% and 48.941%, respectively. The output deficiency rate mainly manifests in the number of authorized invention patents and the number of graduates, with deficiency rates of 26.843% and 14.208%, respectively. The number of papers, competition awards and teacher development index is 0.00%, indicating appropriate output without changing the output scale.

STEM education promotes collaborative behavior among students by offering opportunities to work on projects, participate in competitions and engage in research practices ^[38]. The college's leading discipline is electronic information engineering technology, forming a nationally high-level professional group in electromechanical integration technology. Therefore, regarding information technology investment, the emphasis is placed on hardware facilities such as computers, campus network bandwidth and teaching and research equipment. The emphasis is placed on the teacher development index and the number of competition awards, which align with the requirements of STEM education. The output deficiency mainly lies in the quantity and quality of authorized patents and graduates, reflecting the insufficient practical achievements and vocational talent cultivation in college. Therefore, the college should avoid simply treating information technology as a traditional information delivery support service. Instead, it should focus on integrating industry and education, enhance practical achievements in information technology within the context of characteristic majors and deepen the reform of talent cultivation models.

5.   Shortcomings and prospects

The research collected input data for 82 higher vocational colleges in Henan Province in 2019, and output data for 2020. Due to the lag in output, some colleges may not see immediate results after investing significant information resources. Therefore, relying solely on one year of results to judge the performance level of colleges may have some deviations. In future research, it is necessary to collect data over a more extended period time. Additionally, due to the constraints of the data envelopment analysis (DEA) model, only a limited number of input and output indicators could be selected. In future studies, the variety and quantity of input-output indicators can be increased after obtaining data from more colleges.

Data collection relies on manual collection, needing more automated collection and analysis of intelligent data. This results in a lengthy research process and low precision of research results and limits the utilization and exploration of data, making it difficult to achieve continuous and dynamic evaluation. By utilizing the Internet of Things, cloud computing and big data, a regional educational informationization monitoring platform can be constructed to continuously track and monitor the dynamic changes in the informationization of each college. This can automatically identify weak areas in the information application process for teachers and students, promote the development of evaluation towards real-time and intelligent directions, and provide favorable conditions for achieving a higher level of educational informationization evaluation.

The evaluation of higher education informationization performance is a crucial aspect of promoting educational informationization. As a national strategy, educational informationization requires scientific evaluation methods as support. STEM education emphasizes "learning by doing" ^[39]. Advancing the evaluation of educational informationization performance towards objectivity and practicality can quantitatively reflect the implementation of regional educational informationization planning. It also helps colleges optimize their management and talent cultivation, enhance students' knowledge, abilities and qualities and promote their academic achievements and personal development.

6.   Summary

This chapter combines the entropy method with the DEA method to objectively evaluate the performance level of educational informatization in higher vocational colleges in Henan Province from the perspectives of inputs and outputs. It provides specific data guidance on optimizing resource allocation and achieving the maximum input-output ratio for each college. It also offers suggestions and ideas for the education department to supervise the effectiveness of educational informatization and formulate related policies and services.

The evaluation of higher education informatization performance refers to using quantitative indicators to comprehensively evaluate the efficiency of information input and output to determine what needs to be done and how to do it in the process of higher education informatization. It aims to influence people's behavior and processes based on the selected action measures and maintain a high degree of consistency between the plan and goals of educational informatization.

The entropy method assigns objective values to indicators, where higher entropy values correspond to lower weights and contribution rates, indicating higher degrees of unavailability. In this study, when calculating the entropy values of input and output indicators, it was found that the weight value of the Unified Identity Authentication Platform was zero. Therefore, this input indicator was discarded when conducting DEA analysis. Ultimately, the study selected the five input indicators and five output indicators with the highest weight values.

The DEA method overcomes the correlation between different indicators and avoids the subjectivity of traditional evaluation methods to a large extent. It handles multiple inputs and outputs effectively and automatically considers the optimal input-output scheme of DMU (Decision-Making Units). It provides a wealth of information beneficial for management and decision-making, making it a suitable choice for evaluating the informatization performance of higher vocational colleges in Henan Province. In this study, the BCC/CCR model was selected, combined with the entropy method, to evaluate the informatization performance level.

Among the 46 higher vocational colleges, all were efficient decision-making units, according to DEA, with projection values of zero. This indicates that the input and output of these colleges have reached an optimal level, and their performance in educational informatization is relatively high. On the other hand, 36 higher vocational colleges were inefficient decision-making units, indicating that their input-output ratio needs further adjustment. The corresponding projection values reflect the necessary adjustments for each college, including the values and directions of the adjustments.

Among the 18 colleges, the scale returns were increasing, indicating that the information output growth rate exceed the input growth rate. These colleges have relatively small-scale informatization construction and need to increase investment, such as funds and resources, to achieve the optimal matching of input and output and maximize the efficiency of information resources.

For the nine colleges with decreasing scale returns, it indicates that the growth rate of information output is lower than that of investment. These colleges have invested excessive resources and funds in informatization construction, leading to a mismatch between the scale of informatization and the corresponding benefits. In the future, it is necessary to reduce informatization investment to achieve the optimal input and output matching.

The 55 colleges with a scale efficiency value of 1.00 are currently in the stage of constant returns to scale, indicating that the current input ratio is moderate and has achieved the optimal matching of information input and output. It is only necessary to maintain the current input-output ratio.

Use of AI tools declaration

The authors declare they have not used Artificial Intelligence (AI) tools in the creation of this article.

Conflict of interest

The authors declare there is no conflict of interest in this article.

Ethics declaration

The author declared that the ethics committee approval was waived for the study.

Appendix A

List of higher Vocational colleges in Henan Province

Appendix B

DEA = data envelopment analysis

DMU = decision making unit

CCR = the abbreviation of Charness & Cooper & Rhodes, one of DEA model

BCC = the abbreviation of Banker & Charness & Cooper, one of DEA model

irs = increasing returns to scale

drs = decreasing returns to scale

- = constant returns to scale

n = the number of DMU

m₁ = the number of input indicator

m₂ = the number of output indicator