Edge computing-based intelligent monitoring system for manhole cover

1.   Introduction

As part of urban infrastructure, MCs safeguard drainage systems, power supply networks, communication transmission and safe gas supply ^[1]. Therefore, their management is crucial. However, the movement, loss, damage and flooding of MCs due to various reasons can present significant safety hazards to pedestrians and vehicles on passing roads ^[2]. Currently, many organizations rely on manual inspections and have limited use of technology to monitor and manage MCs. This approach is time-consuming and labor-intensive and has a long recovery time for detecting abnormalities. Additionally, the intelligent recognition system has low precision, and the overall process is inefficient and lacks real-time capabilities ^[3]. Once they are observed, MC abnormalities are usually addressed with temporary maintenance methods. For instance, an ice-cream cone-shaped structure is placed around the outer circle of the MC, along with notices and flashing lights at night, to alert pedestrians and car owners. However, this approach poses significant safety risks. Hence, it is crucial to leverage technology and data to empower the monitoring of MCs. An intelligent manhole cover monitoring system should possess the following advantages:

1) Intelligent perception. The manhole cover detection device, as an embedded device that integrates acquisition, control and communication, is capable of automatically sensing the state of the MC in real-time with high precision and accuracy. It can consistently detect whether the MC is moving, damaged, missing or flooded, and it also includes a localization function.

2) Efficient transmission. Low-power wireless WAN communication (LPWAN) technology facilitates fast and efficient transmission of MC status data to edge gateway devices and cloud servers.

3) Real-time response and alarm. The intelligent manhole cover monitoring system can promptly detect various conditions such as missing or displaced MCs. It utilizes localization facilities to trigger alarms in the form of sound, light, image, video or voice, thereby alerting pedestrians and vehicles on the road. Additionally, municipal administrators can remotely monitor the abnormal state of MCs and promptly notify maintenance personnel for repairs and restoration.

4) Lowering management costs. Information technology enables a significant reduction in human and material resources. By utilizing fewer human resources and minimizing system operating costs, it becomes possible to effectively manage a large number of MCs in the city ^[4].

5) Higher recognition accuracy. This study proposes the use of cloud-based technologies, artificial intelligence and big data to design an intelligent manhole cover monitoring system. By optimizing the system's cost-effectiveness, it aims to achieve higher precision and accuracy in recognizing MCs, thereby reducing misjudgments and labor costs. Ultimately, this system aims to ensure the safe passage of pedestrians and vehicles.

We propose an intelligent manhole cover monitoring system based on edge computing to meet the needs of intelligent manhole cover monitoring and management. The system utilizes an MC terminal monitoring device to collect the dynamic state of the MC. The collected data is then transmitted to the edge gateway of the urban roadside wisdom pole using LoRa communication technology. Through artificial intelligence processing and prediction, the system can detect abnormalities in the MC. If an abnormality is detected, the system provides a real-time response by displaying the abnormal state of the MC and sounding a voice alarm to alert pedestrians and vehicles. Conversely, if no abnormalities are detected, no action is taken. The edge gateway device uploads the abnormal data to the cloud KitLink platform (an IoT cloud platform developed by Kingsun Optoelectronics Co., Ltd.) via LoRa. The cloud platform then sends the recorded abnormal information to remote mobile municipal administrators and maintainers, enabling timely localization, repair and restoration of the abnormal MC.

In this paper, we propose a cloud-edge-end collaboration solution for MCs. The main contribution of this paper can be summarized as follows:

1) EC-MCIMS. It proposes an edge computing-based MCIMS system. The system includes the design of both hardware and software components, as well as the development of the edge gateway and deployment of the KitLink cloud platform. This system enables efficient real-time management of all MCs in the city with intelligent capabilities.

2) Strong real-time performance and higher accuracy. The system achieves edge computing by deploying tiny machine learning models in the edge gateway device on the wisdom pole. The model proposed in this paper demonstrates better performance compared to previous work. Furthermore, by drawing analogies with the cloud computing model and the cloud-edge cooperative computing mode, it is evident that the cloud-edge cooperative computing mode can effectively enhance the response speed, reduce response time and minimize the power consumption of the MC detecting device. This improvement in efficiency leads to an extended lifespan of the device without compromising prediction accuracy. This enables the system to promptly detect MC abnormalities with high accuracy and minimal response time, ensuring the safety of pedestrians and vehicles on the road.

3) Lower storage space usage. The machine learning model for edge computing utilizes a compact one-dimensional convolutional neural network (1D-CNN) ^[5]. This model is then quantized and compressed to int8 using the EON compiler of the edge computing EI platform ^[6]. Experimental results confirm that the compressed and quantized compact model more widely reduces the storage space required for the model flash and the peak memory usage during model runtime, without compromising the model's accuracy. This is particularly crucial for embedded edge gateway devices with limited resources.

The remainder of this paper is organized as follows. Section 2 reviews and analyzes the related work. Section 3 provides an overall description of the proposed EC-MCIMS system. Section 3.1 describes the MCIMS architecture and design of the hardware and software of the system. Section 3.2 particularly presents the principles of tiny machine learning models for edge computing. Sample results of data collected with the proposed system are presented and discussed in Section 4. Section 5 concludes with the future work plan.

2.   Related work

As a crucial component of urban municipal engineering, MCs play a vital role in managing urban underground water, power transmission, communication transmission security and natural gas supply security. The management of MCs ensures the safety of people's daily lives. Consequently, countries worldwide have conducted extensive research on the monitoring and management of MCs.

The management of a large number of MCs in the city has evolved from manual timed inspections to a more advanced approach that involves informatization, digitalization and intelligent monitoring. This transformation has made the process more efficient and accurate, transitioning from time-consuming and inaccurate methods to real-time, networked and convenient solutions. The informatization and digital monitoring of MCs primarily rely on the application of Internet of Things (IoT) technology, sensor technology, embedded technology and various communication technologies. Aly, Hesh et al. ^[7] conducted a study to assess the effects of automated and non-automated monitoring systems on the MC structure. They also proposed an IoT solution for designing a fully automated monitoring system for the MC. Fu ^[8] proposed a regionalized MC intelligent safety management system that is both detectable and maintainable. The system utilizes multiple sensors installed in the MC to monitor its condition in real-time. Through the use of an MCU, RF wireless data communication module and upper computer, the system can understand and control the MC. This enables real-time monitoring of urban MCs and automatic alarm triggering. Undoubtedly, this system has the potential to enhance the management ability of MCs and greatly improve the safety of people traveling. Nallamothu et al. ^[9], Rakesh et al. ^[10] and Saadnoor et al. ^[11] proposed the use of sensor technology to acquire the status of MCs. This approach aims to build an IoT system that enables monitoring and alarming of MCs. Ram et al. ^[12] and Muragesh et al. ^[13] primarily discussed the application of IoT technology in the design and implementation of the underground drainage monitoring system (UDMS) for MC. These IoT systems are designed to detect the status of MCs using different types of sensors. However, the system designs are relatively simple and do not take into account factors such as accuracy, precision, real-time monitoring and cost. Liu et al. ^[14] proposed a method for MC identification using RFID tags. The process involves handheld read-write devices to read and identify the tags, which are then uploaded to the cloud using mobile communication technology or low-power wireless wide area network technology for monitoring and locating the MC. However, this approach is considered to be more expensive and less efficient due to the requirement of additional handheld equipment and inspections ^[15]. Mankotia et al. ^[16] proposed using an Arduino integrated source microcontroller as the main control device for the MC monitoring and management system to simplify the system design, considering the cost of system monitoring. To ensure stable transmission and reliable operation of the system information, various types of wireless communication technologies are adopted. These technologies include, as used by Nataraja et al. ^[17], GSM mobile communication technology, Narrowband Internet of Things (NB-IoT) technology adopted by Guo et al. ^[18], Zhang et al. ^[19], and LoRa wireless communication technology adopted by Sun ^[20], Zhang et al. ^[21], Liu et al. ^[22] and Li et al. ^[23]. These communication technologies have reached a higher level of maturity and are capable of effectively monitoring large areas remotely. However, mobile communication technology faces challenges such as expensive tariffs and high power consumption. NB-IoT technology requires the pre-installation of base stations in the region, making it more difficult to implement in underdeveloped areas. Additionally, it requires a special SIM card and has limitations in terms of coverage, cost and real-time performance. On the other hand, LoRa communication technology offers better solutions to the aforementioned problems. However, its system requires the establishment of additional gateways, which necessitates careful consideration of the convenience and reliability of the gateway setup location. With the advancement of artificial intelligence technology, various methods such as image processing, LiDAR and localization techniques have been employed to enhance monitoring accuracy. For instance, Yu et al. ^[24] and Wei et al. ^[25] proposed a collaborative approach that combines mobile LiDAR point cloud and ultra-high resolution ground images to enable intelligent monitoring of MCs. Vishnani et al. ^[26] proposed a method for detecting MCs based on Google location data. Their approach can be enhanced by avoiding MC paths during emergencies and adverse conditions. Andrijašević et al. ^[27], Krishnan et al. ^[28], Zhang et al. ^[29] and Thakur et al. ^[30] proposed image processing and deep learning methods for intelligent identification, monitoring and prediction of MCs, all of which yielded better results. However, these intelligent methods solely focus on improving the accuracy of MC monitoring and control intelligence, resulting in the need for a large amount of image processing equipment and pre-trained data. Consequently, this leads to a complex system, high costs and challenges in terminal power supply.

Addressing the aforementioned issues, this paper presents a proposal for a manhole cover intelligent monitoring system based on edge computing, which integrates the wisdom pole in the smart city. The system utilizes cloud-edge-end synergy to efficiently manage urban MCs. The edge gateway employs embedded tiny machine learning algorithms to preprocess and analyze the data transmitted by terminal sensor nodes. Real-time processing of localized data and a small amount of anomalous MC data are uploaded to the cloud ^[31]. This enables real-time abnormal alarm maintenance of the MC, effectively ensuring road traffic safety.

3.   Proposed EC-MCIMS architecture

3.1.   MCIMS Framework

The EC-MCIMS is a typical cloud-side-end collaboration architecture, which consists of three main layers, as shown in Figure 1. The actual structure of the system is schematically shown in Figure 2.

The terminal node is an MC detection device that is enclosed in a disc-shaped shell. This shell can be easily installed under the MC, and its shape and working location can be observed in the field test described in Subsection 4.1, as shown in Figure 22(a). The shell is made of non-electromagnetically shielded plastic material, which is waterproof and dustproof, ensuring safe and stable detection of the MC status. The node consists of several components, including a sensor group, microcontroller, power module and LoRa communication module ^[32]. The sensor group primarily detects movement, being missing, waterlogging and localization information of the MC using a three-axis acceleration sensor, waterlogging sensor and positioning sensor. The microcontroller utilizes the low-power and high-performance STM32 series embedded controller to process real-time data collected by the sensor group ^[33]. It then sends this data to the edge gateway of the wisdom pole through the LoRa communication module. The circuit schematic design of the main modules of the manhole cover detection device is shown in Figures 3 and 4.

After receiving the uploaded MC status and localization data from the terminal node, the edge gateway performs inference and prediction using the tiny machine learning model deployed inside the ESP32 ^[34]. This allows it to determine the current status of the MC and take immediate localized actions. The actions include displaying a reminder on the wisdom pole and broadcasting a voice message five times to alert pedestrians and vehicles passing by. This real-time approach ensures the safety of the public, especially at night. Additionally, the ESP32 module calculates and stores the effective data at the edge for backup purposes. It also provides a basis for future big data analysis. The circuit design of one of the edge gateway ESP32 is shown in Figure 5.

The edge gateway will upload the predicted conclusion situation to the cloud KitLink platform, and the abnormal status and localization information of the manhole cover will be presented on the cloud platform, as shown in Figure 6. This allows management personnel to understand any abnormal status of the MC and track it for decision-making. Additionally, it will push information about MCs that require timely maintenance to the management and maintenance personnel of both mobile and PC terminals through HTTP and MQTT protocols. This facilitates prompt maintenance and repair of MCs, thereby enhancing the social service capability of municipal departments and ensuring the safety of people's daily travels.

According to the working requirements and working mode of the system, the overall workflow diagram of EC-MCIMS is designed as shown in Figure 7. Figure 7(a)–(c) presents the workflow diagrams for MC status detection, edge gateway edge computing and KitLink cloud, respectively. EC-MCIMS collects and transmits information locally through the terminal nodes of the MC status detection device. It then uses edge computing at the edge gateway to process the data and provide real-time local alarms for pedestrians and vehicles. The system also uploads the MC abnormality status information to the KitLink cloud for storage and visualization, enabling data mining and regulatory decision-making in the future. Additionally, the information is sent to users of mobile and PC terminals, allowing relevant personnel to be notified promptly for maintenance and repair based on the abnormal status of MCs. This helps to minimize potential risks caused by cover abnormalities.

3.2.   Edge computing-based monitoring system

Smart cities face the challenge of monitoring a vast number of devices, each requiring a real-time understanding of their status and efficient response to data requests. These devices, including access, exit and update devices, come from different manufacturers and require a robust system platform and network. The extensive data in smart cities creates significant strain on network bandwidth, resulting in issues such as delay and coordination ^[35,36]. Moreover, network interruptions pose a substantial risk to the stability of the system's operation. Additionally, traditional cloud computing methods are insufficient to handle the diverse scenarios presented by the vast amount of data in smart cities. MC monitoring is a crucial aspect of smart city development and is closely linked to the safety of citizens and transportation infrastructure. Consequently, it requires high levels of real-time monitoring. This paper focuses on the implementation of smart city construction using wisdom poles. These poles are used to monitor MCs in real time and address security issues through an edge computing model. The model enables efficient collaboration and linkage within a local area, especially when the equipment and facilities are disconnected from the network. This allows for quick response to business needs at the scene, reducing data interaction with the cloud and avoiding excessive bandwidth and time costs. The ultimate goal is to provide timely alerts to pedestrians and vehicles, ensuring road traffic safety and operational efficiency ^[37].

The edge computing gateway device performs four main tasks: 1) collecting and managing edge data, 2) preprocessing the collected edge data, which includes data cleaning, data coding and decoding and semantic integration and is achieved using a tiny machine learning model, 3) monitoring the collection process of the edge data and managing message queues for edge data transmission and 4) monitoring and storing the preprocessing process of the edge data.

3.2.1.   Edge computing workflow

Edge computing involves deploying computing, storage and other network resources closer to the terminal device ^[35]. This approach facilitates a seamless integration of mobile applications, content and networks, minimizing intermediate transmission processes. As a result, edge computing enables faster and more real-time data processing, reducing network latency and pressure while enhancing real-time response and reliability.

When the system detects that the MC is opened or moved, the edge gateway will automatically trigger a series of actions. These actions include controlling nearby display equipment and voice equipment, as well as releasing warning messages through public broadcasting and information release screens. The purpose of these actions is to serve as a reminder or deterrent. The cloud platform also receives the data and can send real-time warning messages to managers through phone calls, SMS, email or WeChat, allowing them to intervene in the management process. The operation process of the specific edge computing gateway device is shown in Figure 8.

The storage component of the edge computing layer device stores configuration information and management policy information. This allows the device to continue executing its saved operation and management policy even when disconnected from the network of the cloud computing layer device. This helps avoid interruptions in the operation of the entire system due to network abnormalities. Additionally, when the edge device is overloaded or lacks computing power, it can proactively send commands to request collaboration with the cloud. This collaboration enables continuous and stable monitoring of well covers as shown in Figure 9.

3.2.2.   Tiny machine learning model principle

The tiny machine learning models in this paper are trained using the EI edge platform, which is developed based on TensorFlow Lite ^[38]. The EI platform offers powerful automation, low code and custom extensions, enabling developers to create and optimize embedded machine learning applications using various types of sensor, audio or visual data. Additionally, it provides a Python SDK that allows developers to optimize and convert their models into C++ libraries for any edge device ^[39]. The development process for tiny machine learning models on this platform consists of five main steps, as illustrated in Figure 10: preprocessing, learning, evaluation, conversion and prediction.

1) Preprocessing. It involves labeling the large amount of raw data collected, dividing the dataset into training and test sets and performing dimensionality reduction. These steps are necessary to prepare for the extraction of data features.

2) Learning. We utilize the EI platform for deep learning to construct a convolutional neural network (CNN) model. The optimization hyper-parameters are iteratively tuned using a validation set to generate an improved machine learning model ^[40].

3) Evaluation. The performance of the trained optimal learning model on the test set is evaluated and examined using the test dataset. This evaluation helps guide further optimization of the model to reach its optimal state.

4) Conversion. It is necessary to optimize the machine learning model for hardware support and efficient operation. Although the high-performance ESP32 device is used in the gateway device at the edge end, it is still insufficient compared to running the model on a PC. This leads to a decrease in edge computation efficiency and real-time performance. To address this, model quantization and compression are typically performed on the trained model of the EI platform. One method is to reduce the precision of data expression, such as converting float32 to int8, which better reduces the hardware requirements for the model. This allows the model to be easily exported and deployed on the embedded edge computing gateway ESP32.

5) Prediction. The edge computing gateway device enables predictive processing of actual MC detection data using a deep learning model deployed within it. This allows for localized actions to improve real-time efficiency in event processing.

The construction of the neural network model structure on the edge computing EI platform is illustrated in Figure 11. It primarily comprises two phases: the feature extraction phase conducted by the convolutional layer, and the classification and identification phase performed by the fully connected layer. During the feature extraction phase, the input data consists of a one-dimensional time series of sensor readings. The sensor set includes three-dimensional acceleration data from triaxial acceleration sensors, converted current data from flooding sensors and latitude and longitude data from the localization chip. The sensor set has a sampling frequency of 30 Hz, resulting in 30 sampling points per second and a data matrix of size 6 × 30. By applying two one-dimensional convolution and pooling operations using the 1D-CNN method ^[41,42], multiple one-dimensional feature maps can be generated. During the classification and recognition phase, the input is first spread into a one-dimensional array of 42 features using the flattened layer. Then, the classification and recognition outputs are obtained sequentially through the implicit layer with 32 and 64 neurons, respectively. These outputs are used to determine the six MC states in the output layer. The parameter settings of the 1D-CNN model are shown in Table 1.

As the data collected from the sensor is in the form of time series, this study utilizes a one-dimensional convolutional neural network model to extract features. The model selects different convolutional kernels to extract various features from the input signal. The output of the $l$ convolutional layer can be represented by Eq (1).

The variables used in the context are as follows: $j$ represents the number of convolution kernels, $k$ denotes the convolution kernel, $N$ indicates the number of channels of input ${x^{l + 1}}$, $w$ represents the weight of the $\left({l - 1} \right)$ layer, $b$ represents the $\left({l - 1} \right)$ layer bias, $f\left(\bullet \right)$ represents the activation function (in this case, the ReLU function is used with the expression as in Eq (2)), and $*$ denotes the convolution operator.

After the multiple feature maps are extracted by the convolutional layer, the pooling layer is used to achieve dimensionality reduction. This helps in reducing the complexity of the model computation, improving computational speed and reducing overfitting. Common pooling operations include maximum pooling and mean pooling. In this paper, the maximum pooling operation (expressed as Eq (3)) is selected, in which $H$ represents the convolution kernel width.

After completing the feature extraction stage, the information from multiple feature maps is spread out into a one-dimensional array of data. This array is then passed to the fully connected layer for classification and recognition output. The classification output is represented by Eq (4).

In this paper, we consider a six-classification problem and use the $soft\max$ activation function, for the output layer. The activation function is expressed as Eq (5), where $w$ represents the weight and $b$ represents the bias.

The error between the output of the forward propagation output layer and the expected result is continuously updated through backpropagation using the stochastic gradient descent method (SGD). This method updates the weights and bias of each neuron until the error value is less than the set threshold or the maximum number of iterations is reached. The training of the model is considered complete when the error value meets the defined criteria. The error is calculated based on Eq (6), in which $y_i^l$ represents the actual output of the model and $y_a^ *$ represents the target output of the vector $a$. The process of updating the weights and bias through backpropagation has been extensively studied by various scholars ^[43,44,45], and the details will not be reiterated here.

In addition, the monitoring data for MCs is inherently unbalanced. This paper addresses a six-classification problem, taking into account the data imbalance. It is not sufficient to only consider the recognition precision of the data. Evaluation of the machine learning model also requires assessing the recall and ${F_{1{\text{ - }}score}}$ metrics, such as the confusion matrix for model evaluation. The recognition precision $P{\text{recision}}$, recall $R{\text{ecall}}$ and the average of precision and recall ${F_{1{\text{ - }}score}}$ metrics are defined in Eqs (7)–(9) respectively.

where $TP$, $FP$ and $FN$ in Eqs (7) and (8) denote true positives, false positives and false negatives, respectively.

4.   Results and discussion

4.1.   Data processing and analysis

By selecting 240 raw data samples from over 3 years of MC status monitoring data, the sensor data was located in two places: Changping Town, Dongguan City, Guangdong Province, Kingsun Optoelectronics Company Limited - South Gate, and Hengli Town, Dongguan City, Guangdong Province, Kingsun Optoelectronics Company Limited. Each place had 120 data points. If the flood sensor detects water-logging, it triggers circuit conduction and generates current. The sample was then randomly divided into training data and test data, with a ratio of 0.79:0.21. The specific details of the raw sample situation are shown in Figure 12. The distribution of the original data on the edge computing EI platform is shown in Figure 13.

To evaluate the feature extraction ability of the 1D-CNN model on the original time-domain signal, we employed principal component analysis (PCA) to downscale and visualize the features learned by the model. The results are presented in Figure 14, which shows (a) and (b) as the principal components of the three-axis acceleration sensor data and the features of the flooding sensor and localization sensor data after dimensionality reduction, respectively. The data belonging to the same state of the MC can be effectively aggregated, while the data representing different states exhibit better separability. This confirms that the 1D-CNN model can adaptively extract valuable features from the original time-domain signals, enabling accurate monitoring of the MC state.

The 1D-CNN network model is configured with an epoch of 300, a batch size of 32, a learning rate of 0.0005, a validation set size share of 20%, a random loss probability of neurons in the dropout layer of 0.5 and a model quantization compression of int8 type. This compression can be used to compare the model training results with the original uncompressed quantization of float32 type. Please refer to Table 2 in Subsection 4.2 for more details. All the model parameters are set and then trained. The model's performance is verified after each epoch, and the hyperparameters are continuously optimized using the verification set. The best training results are shown in Figure 15, where it can be observed that the model achieves 100% accuracy on the verification set after the last training with only 0.10 error. The confusion matrix shows that the value of ${F_{1{\text{ - }}score}}$ is 1, indicating that the model has a strong training effect and good generalization ability. Furthermore, the training model demonstrates high recognition accuracy on all training sets, as depicted in Figure 16. By calculating the recognition situation in the figure, we can determine that the accuracy rate is approximately 96.71% = (round(240*0.79)*(1-20%)-5)/(round(240*0.79)*(1-20%)).

The trained model was tested on the test set and achieved an accuracy rate of 94%, demonstrating good generalization, as shown in Figure 17. The confusion matrix analysis of the test results revealed that the model accurately detected single states of the MC, such as normal, tilted, lost and flooded, with 100% accuracy. There was only a small probability of confusion between the tilted and flooded states, as well as between the lost and flooded states. Importantly, the model never identified an abnormal state of the manhole cover as normal in all the data, indicating its high recognition accuracy. This finding, as shown in Figure 18, has minimal impact on promptly understanding the state of the MC for timely rescue and repair. Moreover, the occurrence of these two confusion situations is relatively small. Therefore, it is feasible to apply the trained model to the ESP32 edge gateway of the MC.

To ensure the validity and accuracy of the model, we implemented the same model using the edge computing platform and also programmed it in Anaconda3. The training and testing were conducted using the same raw data and divisions. The results are shown in Figure 19, which compares the verification accuracy with the training accuracy curve. Additionally, the verification error versus training error curve is depicted in Figure 20. Finally, Figure 21 presents the model testing results from the confusion matrix, which are consistent with those obtained from the model constructed by the edge computing platform.

To further verify the effectiveness and accuracy of the model, we downloaded and loaded the tiny machine learning model trained on the platform into the edge computing gateway ESP32 on the smart-pole. This setup was then subjected to an on-site field test (located in Changping Town, Dongguan City, Guangdong Province, China Kingsun Optoelectronics Co., Ltd), as shown in Figure 22. The results of the test revealed that when the MC is tilted, the display on the wisdom pole near the local MC immediately and correctly shows the state of the MC being turned on. Additionally, the system broadcasts the state of the MC in real time, indicating that the system has good accuracy and real-time performance.

4.2.   Data transfer time, data storage analysis and energy power consumption

In order to run the machine learning model on the edge computing ESP32 microprocessor, it is necessary to convert the model. For the conversion of the tiny machine learning model, we utilize the EON compiler provided by the EI platform. This adds another layer of optimization to the machine learning model by quantizing and compressing it. The float32 types are converted into int8, which reduces the storage space required by the model and decreases the inference time. However, this reduction in size and time does not better affect the prediction recognition accuracy. The actual results are presented in Table 2. The test performance of the EI platform on the device is illustrated in Figure 23. These figures demonstrate that compared to the model before quantization and compression, there are improvements in terms of inference time, peak device memory usage and flash memory usage.

The proposed model for the edge computing-based manhole cover intelligent monitoring system was evaluated through 320 real test experiments. The performance index of the model is presented in Table 3. When compared with other research results, the model demonstrates a smaller average response time and average latency time while achieving mean average precision (mAP). Thus, the system model has certain advantages.

This study also compares the power consumption, response time and accuracy of manhole cover intelligent monitoring in cloud computing mode and cloud-edge cooperative computing mode. Due to the manhole cover detection device being packaged into a disc-type shell installed under the manhole cover, it is challenging to directly measure its power consumption during operation. Therefore, we employed a conversion method to evaluate the power consumption of the two computing modes. The test was conducted on the industrial road in front of the South Gate of Kingsun Optoelectronics Co., Ltd. in Changping Town, Dongguan City, Guangdong Province, China. The cloud-based KitLink server was set up in Kingsun Optoelectronics Co., Ltd. in Hengli Town, Dongguan City, Guangdong Province, China, which is approximately 13 kilometers away from the test site. As depicted in Figure 24, it was observed that during the operational hours of the two computing modes, the cloud computing mode exhibited a shorter lifespan for the manhole cover detection device, exhausting its power within 3 months and rendering it incapable of functioning properly. Conversely, when the same test was conducted using the collaborative computing mode on the cloud side, it was found that the device could operate normally for over a year. This indicates that the cloud-edge collaborative computing mode offers more than four times the working hours compared to the cloud computing mode, thereby reducing the frequency of replacing the power supply for monitoring a large number of urban manhole covers. Furthermore, a rough calculation can be made to support these findings. The manhole cover detection device utilizes a 3000 mAh battery with a power supply voltage of 3.3 V. The transmission power of LoRa transmission is dependent on the transmission distance and typically ranges from 1 mW to 10 mW. By applying Ohm's law, we can easily calculate the working time for the cloud computing mode as tc = (3000*3.3)/4.5 = 2200 hours, which is a difference of at least 3 months. On the other hand, the working time for the cloud edge collaboration computing mode is calculated as tce = (3000*3.3)/1.1 = 9000 hours, equating to over 1 year.

In analyzing the response time and accuracy test results as shown in the last two rows of Table 3, it is evident that the cloud computing mode yields slightly higher recognition accuracy compared to the cloud-edge collaborative computing mode. However, the real-time response speed of the former is significantly lower than that of the latter. This poses a potential risk as it may result in delayed alerts for abnormal states of MCs, failing to promptly notify the authorities. Thus, it is recommended to opt for the cloud-edge cooperative computing model for intelligent monitoring of MCs. In addition, a detailed analysis of the response time for the two computing modes revealed that the response process includes three key times: 1) transmission time for MC state data, 2) processing time for state data and 3) transmission time for decision execution. Figure 25 illustrates that the cloud-side collaborative computing model offers significant advantages in terms of data collection transmission time and decision execution transmission compared to the cloud computing model. However, it has a slight disadvantage due to limited resources in model inference recognition. Overall, it performs better. In scenarios that require real-time monitoring of manhole covers, the cloud-edge collaborative computing model is recommended as it enables computationally intensive tasks to be offloaded to the edge and the cloud for collaborative processing ^[48]. This approach greatly enhances the system's overall performance and serves as a suitable choice for the computing model.

5.   Conclusions and outlooks

In this paper, we propose a cloud-edge-end collaborative system called MCIMS (Manhole Cover Intelligent Monitoring System) that utilizes edge computing. The system involves embedding a monitoring device in each MC in the city, which can effectively detect various statuses such as normal, tilted, lost, flooded, inundated and moving. The device transmits the status data, along with positioning data, to the edge gateway ESP32 microcontroller module located on the nearby wisdom pole using low-power wireless LoRa communication. The edge gateway is equipped with a pre-loaded lightweight machine learning model that has been trained, validated and tested by the edge computing EI platform. The model is compressed and quantized to ensure high reliability and stability. The MC status is derived by analyzing the real-time timing signals received by the edge gateway through this tiny machine learning model. If the MC status is abnormal, the ESP32 immediately performs real-time localization processing to determine the type of abnormality. It then sends a control signal to the display and voice module, which displays the abnormal status of the MC and broadcasts a voice alarm on the big screen. This serves as a reminder for pedestrians and vehicles passing by. Simultaneously, the information is sent to the KitLink cloud platform, allowing remote municipal management personnel to understand the locations and types of MC anomalies. Mobile municipal managers and maintenance personnel can access this information through the HTTP and MQTT protocols, respectively, enabling them to quickly and efficiently address the maintenance and repair needs of the abnormal MC. This ensures the safety of public travel, enhances the service and safety of municipal departments and improves their efficiency. If the MC is normal, no further action is taken. The system model has been applied to various towns and streets in Dongguan City, Guangdong Province. The results indicate that the average response time of the system is less than 5 seconds, with an average delay time of less than 10 milliseconds. Furthermore, by quantizing and compressing the model into a lightweight version, it better reduces the storage space and operating memory required. Despite this, the system maintains an average recognition accuracy of 94%. Overall, the system demonstrates commendable performance. The MC is an integral part of urban life and its control and management can be challenging due to its large volume, decentralization and complexity. While this paper focuses on using edge computing for abnormal manhole cover detection, alarm and rapid repair, there is also a need for in-depth exploration and research in the intelligent management of manhole covers from the following four aspects.

1) Accuracy of MC abnormal state detection and identification. The accurate detection of the MC state requires the use of reliable and precise sensing equipment. This can include high-precision sensors, RFID, video equipment or a combination of multiple modalities. Various types of artificial intelligence algorithms, such as deep learning algorithms, can then be employed for high-precision identification. Different types of research on different modal data can be conducted using these algorithms. For instance, real-time time sequence signals from sensors can be analyzed using 1D-CNN, while video image signals are suitable for 2D-CNN. Additionally, RNN, LSTM or YOLO series algorithms can be utilized for multi-target object detection in images as per the requirements. These methods are reasonable approaches to enhance the accuracy of data recognition.

2) The response speed of MC abnormalities. It is crucial to ensure public safety during travel. When MCs are tilted, moved, lost or flooded, it poses significant safety risks and requires immediate handling and repair. To solve these risks, it is essential to enhance real-time response to MC abnormalities. This can be achieved through research in edge computing, edge intelligence and other areas to improve localized and timely response. Additionally, improvements in the remote response mode in the cloud can help reduce transmission distance and mitigate delays caused by congested transmission channels, especially when dealing with large amounts of data. By addressing these challenges, the lag in processing abnormal MCs can be better reduced. The introduction of edge computing can better enhance the response speed of abnormal MC processing, improving real-time processing efficiency. Additionally, further research is necessary to enhance the efficiency of edge computing, addressing issues such as cache allocation and task job scheduling capabilities.

3) The power consumption of the MC status detection device. To ensure waterproofing, anti-electricity and dustproof capabilities, the detection device is typically sealed and installed as a whole unit within the MC. As a result, it primarily relies on dry batteries for power. To minimize power consumption, low-power, high-performance microcontrollers and low-power wireless transceiver protocols such as NB-IoT, LoRa and ZigBee technologies can be utilized. Additionally, strategies such as putting the device into sleep mode during periods of inactivity, optimizing data transmission and allocation of transmission time slots, can further reduce power consumption and extend the device's survival time.

4) Cost of the system. Cost considerations are crucial for both the development and implementation of the system, particularly with the accuracy of abnormal state detection of the MC. By applying the principle of logical equivalence between hardware and software, a balanced solution can be achieved through technological empowerment, integration and software optimization.

Use of AI tools declaration

The authors declare they have not used artificial intelligence (AI) tools in the creation of this article.

Acknowledgments

This work was supported by 2020 Dongguan Science and Technology Specialist Program (Grant No. 20201800500792), 2021 Guangdong Basic and Applied Fundamental Youth Fund Project (Grant No. 2021A1515110834), 2019 Guangdong Youth Characteristic Innovation Project (Grant No. 2019KQNCX227), 2021 Research Program of Guangdong Institute of Science and Technology (Grant No. GKY-2021KYZDK-5), Dongguan Sci-tech Commissioner Program (Grant No. 20231800500572), Dongguan Science and Technology of Social Development Program (Grant No. 20221800905802), Special Projects in Key Fields of Colleges and Universities in Guangdong Province (Grant No. 2021ZDZX1017), Key Scientific Research Projects of Guangdong University of Science and Technology (Grant No. GKY-2021KYZDK-1 & XJ2022003401), Young Innovative Talents Projects in Colleges and Universities in Guangdong Province (Grant No. 2022KQNCX120), Guangdong Provincial Key Construction Discipline Scientific Research Capacity Improvement Project (Grant No. 2022ZDJS148) and Teaching and Learning Mutual Benefit Project Team under Science and Education Innovation of Guangdong University of Science and Technology (Grant No. GKJXXZ2023030).

Conflict of interest

The authors declare there is no conflict of interest.