CT medical image segmentation algorithm based on deep learning technology

1.   Introduction

Medical images can reflect the anatomical structure or functional tissues of the human body, mainly through imaging techniques such as Computed Tomography (CT), Magnetic Resonance Imaging (MRI), and X-ray. However, due to the constraints of the internal environment of organ tissues, the formed medical images are usually characterized by low contrast, blurred boundaries, and inaccurate image edge recognition. Medical image segmentation is one of the key steps to realize medical image visualization, and different image segmentation regions are often closely related to the related diseases and organs ^[1]. By segmenting medical images into other regions and sections, they are then provided to physicians for different diagnostic tasks such as lesion location determination, symptom identification, tissue and organ localization, description of anatomical structures, and treatment planning ^[2]. Therefore, medical image segmentation assists doctors in disease diagnosis, reduces the workload, improves the effectiveness and quality of treatment, and is widely used in disease diagnoses such as liver segmentation ^[3], cell segmentation ^[4], brain tumor segmentation ^[5], and COVID-19 ^[6].

Early medical image segmentation algorithms mainly used manually formulated rules for segmenting medical images. Zhang et al. ^[7] modified and optimized the objective function of the traditional fuzzy c-mean algorithm and achieved better performance. Ng et al. ^[8] combined the k-means algorithm with an improved watershed algorithm and applied it to medical image segmentation to greatly reduce the number of segmentation maps generated. Mohamed et al. ^[9] improved the fuzzy c-mean classification for the segmentation of brain images. Prabin et al. ^[10] combined the region-growing algorithm and contextual clustering technique and applied it to the lung CT image segmentation task.

This method of relying on manual methods for segmentation is costly and time-consuming, and the accuracy of segmentation markers cannot be guaranteed ^[11]. With the widespread use of deep learning techniques in the field of medical image segmentation, a series of research results have been achieved. Zhang et al. and Irfan et al. ^[12,13] proposed SNELM and HDNNs network structures for COVID-19 recognition. Long et al. ^[14] proposed the Fully Convolutional Network (FCN) to obtain a finer segmentation map by replacing the fully connected layer with a convolutional layer. Ronneberger et al. ^[15] proposed a classical medical image segmentation algorithm for U-Net networks based on FCN networks. Zhou et al. ^[16] proposed a U-Net++ network for medical image segmentation by adding new jump connections to transmit more image feature information. Oktay et al. ^[17] introduced the attention module into the jump connection in U-Net to improve the accuracy of pancreas segmentation. Peng et al. ^[18] proposed a local context-perception Net (LCP-Net) to obtain rich image feature information by parallel inflated convolution. Chen et al. ^[19] proposed a cross-scale residual network (CSR-Net) to achieve feature fusion of different layers by cross-scale residual connections. Wang et al. ^[20] proposed an adaptive, fully dense connected network (AFD-UNet); this network is based on Unet++, which adaptively and effectively uses shallow and deep features. Feng et al. ^[21] proposed U-Net based residual network (URNet) for image denoising to extract more detailed image features. Ge et al. ^[22] proposed a multi-input dilated U-Net (MD-Unet) for segmenting bladder cancer.

Various improved networks based on U-Net have improved the effectiveness of medical image segmentation. Still, medical images inherently suffer from problems such as category imbalance and noise factors that require the introduction of an attention mechanism to improve network segmentation performance. The attention mechanism is characterized by helping U-Net to better learn the interrelationships between multiple content modalities and thus better represent this information, overcoming its uninterruptable and thus difficult to design drawbacks.

Lan et al. ^[23] proposed mixed-attention based residual U-Net (MARU), which uses lightweight mixed attention blocks in the encoder to enhance image features effectively and suppress noise in the encoding stage. Li et al. ^[24] proposed an attention-based nested U-Net (ANU-Net), which can suppress background regions irrelevant to the segmentation task. Guo et al. ^[25] proposed Spatial Attention U-Net (SA-UNet); this model adds a spatial attention module in the bottleneck layer, which can help the network focus on important features, suppress unnecessary features, and improve the network's representation capability.

The above algorithms achieved high classification accuracy, but there are still areas for improvement, such as not fully exploiting the structural advantages of the coding network and the attention mechanism needs to learn the semantic features of the disease fully. In this paper, we improve both the network structure and attention mechanism, propose a dual coding network structure based on multi-scale modules, and design a fully convolutional neural network model combining spatial and channel attention and residual modules. In the encoding and decoding feature extraction stages, the spatial channel attention (SCA) module is added to highlight the key feature information of medical images and suppress the interference of noise factors in medical images.

The innovation points of this paper are as follows:

1) Based on spatial and channel attention mechanisms analysis, an integrated attention mechanism module SCA is proposed, which can give different attention weights from space and channel dimensions so that the model can focus more on the image segmentation task. The model can be integrated into the mainstream neural network segmentation task.

2) Considering that the increase in network depth will lead to network degradation and other problems, this paper introduces the residual network structure and integrates it into the U-net backbone network with the SCA module. While maintaining the network depth, it pays attention to the segmentation task of the target area, enhances the transmission of feature information and gradient information between different levels of networks, learns more detailed feature information of medical images, and improves the segmentation accuracy.

3) The multi-model and multi-perspective comparative analysis of three different medical datasets is conducted to objectively analyze and evaluate the algorithms proposed in this paper.

2.   Related works

This section is about medical image segmentation algorithms, including traditional medical image segmentation algorithms and medical image segmentation algorithms based on deep learning techniques.

2.1.   Traditional methods

Traditional medical image segmentation methods mainly segment images based on physical features such as the shape, angle, and edge structure of medical images.

The region-based image segmentation method mainly uses the similarity and difference of features in the image region for medical image segmentation. The threshold method is simple in calculation and fast in segmentation, but when the gray values of medical images are similar, the segmentation effect could be better. The region-growth method introduces spatial information based on the threshold method, but it must manually select the initial point. The clustering method is sensitive to parameters and easy to falls into local optimization. The random walk method iterates by randomly selecting the initial points, and its randomness leads to extremely unstable segmentation performance.

Bernal et al. ^[26] designed a model based on the appearance of polyps using the median depth of the valley accumulation window. Soltani-Nabipour et al. ^[27] introduced an improved algorithm that can automatically detect the threshold, update the threshold information, and accurately segment medical images in a shorter time. Chong et al. ^[28] used a clustering-based lung nodule algorithm to classify the lung nodule. Savic et al. ^[29] proposed a segmentation algorithm based on a fast marching method to segment the lung nodule image.

Bruntha et al. ^[30] used an image segmentation algorithm with edge-free active contours. By preprocessing, segmenting, and detecting lung nodules in the CT image, the segmentation accuracy reached 91.5%. Manickavasagam et al. ^[31] proposed a gradient-driven active contour algorithm, which uses normalization and gray co-occurrence matrix to extract nodule shape, and finally uses a support vector machine algorithm to detect and classify pulmonary nodules.

To sum up, the traditional segmentation method has a fast segmentation speed. Still, it is sensitive to parameters and cannot accurately segment lung regions that adhere to each other, so it needs rich prior information to obtain more accurate segmentation results. In addition, this method has great uncertainty for the segmentation results of different case images. Because of the complex and changeable structure of medical images, noise, partial volume effect, and other factors, image segmentation is generally combined with many traditional methods.

2.2.   Deep learning

The traditional segmentation methods can no longer meet the clinical requirements for the current medical image segmentation tasks with a large amount of data and higher accuracy and time requirements. But the deep learning algorithm has been greatly improved in the accuracy of segmentation and the degree of automation of the algorithm. The deep learning method is divided into three categories according to its network structure.

2.2.1.   CNN

Lecun et al. ^[32] designed a LeNet network structure for handwritten digit recognition. The convolution layer is responsible for extracting image feature information, and the full connection layer synthesizes and classifies the previously removed features. For medical image segmentation, CNN continuously extracts target features and realizes feature dimensionality reduction by combining the convolution layer and pooling layer, then integrates local features into global features through a full connection layer. Finally uses activation functions, such as softmax for classification and output, to complete the task of segmentation. Simantiris et al. ^[33] used a dilated Convolutional Neural Network method for brain MRI. Thyreau et al. ^[34] used a cortical parcellation method for MR brain images based on Convolutional Neural Networks. Aslan et al. and Akila Agnes et al. ^[35,36] proposed a new CNN segmentation framework for the semantic segmentation of lung disease.

The CNN algorithm has a simple network structure and fast operation, but the ability to extract image feature information is limited, which affects the image segmentation effect.

2.2.2.   FCN

Long et al. ^[14] proposed the full connection layer with convolution layer FCN based on CNN, which solved the problem that CNN can only extract local features. FCN uses convolution and pooling to achieve down-sampling in the coding process which can extract high-level semantic information, de-convolution to achieve up-sampling in the decoding process, and predict each class's score at the pixel level. Du et al. ^[37] introduced an novel framework for retinal vessel segmentation based on deep ensemble learning. Wu et al. and Xia et al. ^[38,39] redesigned the convolution layer. By expanding the convolution layer and adding multi-scale feature information, they segmented medical images and achieved good segmentation results. Liu et al. ^[40] used a residual network structure to segment the image of pulmonary nodules by extracting local features and context information. Roth et al. ^[41] used a two-stage method that can focus on the segmentation of organ and blood vessel images.

The segmentation method based on FCN adopts the idea of down sampling and up sampling path designed instead of full connection and combines deep semantic information with shallow appearance information by designing jump connection.

2.2.3.   U-Net

The FCN network structure is easy to lose image details, which affects segmentation results. Based on FCN, Ronneberger et al. ^[15] proposed a U-Net network including encoding and decoding structures. Each time the decoding is performed, the feature extraction part with the same number of channels is fused.

Lin et al. ^[42] proposed a novel deep medical image segmentation framework called dual swin transformer U-Net (DS-TransUNet). Milletari et al. ^[43] used a suitable network V-Net for 3D image segmentation. Hoorali et al. ^[44] proposed the IRUNet segmentation network, which makes full use of inception and residual blocks in skip connections and combines multi-scale features to extract better features for segmentation. Huang et al. ^[45] proposed the UNet 3+ network, which uses full-scale jump connections to obtain multi-level feature information. Alom et al. ^[46] proposed the R2UNet network, which combines residual connectivity and circular convolution to extract multidimensional image information.

Shen et al. ^[47] used the U-Net network as the basic framework, combined with the HarDNet module and attention module, for polyp image segmentation. Han et al. ^[48] designed the ConvUNeXt model to reduce the number of parameters while retaining the advantage of excellent segmentation. Gu et al. ^[49] proposed an attention-based integrated CNN (CA-Net) to achieve more accurate and interpretable medical image segmentation. Zhang ^[50] proposed the AResU-Net structure, which adds attention and residual network modules for brain tumor segmentation. Tong et al. ^[51] proposed an image segmentation network incorporating a triple attention mechanism to allow the segmentation network to focus more on the segmentation task.

Among these three kinds of deep learning methods, U-Net is improved and developed on FCN, while FCN is developed and improved on CNN. Each kind of image segmentation method can be improved by single network architecture and segmented together with other network architectures. No matter which deep learning segmentation method is still faced with many challenges and tests in clinical application, we need further research.

3.   The proposed architecture

For the problems of blurred edges, uneven background distribution, and much noise interference in medical image segmentation, we propose a medical image segmentation algorithm based on deep neural network technology, which adopts a U-Net-like backbone structure and includes two parts: encoding and decoding, as shown in Figure 1. In the encoding process, the training image is input into the model. The image is passed through an encoder path with residual and convolutional structures to extract image feature information. The number of feature map channels will be doubled; down-sampling uses 2 × 2 Max-pooling convolutional layers for feature integration, and half will reduce the feature map size in aspect size after each down-sampling module. In the decoding process, the number of feature map channels will be reduced by half as each multi-branch residual block passes through the perceptual field of different sizes and adaptively captures the image feature information of different sizes; up-sampling uses Upsampling2D to double the size of the feature map. There is a lot of redundant information in the low-level image feature information extracted in the encoding stage during the jump connection. If it is directly mapped to the corresponding layer in the decoding stage, it will affect the segmentation effect. With the SCA module, the image features relevant to the segmentation task can be learned intensively during the jump connection process to improve the segmentation accuracy. The predicted image is the same size as the input image in the final output.

The block-level representation of our proposed technique is shown in Figure 2.

3.1.   Attention mechanism

The input information is selectively distinguished, located, and analyzed through the attention mechanism. The attention mechanism will also be applied in image segmentation, target tracking, and behavior detection in the neural network learning process. In deep learning, we can obtain the image feature information of different spaces and latitudes by giving different weight information to the input image. How to build an attention mechanism model and integrate it into the mainstream neural network structure so that the simple neural network can achieve complex and high-precision image segmentation tasks is one of the problems that need to be solved.

3.1.1.   Spatial attention

After analyzing the input image, the neural network assigns more weights to the regions closely related to the segmentation task, making the target segmentation region more prominent. At the same time, the image region feature information, which has nothing to do with the segmentation task, is suppressed. The input feature information of the image is multiplied by the spatial attention weight map to get the final output result.

3.1.2.   Channel attention

The feature information on each channel is different, and the importance to the global feature information of the whole image is also different. By analyzing the segmented image, we can assign weight information to each channel, it indicating the importance of the channel information to the global feature description. The input feature information of the image is multiplied by the channel attention weight map to get the final output result.

3.1.3.   Spatial channel attention

In the paper, channel and spatial attention mechanisms are combined and given different weights. Finally, the output image feature information processed by the SCA module is as follows:

${\mathrm{x}}_{\mathrm{o}\mathrm{u}\mathrm{t}\mathrm{p}\mathrm{u}\mathrm{t}\mathrm{s}}$ represent the image feature information processed by the spatial attention mechanism. ${\mathrm{x}}_{\mathrm{o}\mathrm{u}\mathrm{t}\mathrm{p}\mathrm{u}\mathrm{t}\mathrm{c}}$ represent the image feature information through the channel attention mechanism.

Our proposed SCA module, as shown in Figure 3, with a general design idea similar to the architecture proposed by Fu et al. ^[52] integrates spatial and channel attention integration modules into an improved U-net network structure. The SCA module combines spatial and channel attention mechanisms to get comprehensive attention mechanism information. This module enhances the significant features of the up-sampling process by applying attention weights to high-dimensional and low-dimensional features.

The SCA module proposed in this paper can address the feature information of medical images, highlight more of the key feature information of medical images, and suppress the interference of noise factors in medical images.

3.2.   Residual convolution module

He ^[53] first proposed the residual network, which effectively solves the contradiction between neural network depth and recognition accuracy, as shown in Figure 4.

When a neural network reaches a certain depth, the output x of that layer is already optimal, and further deepening the network will result in degradation. In a convolutional neural network, it is difficult to ensure the weight of the following layer network. In the residual structure, the F(x) only needs to update a small part of the weight of F(x). It is more sensitive to output changes, and the parameters are adjusted more widely, which can speed up the learning speed and improve the performance of the model.

where ${\mathrm{W}}_{\mathrm{s}}$ is mainly a 1 × 1 convolution used to match the channel dimensions of the residual structure model input x and model output y. $\mathrm{F}\left(\mathrm{x}, \left\{{\mathrm{W}}_{\mathrm{i}}\right\}\right)$ is the residual mapping that the network needs to learn. When the residual structure has the same input and output dimensions, the definition is as follows:

The input information x is added to the feature calculation process, combined with the feature information of the upper layer to enrich the feature extraction of the network layer.

Through the residual structure design, the degradation problem in the process of deep structure network training can be well solved without adding additional parameters and calculations, improving model run speed and segmentation performance.

3.3.   Loss function

The Loss function is the index of a neural network to find the optimal weight parameters ^[54]. There are many kinds of loss functions, including single loss function and mixed loss function. We combine the Diceloss ^[55] and Focal loss ^[56] as the loss function in this paper. This function can combine the two functions' advantages and fully use semantic information to make the network better finds the optimal parameters for optimization learning.

${\mathcal{L}}_{\mathrm{D}\mathrm{i}\mathrm{c}\mathrm{e}\mathrm{l}\mathrm{o}\mathrm{s}\mathrm{s}}$ is the loss function of Dice loss, it is mainly used to measure the degree of loss of similarity between the segmented image predicted by the model and the real segmented image, and the value range is [0, 1]. The calculation formula of the function is shown in Eq (14). $\left|\mathrm{X}\cap \mathrm{Y}\right|$ represent the number of intersections between an actual segmented image and a model predicted image, $\left|\mathrm{X}\right|$ and $\left|\mathrm{Y}\right|$ represent the number of real segmented images and model predicted images respectively.

${\mathcal{L}}_{\mathrm{F}\mathrm{o}\mathrm{c}\mathrm{a}\mathrm{l}\mathrm{l}\mathrm{o}\mathrm{s}\mathrm{s}}$ is a loss function to deal with the unbalanced classification of samples. According to the difficulty of sample resolution, different weight coefficients α are added to the samples to reduce the adverse effects on training loss caused by the imbalance of sample classification.

$\mathrm{p}\in \left[\mathrm{0, 1}\right]$ is the probability of the model predicting the positive sample. ${\mathcal{L}}_{\mathrm{F}\mathrm{o}\mathrm{c}\mathrm{a}\mathrm{l}\mathrm{l}\mathrm{o}\mathrm{s}\mathrm{s}}$ is a modification of the cross entropy loss function. The total loss function proposed in this paper is ${\mathcal{L}}_{\mathrm{T}\mathrm{o}\mathrm{t}\mathrm{a}\mathrm{l}}$, The formula is shown in Eq (16).

4.   Experimental results

The main content of this section is to compare and verify the models based on the analysis of three medical image data sets. First of all, the three data sets are described and pre-processed. Then it describes the operating parameters of the model and related evaluation indicators. Finally, the segmentation results of different network structures on the three data sets are compared, analyzed, and displayed visually, including different model segmentation results, model ACC diagrams, Loss diagrams, and so on.

4.1.   Datasets

The DRIVE data set is mainly used for the study of vascular segmentation in retinal images. There are 40 retinal vascular images, including 33 fundus images of healthy people and seven fundus images of diabetic retinal lesions. The neural network needs more data to learn, so we carry on the data expansion operation to the picture of the training set. First of all, the input image is segmented into different parts. We generate a total of 100000 parts, of which 90000 are used as training sets and 10000 as validation sets. According to the requirements of the neural network model, the size of each part of the picture is set to 32 × 32.

The ISIC2018 data set is from the Kaggle ^[57]. The dataset has 2594 images, the original image size is 700 × 900, and the image size is resized to 256 × 256 as needed.

The COVID-19 CT contains a series of CT images of lung image segmentation and the corresponding label data, released by Kaggle in 2017. The dataset has 301 images; the original image size is 512 × 512, and the image size is resized to 256 × 256 as needed.

4.2.   Experimental environment and parameter metrics

4.2.1.   Implementation details

All models in this paper were run on the open-source TensorFlow 2.4.1 platform and used the NVIDIA GeForce Gtx1080 for experiments. We all used picture rotation and flipping operations for the samples participating in the training to increase the data. First, rotate each image 60 degrees apart once, followed by a horizontal pan of 30-pixel points, and finally, a random crop once. During the training, we set the picture of the DRIVE data set to 32 × 32 and the ISIC2018 data set and COVID-19 CT data set pictures to 256 × 256. The model learning rate is 0.0001, the dropout ratio is 0.5, and use the Adam optimization algorithm. We set the DRIVE data set and ISIC2018 data set the batch size is 8 and the COVID-19 CT data set batch size to 32. 100 iterations were carried out in each experiment.

To further avoid the problem of over-fitting in the model training process and reasonably evaluate the model's segmentation performance, we use 5-fold cross-validation to optimize the whole network.

4.2.2.   Metrics

We use several evaluation metrics, including Area under the ROC curve (AUC), Sensitivity (SENS), Precision (PRC), Jaccard similarity score (JS), Specificity (SPE), DICE, and IOU.

The JS coefficient measures the similarity between the two sets of predicted and valid segmented pixels.

The DICE coefficient measures the similarity between the predicted segmented labeled graph and the true segmented labeled graph.

The IOU can reflect the overlap rate between the predicted segmented label map and the real segmented label map.

The sensitivity calculates the proportion of correctly segmented target pixels to the target class pixels in the true segmented label map.

where TP is the pixels correctly segmented in the medical segmentation results, TN is the pixels incorrectly segmented in the medical segmentation results, FP is the background pixels incorrectly treated as medical pixels in the medical segmentation results, and FN is the medical pixels incorrectly treated as background pixels in the segmentation results.

4.3.   Analysis of experimental results

To verify the effectiveness and superiority of the proposed Residual-Attention-Unet network, the segmentation experiments are carried out on DRIVE, ISIC2018, and COVID-19 CT datasets and compared with Attention-Unet, Dense-Unet, U-Net, Unet++, and Residual-Unet. The model test results are shown in Tables 1–3.

As can be seen from Table 1, for the DRIVE data set, the AUC, PRC, JS, SENS, SPE, DICE, and IOU of the U-Net network are 0.9559, 0.8707, 0.6628, 0.7348, 0.9728, 0.7314, and 0.9452, respectively. Compared with U-Net, the Residual-Attention-Unet proposed in this paper increased by 2.03, 3.27, 4.42, 7.22, 1.77, 5.12 and 2.31% on AUC, PRC, JS, SENS, DICE and IOU, respectively. This paper proposes that compared with these networks, the indicators of AUC, PRC, JS, SENS, SPE, DICE, and IOU have been improved to a certain extent, up to 0.6, 1.95, 3.15, 5.71, 0.57, 4.67 and 2.0%, respectively.

As can be seen from Table 2, for the ISIC2018 data set, the AUC, PRC, JS, SENS, SPE, DICE, and IOU of the U-Net network are 0.8557, 0.8235, 0.6648, 0.7262, 0.9452, 0.8755, and 0.7786, respectively. Compared with U-Net, the Residual-Attention-Unet proposed in this paper increased by 4.38, 11.84, 10.32, 17.12, 4.13, 1.49 and 2.83% on AUC, PRC, JS, SENS, DICE and IOU, respectively. This paper proposes that compared with these networks, the indicators of AUC, PRC, JS, SENS, SPE, DICE, and IOU have been improved to a certain extent, up to 3.91, 7.3, 6.54, 16.31, 0.64, 1.21 and 3.26%, respectively.

As can be seen from Table 3, for the COVID-19 CT data set, the AUC, PRC, JS, SENS, SPE, DICE, and IOU of the U-Net network are 0.9678, 0.9237, 0.8478, 0.9669, 0.9487, 0.7982, and 0.8253, respectively. Compared with U-Net, the Residual-Attention-Unet proposed in this paper increased by 2.57, 6.78, 13.31, 2.35, 4.84, 14.8 and 12.84% on AUC, PRC, JS, SENS, DICE and IOU, respectively. This paper proposes that compared with these networks, the indicators of AUC, PRC, JS, SENS, SPE, DICE, and IOU have been improved to a certain extent, up to 0.76, 0.66, 1.64, 1.5, 0.29, 6.97 and 5.29%, respectively.

We also compare the proposed method in this paper with several recently proposed segmentation methods for analysis. Table 4 shows the performance of different segmentation methods on the DRIVE, ISIC2018 and COVID-19 CT datasets. The method proposed in this paper achieves good results on all three datasets. Compared with recent research methods, the method in this paper achieves optimal performance on some evaluation metrics on different datasets, and some metrics are close to the optimal segmentation performance.

To verify the effectiveness and superiority of the proposed Residual-Attention-Unet network, the segmentation experiments are carried out on DRIVE, ISIC2018 and COVID-19 CT data sets and compared with Unet, Unet++, Dense-Unet, Residual-Unet, Attention-Unet, and Residual-Attention-Unet. To ensure the fairness of the experimental results, this paper runs the six comparison networks in the same experimental environment, and their visual effects are shown in Figures 5–7.

Figure 5 shows the segmentation effect of various networks on the DRIVE data set. The first and second columns are the original picture and the segmentation result diagram, respectively, and the third and eighth columns are the resulting diagram of six network segmentation, respectively. As seen in Figure 5, several networks can segment the details of the main part of blood vessels. Still, the Residual-Attention-Unet network proposed in this paper can segment the most detailed information, and the segmentation effect is the best.

Figure 6 shows the segmentation effect of various networks on the ISIC2018 data set. The third and eighth columns are the resulting diagram of six network segmentations. As seen in Figure 6, several networks can segment the edge information of skin cancer images. Still, the Residual-Attention-Unet network proposed in this paper is better than others in dealing with the edge part. The segmentation boundary is more precise, the structure is relatively complete, and it achieves the best segmentation performance.

Figure 7 shows the segmentation effect of various networks on the COVID-19 data set. The third and eighth columns are the resulting diagram of six network segmentations. From Figure 7, the U-Net has learned too many redundant features. There are always obvious noise points; several other networks also have good segmentation performance on the segmentation boundary, but it pays too much attention to the image boundary, thus ignoring the internal features of the image. However, the Residual-Attention-Unet network proposed in this paper retains more image details, and the segmentation results are basically consistent with the standard segmented images.

We also compare the accuracy and loss of the different models on the three datasets, as shown in Figures 8–10.

The comparative analysis of the three figures shows that the model proposed in our paper converges fast on the three datasets. Finally, the model is almost converged and achieved a high accuracy rate.

From the loss diagrams of the above three data sets, we can find that after 100 rounds of experimental iterations, the network reaches the convergence state. The convergence rate of each model is also different on different data sets. Dense-Unet and Unet++ converge for DRIVE data sets after 40 rounds of experimental iterations. For ISIC2018 data sets, Dense-Unet and Res-Unet converge after 40 experimental iterations.

5.   Discussion

The U-Net network structure has achieved excellent performance in the field of medical image processing, but the U-Net network itself has problems such as incomplete feature extraction and lack of multi-scale feature information processing capability, therefore, researchers continue to propose improved segmentation networks based on the U-Net structure, such as Dense-Unet, Unet++, Attention-Unet, Residual-Unet, etc. The Dense-Unet segmentation network, by expanding the convolution with different expansion rates, expands the perceptual field without increasing the computational cost and effectively prevents the loss of spatial information, Dense-Unet segmentation network, by convolving with different expansion rates of dilation, the expanded perceptual field does not increase the computational cost and effectively prevents the loss of spatial information. attention-Unet segmentation network, by eliminating the influence of noise and invalid information in the image through the attention mechanism, reconstructs the contextual features in the image.

These network structures further improve the image segmentation accuracy by extracting multi-scale feature information and deepening the segmentation network depth, etc. Based on these excellent segmentation networks, we further optimize them in two aspects: for the medical image feature information extracted by the small number of layers of the U-Net network is not sufficient, the original convolutional layers are replaced by the residual network structure, and more levels of image feature information are extracted by deepening the network depth; the gradient disappearance problem caused by the deepening of the model is avoided while improving the network performance. On the other hand, in the encoding and decoding feature extraction, the network can be used to extract the image information. On the other hand, we add the SCA module in the encoding and decoding feature extraction stages. To make the network achieve a more accurate capture of the main part of medical images, a dual-channel attention mechanism is added to the expansion part of the model to highlight the key feature information of medical images and suppress the interference of noise factors in medical images, which can effectively improve the segmentation accuracy of the whole network.

Tables 1–3 show the evaluation results of the network models of Unet, Unet++, Dense-Unet, Residual-Unet, Attention-Unet and Residual-Attention-Unet on the DRIVE and ISIC2018 and COVID-19CT datasets, respectively. Figures 5–7 show the experimental results of segmentation of network models such as Unet, Unet++, Dense-Unet, Residual-Unet, Attention-Unet and Residual-Attention-Unet on DRIVE and ISIC2018 and COVID-19CT datasets, respectively. Through the analysis of the above graphical results, the proposed networks in this paper have high accuracy and acceptable segmentation results. Meanwhile, the accuracy and loss plots of different models on DRIVE, ISIC2018 and COVID-19CT datasets are compared and analyzed as in Figures 8–10, the proposed model achieves good results in terms of accuracy and convergence speed of loss.

Although we evaluated the performance of the network extensively on three different datasets, our network still needs some improvements. First, due to objective factors, we did not try to validate the effect of the connection method of different attention modules on the network structure; second, our network did not try to validate the 3D medical image segmentation dataset.

Meanwhile, the Transformer structure, which is popular in natural language processing tasks, has been widely used in medical image segmentation in recent years, and we will further investigate the fusion of Transformer and U-Net and other network structures.

6.   Conclusions

Based on the analysis of the U-Net segmentation network and related improved medical image segmentation networks, we propose an optimized medical segmentation network, which is mainly manifested in two aspects: for the U-Net network with few layers, the medical image feature information extracted is not sufficient, the original convolutional layers are replaced by the residual network structure, and more layers of image feature information are extracted by deepening the network. The network performance is improved while avoiding the problem of gradient disappearance caused by the deepening of the model. On the other hand, we add the SCA module in the encoding and decoding feature extraction stages. To make the network achieve a more accurate capture of the main part of the medical image, a dual-channel attention mechanism is added to the expansion part of the model to highlight the key feature information of the medical image and suppress the interference of noise factors in the medical image, which can effectively improve the segmentation accuracy of the whole network. The algorithm is compared and analyzed on several medical image datasets. The proposed network structure has improved the evaluation indexes of DICE, Precision, and IOU compared with other improved U-Net segmentation networks, and better segmentation results are obtained. We will further expand the datasets and extend the method to 3D medical image segmentation and accurate segmentation of other diseases in the future.

Acknowledgments

This research was funded by Excellent Young Talents in Anhui Universities Project (Granted No. gxyq2022026, No. gxyq2020016), Anhui Province Quality Engineering Project (Granted No. 2021jyxm0801, No.2021jxjy035), and Science Foundation of universities in Anhui Province (Granted No. KJ2020A0394, No. 2022AH050428).

Conflict of interest

The authors declare there is no conflict of interest.