Abdomen tissues segmentation from computed tomography images using deep learning and level set methods

1.   Introduction

Abdomen organ segmentation is very important for many clinical applications, such as organ quantification and surgical planning. However, manually marking organs from Computed Tomography (CT) images is subjective and sometimes time-consuming for the physician. Automatic and robust abdomen organ extraction from CT images is a difficult work because the low-intensity contrasts between abdomen organ and its neighbors.

Plenty of abdomen segmentation methods have been designed in recent years. For liver segmentation, Elaziz et al. ^[1] combines intensity analysis and region-growing for automatic segmentation of liver. Rafiei et al. ^[2] uses an 3D region growing model for liver segmentation, subject-specific conditions are integrated into the region growing model. Tran et al. ^[3] use a multiple layer u-net model to segment the liver region. Dilated convolution is also applied for the convolution node. Sofia et al. ^[4] opt for an atlas-based strategy for liver boundary detection. Their method is evaluated on a public dataset, receiving an average dice of 0.94. Song et al. ^[5]. propose a novel full-context convolution neural network to extract the liver region. Context information and temporal information are combined to construct the network. Zhang et al. ^[6] propose a novel 3D residual UNet using context information for liver segmentation. The long-range correlations can be obtained from the context information, and the features can be expanded in contextual information. Lei et al. ^[7] present a new designed deep learning model for liver segmentation. The deformable convolution and the ladder-atrous-spatial-pyramid pooling are combined to learn better context information. Fang et al. ^[8] propose a deep learning-based method to extract the liver region. The input and output features are designed into one network using multi-scale models to abstract the context information. Amina et al. ^[9] presents an active contour model to extract the liver region. Gaussian mixture model is applied pre-processing which can segment the initial liver contour. Gradient Vector Flow model is then used for fine segmentation. Noman et al. ^[10] propose a mask region-deep learning network to extract the liver region, the liver contour can be obtained by the output of the network. Wang et al. ^[11] propose a shape-based liver segmentation model. In their method, the mark label and a priori shape are combined to obtain the liver region. Qin et al. ^[12] combine superpixel information and boundary information in a convolutional neural network to segment the liver contour. Superpixels and entropy-based saliency map are combined to obtain the liver region. Yao et al. ^[13] propose an improved U-Net liver segmentation method. Label fusion and residual connection are combined to segment the liver region. Shao et al. ^[14] propose a context attention strategy for liver segmentation. The method uses spatial information and channel information to obtain the liver features. Li et al. ^[15] propose an attention UNet++ method for liver segmentation. They construct the Unet++ model with attention mechanism which can segment the liver region accurately. For kidney segmentation, Yan et al. ^[16] propose a novel effective hybrid model for kidney segmentation. Sparse point Cloud segmentation networks and contextual information are combined to obtain the kidney region. Thein et al. ^[17] use preprocessing method to segment the liver region. In their method, region growing is used to acquire the liver region. Yang et al. ^[18] propose a three-dimensional (3D) fully convolutional network.3D spatial contextual information and pyramid pooling module are combined to acquire the liver contour. Arafat et al. ^[19] propose a Cascaded Regression deep learning method to acquire the kidney region. In their method, mask-RCNN is used to obtain the kidney region. Chen et al. ^[20] present a 3D segmentation method for kidney segmentation. A deformable mesh model is constructed to obtain the kidney region. Yin et al. ^[21] propose a boundary based deep learning method for kidney segmentation. Experimental results have demonstrated the effectiveness of their method. Statkevych et al. ^[22] use U-net model for kidney segmentation and receive an acceptable result. Li et al. ^[23] propose an iterative convolution threshold method for kidney segmentation. The iterative convolution threshold method is constructed to minimize the energy function during kidney segmentation, which is significantly improving segmentation efficiency. Les et al. ^[24] combine the U-Net model and batch-based synthesis to obtain the kidney region. U-Net model is used for initial segmentation, batch-based synthesis is then applied for fine segmentation. Torres et al. ^[25] propose a fast phase-based approach for kidney segmentation. In their method, the initial kidney region is obtained by a multi-step feature detection method. And an active surface method is designed to fine the final kidney region. Weerasinghe et al. ^[26] propose a deep learning method for kidney segmentation. Label fusion and 3D B-Mode are combined to acquire the kidney region. Guo et al. ^[27] propose a residual and attention-based U-net model for kidney segmentation, the output of the network shows the segmentation result. Geethanjali et al. ^[28] use an attention U-Net model to obtain the kidney region, and receiving an acceptable results.

In this paper, we present a hybrid method which combines deep learning model and level set evolution for abdomen tissues segmentation. An attention-based deep CNN is constructed to obtain the initial abdomen tissues region. Level set evolution is then constructed to refine the segmentation. Figure 1 exhibits the pipeline of our framework.

Figure 1.  The workflow of the proposed method.

DownLoad: Full-Size Img PowerPoint

2.   Materials and method

2.1.   Image cropping

The choice of the input image is particularly important as it is related to computational reasons and overfitting problem of CNN. Liver occupies parts of the abdomenarea. Therefore, an automatic image cropping method is presented in this section. The cropped images capture enough information but require a significantly smaller amount of memory for storage. These cropped MR images are used as input for training the CNN. Figure 2 shows the process of image cropping. The algorithm of the proposed cropping method is summarized in Algorithm 1.

Figure 2.  The process of image cropping. (a) original image (b) Result of image cropping.

DownLoad: Full-Size Img PowerPoint

Image cropping

Input: Initial abdomen image I

Output: The cropped image L

1) Compute the threshold T according to the histograms of I, T is used to distinguish the background and the abdomen region;

2) Acquire the bounding box (redline in Figure 2(a)) of abdomen region according to T;

3) The area of the bounding box is the cropping image L.

2.2.   Convolutional neural networks

The overall architecture of proposed attention-based U-net is exhibited in Figure 3. The model consists of the up-sampling stage and the down-sampling stage. The numbers of feature maps in the down-sampling part of the network are 16, 32, 64,128,256,512. To obtain more context information from the deeper pathways, more features are added in the corresponding layers. The attention module is used to capture multi-scale contextual information. Binary cross entropy is used as the loss function and the stochastic gradient descent (SGD) as the learning optimizer. The initial learning rate is 0.0001 and decreased every 8 epochs. The size of convolutional kernel is 3 × 3.

Figure 3.  Structure of the attention-based N-net model.

DownLoad: Full-Size Img PowerPoint

2.3.   Level set evolution

Li et al. ^[29] design a Distance Regularized Level Set Evolution (DRLSE)model which consists of two terms: a distance regularization term and an energy constraint term. The proposed deep learning method obtains the initial abdomen tissues region. The boundary of the segmented region can be taken as the zero contour of the level set model. DRLSE model is then applied for fine segmentation. The final abdomen tissues segmentation can be obtained by using the DRLSE model.

Let $I$ denotes an image of domain $\Omega$, and $\varpi$ is defined as

where ${G_\sigma }$ is a Gaussian kernel with a standard deviation $\sigma$.

The DRLSE model consists of three energy terms:

where $\alpha, \lambda$ and $\beta$ are positive values.

The energy functional $\Lambda (\phi)$, ${\rm B}(\phi)$ and $\Theta (\phi)$ are defined by

where the delta function and the Heaviside function are denoted as $\ell$ and ${\rm T}$, respectively, the potential function p is defined as:$p(s) = {s^2}$. $\Theta (\phi)$ is used to maintain the signed distance property in the entire domain. The energy $\Lambda (\phi)$ is minimized when the zero-level contour achieves boundaries of the object. ${\rm B}(\phi)$ can accelerate the motion of the zero-level contour during the evolution process, which is important for improving the segmentation accuracy.

${\rm T}(\cdot)$ and $\ell (\cdot)$ are defined as:

The final energy function of DRLSE model is formulated as follows:

The gradient flow of the energy functional (7) is:

3.   Results

Our method has been validated on the LiTS dataset and FLARE21 dataset. LiTS provides liver dataset which is publicly available. 110 subsets are applied for training and 20 subsets are applied for testing. FLARE21 provides both liver and kidney data. 360 subsets are applied for training and 50 subsets are applied for testing. The parameters of the DRLSE model are set as follows: $\alpha {\text{ = 1, }}\lambda {\text{ = 1, }}\beta {\text{ = 1, }}\varepsilon {\text{ = 1}}{\text{.5}}$, The experiment was evaluated on a Windows 10 server with an i7-11800H CPU (2.3 GHz, and 16 memory) and Nvidia GPU GeForce 3060.

3.1.   Segmentation results

Figure 4 exhibits a liver segmentation result and a kidney segmentation result by the proposed method. Figure 4 (a), (c) are the segmentation results obtained by our method. Figure 4 (b), (d) are the corresponding manual labels. We can see from the picture that our proposed model are most similar to that from the ground truth.

Figure 4.  Visual inspection of the segmentation results for liver labels and kidney labels using the proposed method. (a), (c) are the proposed segmentation results; (b), (d) are the corresponding ground true.

DownLoad: Full-Size Img PowerPoint

The performance of CNN+DRLSE (Figure 5 red line) with CNN (Figure 5 green line) are compared on the same dataset. Several slices are selected randomly. From the picture we can see that CNN model produces poor segmentations at the depression area of the liver image (Figure 5 the first row) on three directions (axial, sagittal and coronal), it also produces many noise points on the kidney region (Figure 5 the second row). The CNN+DRLSE method has more accurate contouring results of both liver and kidney in comparison to CNN method.

Figure 5.  Contouring results in terms of different directions (axial, sagittal and coronal) between CNN and CNN+DRLSE. The first row: Liver contouring; The second row: kidney contouring. Green lines and red lines are the results of CNN and CNN+DRLSE, respectively.

DownLoad: Full-Size Img PowerPoint

3.2.   Evaluation metrics

To evaluate the segmentation techniques, we chose four metrics to evaluate the algorithm performance between different segmentation results and the ground true, namely Dice Coefficient (DC), Positive Predictive Value rate (PPV), Jacard Index (JI) and Mean Surface Distance (MSD). The segmentation results are better when the MSD achieves smaller value. And the larger the remaining three values are, the better the segmentation results will be.

3.3.   Implementation details

We compare the proposed method with other four level set methods: CV model ^[30], DRLSE ^[29] model, LINC ^[31] model and LBF ^[32] model. To validate the capability of DRLSE, we first applied it to the liver images. As shown in Figure 6, the median dice values reach 0.963 for DRLSE, followed by 0.877 for LINC, 0.832 for LBF and 0.782 for CV. We then applied DRLSE model to the kidney images. The median dice values reach 0.964 for DRLSE, followed by 0.883 for LINC, 0.848 for LBF and 0.822 for CV.

Figure 6.  The box plots of the DRLSE method with other level set methods in terms of dice coefficient.

DownLoad: Full-Size Img PowerPoint

Table 1 exhibits the quantitative results of our method. First, we validate our method on the LiTS dataset, the average DC are 95.1 and 95.7%, the JI values are 89.4 and 88.9%, the PPV values are 94.6 and 93.7 %, and the MSD values are 9.13 and 10.21 for the liver and the kidney, respectively. Then, we validate our method on the FLARE21 dataset, the average DC are 96.2 and 96.6%, the JI values are 90.5 and 91.8%, the PPV values are 95.5 and 96.1 %, and the MSD values are 8.33 and 8.02 for the liver and the kidney, respectively. Generally, the proposed method can segment the liver and the kidney more accurately, which is useful for clinical applications.

Table 1.  Performance of the proposed abdomen segmentation method.

Type	LiTS				FLARE21
	DC	JI	PPV	MSD	DC	JI	PPV	MSD
Liver	0.951	0.894	0.946	9.13	0.962	0.905	0.955	8.33
Kidney	0.957	0.889	0.937	10.21	0.966	0.918	0.961	8.02

---

 | Show Table

DownLoad: CSV

As shown in Table 2, different network are evaluated. We validate the experiment on liver images from both the LiTS dataset and the FLARE21 dataset. A lower MSD values indicates a better match between a segmentation result and the ground truth. The comparison of the values of these metrics shows that Attention U-net+DRLSE gave more robust performance, which matched with the ground truth better than segmentation results of U-net, U-net++, Attention U-net and Attention U-net++.

Table 2.  Comparison of different CNN segmentation methods.

methods	LiTS				FLARE21
	DC	MSD	JI	PPV	DC	MSD	JI	PPV
U-net	0.931	12.21	0.82	0.931	0.928	11.94	0.85	0.939
U-net++	0.937	11.49	0.831	0.935	0.941	10.88	0.852	0.941
Attention U-net++	0.942	10.54	0.895	0.948	0.949	10.22	0.902	0.95
Attention U-net	0.94	10.98	0.881	0.942	0.944	10.16	0.892	0.944
Attention U-net+DRLSE	0.957	9.11	0.921	0.952	0.964	8.35	0.9	0.953

---

 | Show Table

DownLoad: CSV

4.   Conclusions

In this paper, we proposed a combination of CNN and level set framework for abdomen segmentation. Image cropping is first used to resize the target region. CNN is then constructed to generate an initial label for the abdomen region. Finally, level set evolution is constructed to fine the segmentation. The experimental results show that compared with other CNN-based networks, the proposed method can improve the segmentation results more accurately. The level set model performs well on 2D images. In the future, we will extend the proposed method on 3D images. It will also be integrated into the ITK & VTK based software which can help the doctors to diagnose the related abdominal diseases.

Acknowledgments

This work was supported in part by National Natural Science Foundation of China (No.61971118), National Natural Science Foundation of Liaoning (No. 2020-MS-239). Foundation of Liaoning Education Department (No. LJKZ0210, No. JYT19040). Aviation Science Foundation (2019ZE054009). Outsourcing project of Northeastern University (220120099). Guodong Zhang and Wei Guo contributed equally to this work, and should be regarded as joint corresponding authors.

Conflict of interest

The authors declare there is no conflict of interest.