Deep learning and radiomics analysis for prediction of placenta invasion based on T2WI

Abbreviations: MRI: Magnetic resonance imaging; LASSO: Least absolute shrinkage and selection operator; PI: Placenta invasion; PP: Placenta previa

1.   Introduction

Placental invasion is defined as the abnormal invasion of trophoblast cells into the myometrium at different depths of infiltration ^[1]. It occurs mostly in patients with placenta previa or prior cesarean section, with common complications including catastrophic perinatal hemorrhage and bladder, bowel, and urethral injuries ^[2]. Early diagnosis of placental invasion before delivery is critical for appropriate treatment planning ^[3]. Ultrasound is the mainstay in the imaging of placental invasion ^[4]. However, due to the influence of the shape of pregnant women, gas in the intestines and bones, ultrasonic penetration power would decrease and reflected waves increase, which results in the unclear observation of lesions, especially the lesions of the posterior wall of the uterus and placenta. While MRI is less affected by intestinal gas and bones, has high tissue resolution and can be imaged at any angle in multiple directions, so it is especially recommended for cases of posterior placenta and cases whose ultrasound results are equivocal and/or clinical suspicion is high ^[5,6].

In the clinical practice of detecting placenta invasion, a recent study showed that the performance of experienced radiologists was significantly better than that of junior radiologists ^[7]. In order to reduce the dependence on doctors' clinical experience and improve the diagnosis of placental invasion, some scoring systems for diagnosing placental invasion have been proposed in recent years, but they have not been extensively tested ^[8]. Obvious placental heterogeneity and irregular thick dark intraplacental band in the placenta on T2WI are recognized signs of placental infiltration in MR images ^[9]. However, the visual representation of placental heterogeneity and the identification of abnormal dark intraplacental bands on T2WI may be difficult and highly subjective. Therefore, quantitative image analysis has become an attractive research field that overcomes the subjectivity of visual interpretation and improves diagnostic accuracy ^[10].

With the rapid development of medical imaging technology, a comprehensive analysis method of medical images called radiomics has received extensive attention in recent years ^[11]. Radiomics usually refers to extracting high-throughput quantitative features from medical images and realizing auxiliary diagnosis of diseases through machine learning. Researchers can use complex machine learning tools to develop a variety of radiomics models, thereby potentially improving the accuracy of disease diagnosis, prognosis and prediction because radiomics transforms medical images into obtainable data ^[12,13]. At the same time, deep learning has achieved great success in image analysis in recent years as an important means of realizing artificial intelligence (AI), and has also greatly promoted the development of medical image processing technology ^[14,15]. Different from the low-level image features extracted by radiomics methods, deep learning can mine various high-level semantic information of images. Therefore, we hope to combine deep learning with traditional radiomics to establish a more effective placenta invasion detection model. At present, the conventional idea of combining radiomics and deep learning is to extract radiomics features and deep features of medical images, and then build better auxiliary diagnostic models through feature-level fusion. This idea is easy to implement, but radiomics features and the deep features are at different semantic levels, which restricts the fusion effect. Therefore, the idea of using the radiomics features and the deep features to independently train two models, and then the decision-making fusion of the model output has also attracted scholars' attention.

Inspired by the above research, we combined the radiomics and deep features, used the LASSO algorithm for feature screening, formed an effective feature combination and combined with clinical factors, and established a placenta invasion detection model by constructing a nomogram. This method not only takes full advantage of the ability to represent different levels of placental heterogeneity of radiomics features, deep features and clinical factors, but also takes into account the respective advantages of feature-level fusion and decision-level fusion, thereby contributing to the development of auxiliary diagnosis of placenta invasion.

2.   Materials

2.1.   Participants

This retrospective study was approved by the Ethics Committee of the Affiliated Hospital of Medical School of Ningbo University, and identity information of all patients had been de-identified to protect patient privacy. One hundred forty-one pregnant women were initially selected. In all, twenty-nine patients were excluded for various reasons including multiple pregnancies (n = 2), MRI performed before 24th gestational weeks (n = 18), presence of significant fetal/maternal anomalies (n = 7), and severe motion or other type of artifact on either imaging sequence (n = 2). The final cohort included 112 patients. Based on these cases, we conducted a multi-center study. We took 78 cases from Ningbo Women & Children's Hospital as the training cohort, and 34 cases from the Affiliated Hospital of Medical School of Ningbo University as the independent validation cohort, as shown in Table 1.

2.2.   MR imaging parameters

All patients underwent placental MRI using one of the two 1.5T MRI systems. The scanning sequence and parameters were as follows: (1) Sagittal T1WI single-shot fast spin echo (SSh-T1WI-TFE) sequence, TE 6.8ms, TR 13.9, FOV 375 × 375 mm, layer thickness 8.5 mm, layer spacing 1.5 mm, matrix 375 × 375; (2) T2WI fast self-selected echo (FSE T2WI) sequence of axial, sagittal and coronal planes, TE 1200ms, TR 80ms, FOV 375 × 375, layer thickness 4.5 mm, layer spacing 2 mm, matrix 256 × 256.

Documents showed that the main signs of placenta invasion in MR images included: uneven placental signal, abnormally prominent placenta or (and) uterine limitations and low signal banding in the placenta, etc. Therefore, we finally chose the supine sagittal image of the conventional T2WI sequence (side-lying imaging is prone to curling artifacts due to the bulge of the abdomen) as the reference imaging sequence for this study. Typical T2WI sagittal images are shown in Figure 1.

In Figure 1, (a) The patient was 35 weeks gestational and had a placenta increta. The arrow in the figure showed that the placenta and the anterior inferior wall of the uterus were unclear, and signs of implantation were seen locally. (b) The patient was 35 weeks gestation and had a placenta percreta. The arrow in the figure showed that the placenta penetrated the lower wall of the uterus and involved the bladder.

Some researchers reported that the use of contrast media to observe the structure of the placenta would help the detection of placental invasion ^[16,17,18]. These researchers believed that for the placental tissue rich in blood vessels, the application of contrast media could show the placenta more clearly. However, since the contrast media might pass through the placental barrier, the side effects caused could not be ignored. Zhu ^[19] believed that although there are no reports on the adverse effects of gadolinium contrast media on human fetuses, the half-life of the contrast media in the fetus was still unclear, so the MR images used in this study were all obtained by non-enhanced scanning.

3.   Methods

3.1.   Region of interest segmentation

The features we extracted mainly include two types: radiomics features and deep features. The radiomics features mainly included the geometric shape features of the target area, the pixel grayscale distribution features and the texture features. In order to better characterize the feature information contained in the placental tissue, we segmented the placental region to extract the radiomics features. Considering that most patients with placenta invasion have placenta previa, that is to say, the relative position of the placenta in the uterus is also an important factor in predicting placenta invasion. The deep features extracted by deep neural networks can indicate the relationship between the targets. Therefore, we segmented the entire uterine region from the MR image and use it as the input of the deep neural network to extract deep features.

Segmentation of the uterine region: Background information occupies a large proportion of MR images and it is not helpful for extracting radiomic features and deep features. Radiologists usually focus on the uterus area and automatically filter the background information. Therefore, we cut out the black background area as much as possible while preserving the complete uterine area. Finally, the size of each slice after cropping was determined to be 256 × 256. Each sequence of the original data contained 24 slices. The size of the MRI was 256 × 256 × 24 after the above processing. We normalized the images after the above processing by z-score and used them as the input of the neural network to extract deep features.

Segmentation of the placenta area: In the uterus area, the placenta was further segmented by an MRI radiologist with 15 years of work experience and a radiologist with 10 years of work experience using the open source software ITK-SNAP version 3.8.0. Both radiologists were blind to the pathological results. The segmentation results were also evaluated by three independent reviewers to make sure they agree. We normalized the segmented placental region images by z-score and used them to extract radiomics features.

3.2.   Radiomic feature extraction

In this paper, radiomics features were extracted from the region of placenta. The extracted radiomics features were divided into three categories: (I) geometry, (II) intensity, (III) texture. Geometric features described the three-dimensional shape of the placenta. The intensity features described the first-order statistical distribution of voxel intensity in the placenta. Texture features described the second- and higher-order spatial distribution of patterns or intensity. A total of 93 radiomics features were extracted. The specific features can be viewed in Appendix A. All the features were extracted through the pyradiomics package version 3.0 ^[20].

3.3.   Deep feature extraction

Radiomics features are clearly designed or hand-made. Although the number of features can reach tens of thousands, these features are shallow and low-level image features, which may not be able to fully characterize the heterogeneity of the placenta, and therefore may limit the potential of survival prediction models. In this case, it is necessary to extract deeper and higher-order features.

Given a target task (for example, in our case, diagnosing invasive placentation by MRI), the training data for this target task is limited. Transfer learning aimed to use large-scale data from other sources of tasks and artificially provided labels to learn expressive and universal feature representations to help achieve the target task. A commonly used method ^[21] is to pretrain a deep neural network for feature extraction on a large data set in the source task (such as ImageNet) by inserting human annotation labels in it, and then finetune the pre-trained network in the target task. When applying this strategy, we should pay attention to the following points. First of all, there is a big difference between the image data in the source task and the MR image of placenta invasion. For example, ImageNet images usually belong to general domain categories, such as cats, dogs, chairs, etc., while the images in our task are MR images, which casts doubt that the portability of MRI from other image sources to placenta invasion. In order to solve this problem, we conducted neural network pretraining on 23 medical data sets (including brain MR images and lung CT images, etc.). Compared with ImageNet, the difference between the pretrained image data and the image data of the target task is smaller.

In this research, we chose 3D-ResNet50 as the pretrained CNN model ^[22]. The hyperparameters of 3D-ResNet50 are as follows: weight attenuation was 0.001, momentum was 0.9 and the initial learning rate was 0.001. Deep feature extraction includes 3 steps: pre-training of 3D-ResNet50, fine-tuning and feature extraction.

1) Pre-training: The 3D-ResNet50 model was pre-trained on 23 magnetic data sets. The model had the ability to recognize the basic contours and details in the magnetic resonance image after pre-training which could accelerate the gradient descent in the subsequent fine-tuning process.

2) Fine-tuning: The placenta training data was used to fine-tune the model. The specific method is as follows, set the output dimension of the softmax layer to 2, that is to predict the invasive placentation (the prediction result is 2 classes: normal or invasion), then calculate the loss between prediction result and the labels and fine-tune the entire network according to the back propagation error. The fine-tuned network has the ability to identify invasive placentation.

3) Feature extraction: First, linear transformation was used to normalize the image gray values to the range of [0,255]. Then, the entire placenta area was used as the input of 3D-ResNet50. Finally, deep features that can only be calculated by forward propagation are extracted from the fully connected layer before the softmax layer. In total, 128 deep features can be extracted for each patient. This procedure was accomplished by using the deep learning toolkit Pytorch. The process of transfer learning to extract deep features is shown in Figure 2. The details of 3D-ResNet50 can be found in Appendix B.

3.4.   Selection of clinical factors

One hundred twelve pregnant women who meet inclusion criteria were enrolled in the current study. Among them, seventy-eight are training cohort and thirty-four are independent validation cohort. In the training cohort, fifty-one had histologically confirmed placenta invasion (PI) (16 accreta, 32 increta, and 3 percreta) and 27 were with normal placenta. We collected clinical data of all patients from the hospital's HIS system, including age, weight, history of cesarean section or curettage (0 means no history of cesarean section or curettage, 1 means the patient had a history of cesarean section or curettage) and placenta previa (0 means no placenta previa, 1 means placenta previa), the relationship between each clinical data and placenta invasion was analyzed using paired sample t test. IBM SPSS 25.0 for Windows was used to perform the t-test. The results are shown in Table 2.

Table 2 shows that the mean age for the PI group ($\text{35.02}\text{}\text{±}\text{}\text{4.94}$ years, range from 22 to 44) was significantly higher than the normal group ($\text{33.26}\text{}\text{±}\text{}\text{5.02}$ years, range from 25 to 44) (p < 0.001, two-sample t test). The mean weight for the PI group was $\text{62.65±5.27}$ kg (range from 50 to 75), while for the normal group it was $\text{61.15}\text{}\text{±}\text{}\text{5.11}$ kg (range from 50 to 70). No significant difference was observed in weight (p = 0.137, two-sample t test) between the two groups. The number of women who had a history of cesarean section or curettage in the PI group was significantly more than that in the normal group. And the number of women who had PP in the PI group was significantly more than that in the normal group. Therefore, we combined three clinical factors (excluding weight) with combined signatures to construct the nomogram.

3.5.   Model building and comparison

The flowchart of the study is shown in Figure 3, radiomics and deep features were combined to build the NDRC model.

In Figure 3, a variety of machine learning methods were performed to classify placental invasion (invasion or normal) based on the extracted radiomics features, including decision tree, Random Forest, Naive Bayes and Support Vector Machine in order to compare the prediction accuracy of the proposed NDRC model. Random Forest was performed to classify placental invasion based on the deep features. Their implementation details can be viewed in Appendix C. In addition to comparing the prediction accuracy with the NDRC model, we explored whether it was possible to predict placental invasion accurately by using radiomic features or deep features alone, and which features have higher prediction accuracy. Finally, the prediction accuracy of traditional machine learning methods and proposed NDRC was compared by the four indicators of accuracy, sensitivity, specificity and area under the ROC.

For each patient, we combined 93 radiomics features and 128 deep features, used the LASSO ^[23] algorithm to filter the merged 221 features, then used the result of the linear combination of the 12 features and their corresponding weights as the combined signatures. The paired sample t-test was used to analyze the relationship between various clinical factors and placental invasion, and the factors that were significantly related to placental invasion were retained (p-value < 0.05). In order to make the model more robust and explanatory, clinical factors remained after the t-test analysis and combined signatures were used to construct the nomogram. The method of construction is multi-factor logistic regression.

4.   Results

4.1.   Combination and selection of features

A total of 221 features were extracted from the MRI images of the training cohort, and these features were filtered by using the LASSO model. As shown in Figure 4, (a) represents the coefficient distribution of each feature, and a coefficient distribution map is generated for log(λ) (penalty intensity). It can be seen that as log(λ) increases, the coefficients corresponding to less and less important features become 0; (b) is to use 10-fold cross-validation to adjust the parameters in the LASSO model to obtain the minimum standard. The mean square error was plotted versus log(λ). The dotted vertical lines were drawn with the best value by using the minimum criterion, that is, the dotted line indicates the number of selected features. As shown in the figure, twelve features with non-zero coefficients were selected, including 7 radiomics features and 5 deep features.

In Figure 4, each dotted vertical line represents a feature selection result for a feature group. The base of log(λ) is e. (a) LASSO coefficients produced by the regression analysis (in b). Twelve resulting predictors with nonzero coefficients were indicated in the plot. (b) Selection of tuning parameter (λ) in the LASSO model via 10-fold cross-validation. Mean square errors from the LASSO regression's cross-validation procedure were plotted as a function of log(λ). λ is the tuning parameter. Numbers along the upper x-axis represent the average number of predictors. Red dots indicate average error values for each model with given λ, and vertical bars through the red dots show the upper and lower values of the errors. The vertical black lines define the optimal values of λ, where the model provides its best fits to the data. The optimal λ value of 1.0299 with log (λ) = 0.0295 was chosen via 10-fold cross-validation based on the minimum criteria.

4.2.   The nomogram and its performance

In the training cohort, we integrated the combined signature with clinical information to generate a nomogram (Figure 5). For the independent validation cohort, the nomogram was used to predict of placenta invasion with the accuracy, sensitivity, specificity and AUC of 0.941, 0.952, 0.923 and 0.985 respectively.

In the nomogram, CS, CSC and PP stand for combined signature, history of cesarean section or curettage and placenta previa respectively.

4.3.   Compared with traditional machine learning methods

Various techniques and methods were used for prediction of placenta invasion, each technique has its own respective accuracy level. Here, NDRC was compared with Decision Tree, Random Forest (RF), Naive Bayes and Support Vector Machine methods based on radiomic features in terms of accuracy, sensitivity, specificity and AUC. Then, NDRC was also compared with RF method based on deep features. Table 3 summarizes the results of all different methods on the independent validation cohort with a classification cutoff of 0.5.

It can be seen in Table 3 that the prediction accuracy of the machine learning method based on deep features is slightly higher than that of the machine learning method based on radiomics features, and the accuracy of the NDRC method is higher than the above two methods. In addition to the accuracy, sensitivity, and specificity, we have evaluated different methods using receiver operating characteristic (ROC) curves (Figure 6).

5.   Conclusions

In this study, we developed a nomogram combining radiomics, deep learning and clinical data, which can predict placenta invasion prenatally after 24 weeks of pregnancy. Radiomics features are clearly designed or hand-made and these features reflect the shallow and low-level characteristics of the image which cannot fully characterize the heterogeneity inside the placenta when applied to the detection of placenta invasion. According to previous reports, excessive hypertrophy and fibrin deposition in the placenta can lead to intra-placenta heterogeneity ^[24]. Therefore, the use of imaging data to find intra-placenta heterogeneity can effectively predict placenta invasion. However, placental heterogeneity the imaging data reflects is difficult to quantify and usually develops with gestational age after the second trimester, which complicate the visual assessment of placenta invasion. Therefore, we further train the deep neural network to extract deep features inside the placenta to better characterize the placental heterogeneity.

On the other hand, previous studies have found that as the number of cesarean delivery increases, the risk of PI also increases ^[25,26,27]. In order to further improve the accuracy of the prediction model, we take age, history of cesarean section or curettage and placenta previa as clinical factors of placental invasion and combine them with the extracted radiomics and deep features optimizing through feature screening, then construct a nomogram that can be used for prediction of placenta invasion. The results show that the nomogram model constructed in this research achieves higher detection accuracy and AUC than machine learning methods based on radiomics or deep features alone. At the same time, the nomogram model also confirmed that history of cesarean section or curettage and placenta previa are the factors that have the dominant ability to predict placenta invasion.

We need to point out some limitations of current research. Because this study is retrospective, there will inevitably be some biases or may affect the analysis and the sample size (n = 112) is relatively small. Despite these limitations, it has the advantage that not all data is collected in a single location using a single scanner. The training cohort and validation cohort data are from different hospitals, so the results obtained in this study have been externally verified by data from different vendors with different acquisition parameters.

A number of studies have shown that delivery in a specialized center for PI patients can significantly reduce the incidence of maternal morbidity ^[28,29,30]. However, due to the uneven distribution of medical resources and the uneven development of urban and rural areas, medical institutions at and below the county level cannot detect and diagnose PI early and lack treatment experience, which lead to a poor prognosis for the parturient and perinatal period. The nomogram developed in this paper can help doctors make clinical decisions for pregnant women with suspected placental invasion. These results also provided support for the initiation of a new prospective study to systematically evaluate the diagnostic capabilities of the combination of radiomics features and clinical parameters and to improve diagnosis and patient care.

Acknowledgments

This study has received funding by the Natural Science Foundation of Zhejiang Province under Grant LY20H180003; the Natural Science Foundation of Ningbo under Grant 2019A610104; the Public Welfare Science and Technology Project of Ningbo under Grant 202002N3104 and the Project in Science and Technique Plans of Ningbo under Grant 2019C50081.

Conflict of interest

The authors declare there is no conflict of interest.

Appendix A

Appendix B

In Table B.1, 1 × 1 × 1, 3 × 3 × 3 and 7 × 7 × 7 in the table are the size of the convolution kernel; 64,128,256, 1024 and 2048 are the number of channels output after each convolution; The network structure contained in each square bracket is a "bottleneck" building block, take conv3_x for example as shown in Figure B.1.

Bottleneck is designed to reduce the number of parameters. Considering the cost of calculation, the calculation of the residual block is optimized, that is, the convolutional layer is designed to be 1× 1 × 1 + 3 × 3 × 3 + 1 × 1 × 1. First, the dimension is reduced by the convolution kernel of 1 × 1 × 1, and the result is input to the intermediate layer, and then restored under the convolutional layer of 1 × 1 × 1, which not only maintains the accuracy but also reduces the amount of calculation. The first 1 × 1 × 1 convolution reduces the 512-dimensional channel to 128 dimensions, and then recovers the 512 dimensions by 1 × 1 × 1 convolution at the end.

Appendix C (Implementation details of decision tree, random forest, naive Bayes, and support vector machine)

C.1.   Decision tree

Decision tree is a predictive analysis model expressed in the form of a tree structure. Each of its unique tree-type classification graphs represents a feature from the root node to the leaf node. The important theoretical basis of the decision tree algorithm is "Gini Index" and "Information Entropy" which are analytical tools for quantifying information. Entropy represents the degree of randomness of the element, and its calculation formula is shown in (1):

where $x$ represents the discrete random variable, $p\left( x \right)$ represents the probability of the variable $x$ occurrence, the greater the probability, the smaller the entropy value. The definition of Gini coefficient is similar to entropy value. The larger the Gini coefficient, the larger the entropy value, indicating the higher the degree of randomization of the element.

C.2.   Random forest

The steps of the random forest algorithm are as follows: 1) Generate a decision tree. 2) Combine the decision trees into a random forest. 3) The classification results of all decision trees in the random forest are voted, and the voting result is the classification result of the final model. The classification voting process is shown in Eq (2):

where $Y$ is the output variable; $H\left( x \right)$ is the $Y$ that returns the most votes; $k$ is the number of decision trees; ${h_i}\left( x \right)$ is the classification model of a single decision tree; $I$ is the indicative function.

C.3.   Naive Bayes

Naive Bayes is a method based on the classic Bayes principle, which can be used to build models for classification, prediction and evaluation. The classic Bayes principle can be expressed as Eq (3):

where $P\left( {A/B} \right)$ is the posterior probability which refers to the probability of the event $A$ occurring when the event $B$ occurs; $P\left( A \right)$ and $P\left( B \right)$ are the probability of occurrence of events $A$ and $B$ respectively, and $P\left( A \right)$ is also called prior probability; $P\left( {B/A} \right)$ is the probability of event $B$ when event $A$ occurs which is the posterior probability of event $A$. Given there is a data set $D = \left\{ {{{{d}}_1}, {d_2}, ..., {d_m}} \right\}$, a total of $m$ data, each data has n-dimensional features, the feature set is $C = \left\{ {{{{c}}_1}, {c_2}, ..., {c_n}} \right\}$, and each influencing factor feature has several levels, denoted as ${c_{ij}}$. According to the feature set, the data set can be divided into $k$ categories. Assuming that the features are independent of each other and the influence weights are equal, according to the principle of naive Bayes, the probability of being classified into ${y_i}$ under the horizontal condition ${{{c}}_{1i}}, {c_{2i}}, ..., {c_{ni}}$ is shown in Eq (4):

C.4.   Support vector machine

The basic idea of support vector machine is to solve the separation hyperplane that can divide the data set correctly and has the largest geometric interval. It has high application value in both linear and non-linear data structures. When dealing with data sets with linear relationships, the optimal linear partition equation is shown in Eq (5):

The formula for the distance from the point in the space to the best classification hyperplane is shown in Eq (6):

As long as the distance is maximized, the best dividing line and the best classification hyperplane can be found. For the nonlinear classification problem in the input space, it is transformed into a linear classification problem in a certain dimensional feature space by nonlinear transformation, and linear support vector machines are learned in the high-dimensional feature space.