Hierarchical volumetric transformer with comprehensive attention for medical image segmentation

Zhuang Zhang; Wenjie Luo; Zhuang Zhang; Wenjie Luo

doi:10.3934/mbe.2023149

Mathematical Biosciences and Engineering

2023, Volume 20, Issue 2: 3177-3190. doi: 10.3934/mbe.2023149

Previous Article Next Article

Research article Special Issues

Hierarchical volumetric transformer with comprehensive attention for medical image segmentation

Zhuang Zhang ¹,
Wenjie Luo ^{1,2
,
,}

1.
School of Cybersecurity and Computer, Hebei University, Baoding 071002, China
2.
Laboratory of Intelligence Image and Text, Hebei University, Baoding 071002, China

Academic Editor: Shangce Gao

Received: 02 October 2022 Revised: 21 November 2022 Accepted: 23 November 2022 Published: 02 December 2022

Transformer is widely used in medical image segmentation tasks due to its powerful ability to model global dependencies. However, most of the existing transformer-based methods are two-dimensional networks, which are only suitable for processing two-dimensional slices and ignore the linguistic association between different slices of the original volume image blocks. To solve this problem, we propose a novel segmentation framework by deeply exploring the respective characteristic of convolution, comprehensive attention mechanism, and transformer, and assembling them hierarchically to fully exploit their complementary advantages. Specifically, we first propose a novel volumetric transformer block to help extract features serially in the encoder and restore the feature map resolution to the original level in parallel in the decoder. It can not only obtain the information of the plane, but also make full use of the correlation information between different slices. Then the local multi-channel attention block is proposed to adaptively enhance the effective features of the encoder branch at the channel level, while suppressing the invalid features. Finally, the global multi-scale attention block with deep supervision is introduced to adaptively extract valid information at different scale levels while filtering out useless information. Extensive experiments demonstrate that our proposed method achieves promising performance on multi-organ CT and cardiac MR image segmentation.

Keywords:

Citation: Zhuang Zhang, Wenjie Luo. Hierarchical volumetric transformer with comprehensive attention for medical image segmentation[J]. Mathematical Biosciences and Engineering, 2023, 20(2): 3177-3190. doi: 10.3934/mbe.2023149

Related Papers:

[1]	Dan Liu, Shigui Ruan, Deming Zhu . Stable periodic oscillations in a two-stage cancer model of tumor and immune system interactions. Mathematical Biosciences and Engineering, 2012, 9(2): 347-368. doi: 10.3934/mbe.2012.9.347
[2]	Yuyang Xiao, Juan Shen, Xiufen Zou . Mathematical modeling and dynamical analysis of anti-tumor drug dose-response. Mathematical Biosciences and Engineering, 2022, 19(4): 4120-4144. doi: 10.3934/mbe.2022190
[3]	Marek Bodnar, Urszula Foryś . Time Delay In Necrotic Core Formation. Mathematical Biosciences and Engineering, 2005, 2(3): 461-472. doi: 10.3934/mbe.2005.2.461
[4]	Svetlana Bunimovich-Mendrazitsky, Yakov Goltser . Use of quasi-normal form to examine stability of tumor-free equilibrium in a mathematical model of bcg treatment of bladder cancer. Mathematical Biosciences and Engineering, 2011, 8(2): 529-547. doi: 10.3934/mbe.2011.8.529
[5]	Konstantin E. Starkov, Giovana Andres Garfias . Dynamics of the tumor-immune-virus interactions: Convergence conditions to tumor-only or tumor-free equilibrium points. Mathematical Biosciences and Engineering, 2019, 16(1): 421-437. doi: 10.3934/mbe.2019020
[6]	Bertin Hoffmann, Udo Schumacher, Gero Wedemann . Absence of convection in solid tumors caused by raised interstitial fluid pressure severely limits success of chemotherapy—a numerical study in cancers. Mathematical Biosciences and Engineering, 2020, 17(5): 6128-6148. doi: 10.3934/mbe.2020325
[7]	Maria Vittoria Barbarossa, Christina Kuttler, Jonathan Zinsl . Delay equations modeling the effects of phase-specific drugs and immunotherapy on proliferating tumor cells. Mathematical Biosciences and Engineering, 2012, 9(2): 241-257. doi: 10.3934/mbe.2012.9.241
[8]	Ge Song, Tianhai Tian, Xinan Zhang . A mathematical model of cell-mediated immune response to tumor. Mathematical Biosciences and Engineering, 2021, 18(1): 373-385. doi: 10.3934/mbe.2021020
[9]	Ghassen Haddad, Amira Kebir, Nadia Raissi, Amira Bouhali, Slimane Ben Miled . Optimal control model of tumor treatment in the context of cancer stem cell. Mathematical Biosciences and Engineering, 2022, 19(5): 4627-4642. doi: 10.3934/mbe.2022214
[10]	Yuting Ding, Gaoyang Liu, Yong An . Stability and bifurcation analysis of a tumor-immune system with two delays and diffusion. Mathematical Biosciences and Engineering, 2022, 19(2): 1154-1173. doi: 10.3934/mbe.2022053

Abstract

1. Introduction

Breast cancer is the second most common cancer in women after skin cancer ^[1]. According to WHO statistics, about 7.8 million women were diagnosed with breast cancer by the end of 2020 ^[2]. It is getting more ubiquitous with each passing day. Resultantly, it is expediting the mortality rate all over world. Moreover, it has become the second greatest cause of mortality among women ^[3]. This threatening trend can be countered if the lumps are detected earlier ^[4]. In the early screening of breast cancer diagnosis, the mammogram is extensively used due to its comparatively inexpensive characteristics ^[5]. During the diagnosis process, factors like radiologist experience and activeness play a vital role ^[6,7]. Breast cancer is caused by the irregular growth of cells in the breast. It starts as cells begin to develop in an unbalanced fashion ^[8]. Cancer-free breast tissue may expand unusually, but it does not expand beyond the breast. A healthcare provider must test any breast lump or shift to see whether it is benign or malignant? A tumor can be benign (non-cancerous) or malignant (cancerous). Benign cells are universal. Further, they grow rapidly and do not infiltrate neighboring tissues or spread to other body parts. Malignant tumors, on the other hand, are cancerous in character. Malignant cells, if left untreated, ultimately go beyond the initial tumor's area and extend their reach to the other parts of the body ^[9].

Plethora of imaging systems have been developed to diagnose and cure the breast cancer. Besides, many models meant for the earlier detection of this menace have been suggested by the experts. Moreover, many studies have been carried out ^[10,11] to enhance the diagnosis accuracy of this threat. Data mining has proved very useful in extracting the important data from a large set of data ^[12]. Further, the varied techniques of this discipline have been extensively exploited in the discovery of many diseases. Moreover, the techniques like machine learning (ML), statistical analysis, data warehouse, fuzzy systems, databases, and neural networks have been employed in the prediction and diagnosis of various types of cancer ^[13]. Additionally, deep models helped a lot to the academicians and practitioners while they grappled the intricacies in the real-world training ^[14,15,16]. In the training of ML and deep learning (DL) models, the gradient descent optimization method is crucial. In recent years, many new variant algorithms ^{[17,18,19,20]} have been developed to improve it further. Machine learning is a branch of computer science that make computers to comprehend the tasks without explicitly teaching them to do so ^[21,22,23]. Introducing a cost function to machine learning and data mining enables the machines to discover appropriate weights for outcomes ^[24]. Optimization finds the function parameters in a way that the solution of the problem becomes simpler. Many machine learning methods are plagued by this problem ^[25,26].

Gradient descent is applied for optimization of multiple loss functions like Support Vector Machine (SVM), and Logistic Regression (LR) methods ^[27]. Gradient descent optimization-based techniques for binary classifications have produced better accuracy for the detection of diseases. These diseases, in turn, are based on certain parameters which provide the necessary basis to the research for carrying out breast cancer detection ^[28]. Deep learning is an offshoot of ML techniques whose primary thrust is on the acquisition of data models rather than job-specific algorithms ^[29]. Common classical diagnosis methods are not delivering the way the current era needs. So, more accurate and reliable breast cancer diagnosis techniques are required to thwart the rising numbers of death of women ^[30]. The objective of this study is to build a deep learning-based model for breast cancer diagnosis that has a better prediction rate with minimum complexities. To develop such a diagnostic system, deep extreme machine learning using gradient descent optimization technique is being proposed in this work. DEGDO based model consists of two major phases; first is the training and the second one is the validation phase. There are three major layers in the training phase. The data acquisition layer gets the data to be used in training that is, in turn, taken from some source or test reports. Data acquisition layer stores that data in the object layer where it is in raw form. Data in the object layer can contain noise which is needed to be removed. Preprocessing layer processes the raw data in order to remove noise and to handle missing values. The application layer is the main part of the training phase which consists of two sub layers, i.e., predication layer and the evaluation layer. The prediction layer holds the actual DEGDO to predict the disease and then the result is evaluated by the evaluation layer based on certain evaluation parameters. Moreover, accuracy is measured and get it compared with the required accuracy. If it meets the training criterion, training process ends and the trained model is stored on the server. In case, training criterion is not met, the required accuracy model is retrained. This process continues unless it obtains the required accuracy. In the validation phase, data is provided to the trained model imported from the server and the result of the diagnosis is predicted. To carry out the experiments, a dataset has been taken from UCI Machine Learning Repository for the proposed research. This dataset is based on 569 instances with 32 attributes for each record ^[31]. The proposed model predicts the diagnostic status as positive or negative based on the provided set of parameters.

The current study is geared towards the enhancement of the prognosis accuracy of the breast cancer detection. Moreover, it introduces a more reliable classification method using deep extreme with gradient descent optimization technique resulting in a higher diagnostic accuracy rate. Apart from that, this work also presents a comparative study of the state-of-the-art methods on the same dataset. Ten different evaluation metrics have been employed to demonstrate the utility, effectiveness and authenticity of the proposed work. These metrics span accuracy, specificity, sensitivity, positive predicted value (PPV), negative predictive value (NPV), false-positive rate (FPR), false-negative rate (FNR), false discovery rate (FDR), F-score (F1) and Matthews correlation coefficient (MCC). Moreover, the area under the curve (AUC) from the receiver operating characteristic (ROC) curve has also been used for the evaluation of the proposed model. Besides, both the split and K-fold cross-validation have also been applied.

Rest of the article has been formatted like this. Section 2 summarizes the relevant literature. Section 3 describes the proposed model and the procedure for conducting a detailed evaluation about the Breast cancer like malignant or benign. Details of the dataset used for research have been given in the Section 4. Section 5 sheds the necessary light on the experimental results, discussion and the ensuing findings. Finally, the paper has been concluded in the last Section 6.

2. Literature review

The conventional approach to cancer diagnosis rests upon a technique called colloquially a "gold standard" which entails three screenings: clinical assessment, radiographic, and pathology ^[32]. The traditional approach, rooted in regression, shows the existence of malignancy. Whereas, the state-of-the-art machine learning methods and algorithms are based on model development. A model is created to predict previously unknown data. As this model is sparked with a list of parameters depending upon the nature of the problem, it generates the required outcome through the twin processes of training and testing ^[33]. Pre-processing, feature selection or extraction, and classification are the three major processes used in the machine learning ^[34].

Breast cancer is the second leading cause of death after the lung cancer. 8% of women throughout their lifetime are diagnosed with breast cancer. Machine learning techniques have been frequently employed to categorize the breast cancer. Researches carried out to diagnose breast cancer with the help of KNN classifier offered better accuracy (97.5%) with a lower error rate compared to its Naive Bayes counterpart (96.19%) ^[35]. Detection of breast cancer with the help of algorithms has become a pronounced medical problem in the current era. Diagnosis of early breast cancer is, no doubt, a key to the survival of a patient. This study showed how a decision tree algorithm, along with some other approaches, was used to construct a real breast cancer diagnosis model for a clinical and a systematic treatment. The experimental results demonstrated the viability and do-ability of the proposed concept. The effectiveness of the decision tree technique for the detection of breast cancer has been studied and shown through the experiments ^[36]. Besides, a combination of K-means and K-Support Vector Machine (K-SVM) algorithms was developed for extracting valuable information which could be utilized to diagnose the tumor. Moreover, K-means algorithm has been used for the detection of the secret/hidden patterns of the cancerous cells. Each tumor membership was measured and viewed as a new trait in these patterns. Apart from that, a support vector machine was employed to acquire a novel classifier in order to distinguish between benign and malignant tumors. The proposed technique raised the accuracy of Wisconsin diagnostic breast cancer results to the mark of 97.38%. Findings given in ^[37] not only reflected the potential of the recommended solution for the diagnosis of breast cancer but it also indicated the lower cost of time during preparation phase.

To identify the hematoxylin and eosin-stained-breast biopsy photos using the convolutional neural network, the researcher utilized a deep-learning-based method and obtained 83.30% accuracy ^[38]. Moreover, the Sequential Minimum Optimization (SMO) and K Nearest Neighbor Classification Algorithms (IBk) were used for the breast cancer estimation using certain ensemble techniques. Apart from that, the dataset used for the experiments purpose consisted of 683 records with 9 parameters for each record. Moreover, Weka data mining tool was used for this research, which used cross-validation (K Fold) technique to determine the accuracy of the proposed method for breast cancer diagnosis. Additionally, the SMO achieved 96.19% accuracy while IBk reached to the mark of 95.90% for breast cancer detection ^[39]. Breast cancer data was collected from Iranian Centre on Breast Cancer (ICBC). The results were evaluated by the researchers through an array of validation metrics like specificity, and sensitivity of DT (C4.5), ANN, and SVM-based models. The ensuring results showed t accuracy hat SVM, with an accuracy of 95.70%, was the best predictive algorithm for breast cancer screening ^[40]. Apart from that, using the ADTree, J48, and CART algorithms taking digital files in DOCOM format, the Indian Breast Cancer Center Adyar, Chennai's breast cancer dataset was examined to build a model for breast cancer diagnosis. The dataset used by the researchers was in the CSV format. In this work, three different data mining (DM) algorithms were used to investigate the accuracy of prediction. The findings out of this study revealed that the CART algorithm is more suitable than the others as CART achieved an accuracy of 98.5% while ADTree and DT J48 achieved accuracies of 97.7 and 98.1% respectively ^[41]. Moreover, researchers performed a comparative study on NB, RF, LR, ANN-MLP, and KNN based models for breast cancer diagnosis. Apart from that, UCI data was used to diagnose breast cancer by taking the top ten different parameters from the dataset. Each algorithm was applied to the dataset to test the output of each model. The accuracies of the instances identified by KNN, NB, and RF were 72.3, 71.6 and 69.5% in a respective fashion. While, the accuracies of LR and ANN-MLP remained just 68.8 and 64.6% respectively ^[42]. In Nigeria, breast cancer is a very common disease. Further, no diagnosis is available for such a heterogeneous disease in Nigeria. Dataset having 17 attributes was taken by LASUTH, Nigerian Cancer Registry. The NB probabilistic method was applied for controlling the dependent group count on the probabilistic model. Top to bottom greedy checks on training data were implemented in the decision tree J48. J48 came out to be the most suitable procedure to predict and diagnose breast cancer as its accuracy was 94.20% while NB obtained an accuracy of just 82.60% ^[43].

The deep learning approaches including multiple kernel/activation functions like maxout, tanh, and exprectifier to diagnose breast cancer on infected cells, were applied. Moreover, a comparison of different ML techniques like NB, DT, SVM, and RF was carried out on the Wisconsin dataset. It was shown that the highest accuracy of 96.99% was obtained by Exponential Rectifier Linear Unit activation function - a deep learning algorithm in order to diagnose breast cancer ^[44]. Apart from the breast cancer diagnosis dataset, DT, NB, and KNN were also introduced to build a model for diagnosing the breast cancer. In this study, researchers utilized the original Wisconsin dataset. The results indicated that the accuracy of the NB classifier reached to the mark of 95.99% which was higher than those of DT and KNN algorithms ^[45]. Moreover, comparative analyses of different nonlinear supervised learning models like MLP, KNN, SVM, CAR Tree and Gaussian NB have been carried out for the detection of breast cancer. Comparative analysis for the said methods was the main theme of the research for efficient breast cancer identification. Apart from that, prediction accuracy for every algorithm was independently calculated using the Wisconsin breast cancer dataset. Moreover, for performance analysis of algorithms, a cross-validation (K-Fold) approach has been applied. MLP produced 96.70% accuracy for breast cancer detection which was higher among all ^[46]. To predict the restorative emergence of breast cancer, a data mining-based model was developed. Additionally, there were two major methods for this model namely Extreme Learning Machine (ELM) and the Bat algorithm. The prejudices and random weights were generated using the Bat algorithm. MATLAB software was used for carrying out experiments on the Wisconsin breast cancer dataset with certain selected attributes. Besides, the coefficient correlation approach was used for attributes selection. ELM and Bat algorithms have been employed to predict whether the breast cancer was recurring or non-recurring. To verify the consistency of research at various training levels, tanh and sigmoid activation functions were applied. When tanh was used as an activation function, 93.75% accuracy was recorded with a minimum error rate (RMSE = 0.30) ^[47]. The greedy search algorithm was proposed to build a diagnostic system to predict breast cancer diagnosis.

For the selection of features that are important ranging from the broad set of data to the trivial ones, SVM with Constrained Search Sequential Floating Forward Search (CSSFFS) is used. For this experiment, the dataset was compiled from the WDBC machine learning database. Researchers used the cross-validation (K-Fold) technique to establish results for CSSFFS with SVM. The main purpose of using SVM was to eliminate irrelevant features. Using the CSSFFS method, with some top attributes, accuracy was enhanced up to 98.25%. RBF network produced a decent accuracy of 93.60% when all the attributes were taken into consideration ^[48]. Besides, a deep learning-based automated mammography processing technique was employed for estimating the patients' risk of getting breast cancer ^[49]. The automatic classification was conducted for the region containing the cancerous part in images of the breast. Grasshopper optimization algorithm and CNNs were used in their proposed research. Their research model was able to manage 93% accuracy ^[50]. Moreover, breast cancer detection with the help of histogram images using deep learning CNN was carried out by the authors. This study showed reasonable results by obtaining an 86.60% accurate detection rate ^[51]. In a yet another work, a deep feature-based model was used for the breast mass categorization which, in turn, employed CNNs and decision trees ^[52]. A Patch-based LeNet, a U-Net, and transfer learning techniques were employed using a pre-trained FCN-AlexNet in order to identify lesions in breast ultrasound images ^[53]. A CNN which is pre-trained and its learned parameters were transferred to some other CNN for classification of mitoses obtained a 0.80 F-Score ^[54]. In another work, ELM was used to categorize breast tumor characteristics and its results were compared to the SVM classifier ^[55]. By training CNN with a huge quantity of time series data, ^[56] used CNN for the risk prediction of breast cancer. Based on 420 mammography time series data, ^[57] utilized deep neural networks to forecast the probability of breast cancer in near future. Moreover, ^[58] abstracted breast tumor representations using CNN and subsequently categorized the tumors as malignant or benign. Because features may influence both the efficacy and efficiency of a breast CAD system, ^[59] proposed an image retrieval system utilizing Zernike Moments (ZMs) to get the required features. Apart from that, machine learning and deep leaning optimization are popular techniques which have the potential to get used in the diverse fields like price controlling in medical domain, agriculture, business intelligence etc. Moreover, Chun-Hui He' iteration algorithms can also be used for optimization ^[60,61].

3. Research methodology

Deep extreme gradient descent optimization-based model comprises of two stages: training and validation. Moreover, the data acquisition layer, the pre-processing layer, and the application layer are the three levels that make up the training phase. Additionally, the data acquisition layer collects data from some source and stores it in raw form which is used as a database in future. This raw data may contain noisy values as well since it is transmitted through an online link from the source to the acquisition layer. Apart from that, pre-processing layer deals with missing values and eliminate noise from the given data. Moreover, the moving average technique is employed to approximate the missing data. Further, normalization addresses the problem of noise. As pre-processing is completed, the application layer starts its work. The application layer comprises of two sub-layers: prediction and performance evaluation. Deep extreme gradient descent optimization method is used in the prediction layer. Moreover, the performance evaluation layer assesses the predictive model's performance in terms of the validation metrics like accuracy, sensitivity, specificity, precision, and miss rate. As the required learning criterion is met, the trained model is stored on the server which, of course, is used in a later phase. Apart from that, the validation phase uses the data acquisition layer which provides data as an input to the trained model. This trained model is, in turn, imported from the server to predict the disease. Figure 1 demonstrates the methodological diagram of the proposed model based on deep extreme gradient descent optimization.

Figure 1. Methodological diagram of proposed deep extreme gradient descent optimization-based model.

DownLoad: Full-Size Img PowerPoint

Deep extreme learning machine has been applied in the varied fields. Since conventional artificial neural network needs more samples so it consumes more time for learning. Besides, it can produce over-fit results ^[62]. The deep extreme learning machine is extensively used for regression and classification problems in different fields. Its learning rate is better and computational complexity is much lower than that of traditional artificial neural networks. The structure of a deep extreme learning machine model consists of three layers namely the input layer, multiple hidden layers, and the output layer ^[63]. Extreme learning was firstly idealized by ^[64].

shows the diagrammatical model for the proposed system based on DEGDO, where $\mathrm{i}\mathrm{p}$ denotes the nodes of the input layer; hidden layer nodes are represented by $\mathrm{h}$ , and $\mathrm{O}\mathrm{D}\mathrm{P}$ shows the output layer node.

Figure 2. The diagrammatic representation of deep extreme learning.

DownLoad: Full-Size Img PowerPoint

A mathematical representation of the filter of moving average is given in Eq (1) ^[77].

$P\left[x\right] = \frac{1}{G}{\sum }_{T = 0}^{G-1}\mathrm{u}\;(\mathrm{x}\;+\;\mathrm{T}\;)$

(1)

Here u represents inputs; P denotes output, and the point of moving average is denoted by G. To increase the predictive ability and to improve the training process of the machine learning model, dataset is standardized for the interval [0, 1] with the help of Eq (2) ^[77,78].

$C = \frac{{u}_{x}-{u}_{min}}{{u}_{max}-{u}_{min}};\mathrm{x} = \mathrm{1, 2}, 3\dots \mathrm{N}\;\;\;\;\;$

(2)

At first taking training $[A, B] = [{a}_{v}, {b}_{v}] (x = 1, 2, ..., R)$ and input $A = [{a}_{v1}, {a}_{v2}, {a}_{v3}...{a}_{vr}]$ samples and $B = \left[{b}_{11}, {b}_{12}, {b}_{13}, ..., {b}_{1r}\right]$ as target matrix, has been taken from training samples then matrices $A$ and $B$ , which can be presented as provided in Eqs (3) and (4) respectively ^[77,78,79]. While $A$ and $B$ are considered the dimensions of the input and output matrix. The extreme learning machine is utilized to adjust the weights between the input and the hidden layers. Considering the ${\mathrm{C}}^{\mathrm{t}\mathrm{h}}$ as the input layer node and ${l}^{th}$ as the hidden layer node, the weights between them are represented by ${C}_{E1}$ as given in Eq (5) ^[80]. Where Matrix A represents as input features; B as target matrix, C as Input layer node, weights between Input and hidden layer and D as weights between hidden neurons and output layer neurons.

$A = \left[\begin{array}{c}a11\;\;a12 \;\dots\; a1v\\ a21\;\;a22 \;\dots \;a2v\\ a31\;\;a32 \;\dots\; a2v\\ .\;\; \;\;\;\;\;\;. \;\;\;\; .\;\;\;\;\\ {a}_{E1}\;\;{a}_{E2}\;\dots \;{a}_{Ev}\end{array}\right]$

(3)

$B = \left[\begin{array}{c}b11\;\;b12 \;\dots\; b1v\\ b21\;\;b22\; \dots \;b2v\\ b31\;\;b32\; \dots\; b2v\\ .\;\; \;\;\;\;\;\;. \;\;\;\; .\;\;\;\;\\ {b}_{E1}\;\;{b}_{E2}\;\dots \;{b}_{Ev}\end{array}\right]$

(4)

$C = \left[\begin{array}{c}c11\;\;c12 \;\dots\; c1v\\ c21\;\;c22 \;\dots\; c2v\\ c31\;\;c32 \;\dots\; c2v\\ .\;\; \;\;\;\;\;\;. \;\;\;\; .\;\;\;\;\\ {c}_{E1}\;\;{c}_{E2}\;\dots\; {c}_{Ev}\end{array}\right]$

(5)

$D = \left[\begin{array}{c}d11\;\;d12 \;\dots \;d1v\\ d21\;\;d22 \;\dots \;d2v\\ d31\;\;d32\; \dots \;d2v\\ .\;\; \;\;\;\;\;\;. \;\;\;\; .\;\;\;\;\\ {d}_{E1}\;\;{d}_{E2}\;\dots \;{d}_{Ev}\end{array}\right]$

(6)

Furthermore, Eq (7) shows the randomly selected biases for the hidden layer nodes by Extreme Learning Machine ^[81,82]. A function represented as $f\left(x\right)$ is a network activation function preferred by Extreme Machine Learning. Eq (8) shows the resulting matrix given in the data acquisition layer. Equation (9) represents the column vector of $S$ i.e., the resulting matrix ^[60,77,80].

$B = {\left[b1, \;\;b2, \;\;b3 \;\;\dots \;\;bE\right]{'}}$

(7)

$H = {\left[h1, \;\; h2, \;\; h3\;\; \dots \;\;{h}_{z}\right]}_{x\times y}$

(8)

$\begin{array}{l} \;\;\;\;\;\;\;\;\;\;\;{\rm{h}}1 = \left[ {\begin{array}{*{20}{c}} {{h_{1j}}}\\ {{h_{2j}}}\\ {{T_{3j}}}\\ \vdots \\ {{T_{xj}}} \end{array}} \right]\\ = \left[ {\begin{array}{*{20}{c}} {\sum _{q = 1}^ \propto {M_{l1}}f({\omega _l}{\alpha _q} + {b_l})}\\ {\sum _{q = 1}^ \propto {M_{l2}}f({\omega _l}{\alpha _q} + {b_l})}\\ {\sum _{q = 1}^ \propto {M_{l3}}f({\omega _l}{\alpha _q} + {b_l})}\\ \vdots \\ {\sum _{q = 1}^ \propto {M_{ly}}f({\omega _l}{\alpha _q} + {b_l})} \end{array}} \right]\left( {q = {\rm{1}}, {\rm{2}}, 3, \ldots , y} \right) \end{array}$

(9)

Z is the hidden layer outcome and transposition of $H$ has been denoted by $H{'}$ . Equation (11) shows the calculations of matrix $\beta$ 's weighted values by using the least square method ^[79,81].

$Z\beta = {H}^{{'}}$

(10)

$\beta = {Z}^{+}{H}^{{'}}$

(11)

To increase the overall stability of the network, $\beta$ regularization term has been utilized ^[65].

Deep learning has become a topnotch research niche for the scientists due to the marvelous it has. Minimum four input layers and one output layer is needed for a system to qualify as a deep learning system ^[64]. Neurons present in different layers of deep learning networks are trained with different parameters based on the result of the previous layer. Besides, a deep learning network bears immense promise to process extensive datasets. In order to capture the positive and outstanding features of both ELM and DL, the proposed work is utilizing a deep extreme gradient descent optimization-based approach. The proposed model is based on a deep extreme learning machine with gradient descent that consists of one input layer, six hidden layers, and one output layer. Moreover, the input layer contains sixteen (16) neurons. Besides, each of the hidden layers consists of sixteen (16) neurons as well. Whereas, the output layer consists of only one (1) neuron. For the selection of the hidden layer's number of nodes, a test and error scheme is applied. The output of 2^nd hidden layer is obtained as ^[77,81]:

${Z}_{g} = H{\beta }_{g}^{+};\;\mathrm{g} = \;\mathrm{1, 2}, \mathrm{3, 4}\;\dots , \;6$

(12)

where ${\beta }^{+}$ represents the general inverse of the $\beta$ matrix. Equation (11) can be helpful for obtaining values for the 2^nd hidden layer ^[77,80].

$f\left({W}_{g}Z+{B}_{g}\right) = {Z}_{g}$

(13)

Four parameters are present in Eq (13). Among them, ${W}_{g}$ represents the weight of the first 2 hidden layers. In this layer, the first neurons preference is shown by $Z$ . ${B}_{g}$ is the measured first hidden layer's output and ${Z}_{g}$ as the second hidden layer's projected output ^[78,79].

${{W}_{F}}_{G} = {f}^{-1}\left(g\right){F}_{G}^{+}$

(14)

${F}_{G}^{+}$ represents the inverse of ${F}_{G}.$ Moreover, to calculate Eq (5), $f\left(x\right)$ has been used as an activation function ^[80,81]. Therefore, $f\left(x\right), \mathrm{t}\mathrm{h}\mathrm{e}$ activation function is corrected to revise the second hidden layer's outcome given below.

${Z}_{g+1} = f\left({{W}_{F}}_{G}{Z}_{G}\right)$

Such as ${{W}_{F}}_{G}{Z}_{G} = Q{h}_{g+1}$

${Z}_{g+1} = f\left(Q{h}_{g+1}\right)$

(15)

As per Eq (16), weighted matrix $\beta$ between the 2^nd and 3^rd layers has been updated ^[80]. ${Z}_{g+1}^{+}$ is an inverted form of ${Z}_{g+1}$ . Equation (17) provides the result of the estimated layer ^[80,81].

${\beta }_{g+1} = {Z}_{g+1}^{+}H$

(16)

${H}_{\beta }^{+}$ is the inverse of the weight matrix ${\mu }_{g+1}$ . Then matrix ${{FW}_{F}}_{G}$ = [ ${B}_{g+1}$ , ${W}_{g+1}$ ] is set by a deep extreme learning machine. Equations (10) and (11) help to achieve the output of the further layer ^[77,78,79].

$f\left(x\right) = \frac{1}{{1+e}^{-x}}$

(17)

The back-propagation algorithm provides weight initialization, feed-forward, error back-propagation, and updating weights and bias. $f\left(x\right) = sigmoid\left(x\right)$ being an activation function is present on hidden layers of every neuron. The hidden layer and sigmoid input function of DELM can be composed through this method ^[80,81];

${\mathrm{E}\mathrm{r}\mathrm{r}\mathrm{o}\mathrm{r}}_{\mathrm{B}\mathrm{P}} = \frac{1}{2}{{\sum }_{n}\left(a{o}_{n}-{to}_{n}\right)}^{2}$

(18)

$to$ = desired output t

$ao$ = measured or calculated output

Equation (18) shows the error's back-propagation. Adjustment of weights is required to minimize the overall error ^{[77,78,79,80,81]}. Equation (19) presents the output layer's rate of weight change ^{[77,78,79,80,81]}.

$\Delta {Z}_{m, n}^{hd = 6}\propto \frac{\partial E}{\partial {Z}^{hd = 6}}$

(19)

where m = 1, 2, 3... 10 (Neurons) and n = output layer

$\Delta {Z}_{m, n}^{hd = 6} = -constant\frac{\partial E}{\partial {Z}^{hd = 6}}$

(20)

Applying the chain rule on Eq (20) generates Eq (21) ^[79,80].

$\Delta {Z}_{m, n}^{hd = 6} = -constant\frac{\partial E}{\partial {ao}_{n}^{hd}}\times \frac{\partial {ao}_{n}^{hd}}{\partial {QhZ}_{n}^{hd}}\times \frac{\partial {QhZ}_{n}^{hd}}{\partial {Z}_{n}^{hd}}$

(21)

After simplification of Eq (21) it can be written as ^[80,82]:

$\Delta {Z}_{m, n}^{hd = 6} = constant\left(t{o}_{n}-{ao}_{n}\right)\times \left({ao}_{n}^{hd}\left(1-{ao}_{n}^{hd}\right)\times {ao}_{n}^{hd}\right)$

Through ${a}_{o}to{Z}_{6}$

$\Delta {Z}_{m, n}^{hd = 6} = constant\;{\rho }_{n}\;{ao}_{n}^{hd}$

(22)

The following method shows calculation to find out the proper weight change to hidden weight ^[80,82]. This is considered more complex because weighted links can become a reason for errors at every node.

Through ${Z}_{6}\;to\;{Z}_{1}\;or\;{Z}_{k}$

where k = 5, 4, 3, 2, 1

$\Delta {Z}_{m, k}^{hd}\propto -\left[\sum \limits_{n}\frac{\partial E}{\partial {ao}_{n}^{hd}}\times \frac{\partial {ao}_{n}^{hd}}{\partial {QhZ}_{n}^{hd}}\times \frac{\partial {Z}_{n}^{hd}}{\partial {Z}_{k}^{hd}}\right]\times \frac{\partial {ao}_{k}^{hd}}{\partial {QhZ}_{k}^{hd}}\times \frac{\partial {QhZ}_{k}^{hd}}{\partial {Z}_{m, k}^{hd}}$

$\Delta {Z}_{m, k}^{hd} = -\mathit{\boldsymbol{E}}\left[\sum \limits_{n}\frac{\partial E}{\partial {ao}_{n}^{hd}}\times \frac{\partial {ao}_{n}^{hd}}{\partial {QhZ}_{n}^{hd}}\times \frac{\partial {Z}_{k}^{hd}}{\partial {ao}_{k}^{hd}}\right]\times \frac{\partial {ao}_{k}^{hd}}{\partial {Z}_{k}^{hd}}\times \frac{\partial {QhZ}_{k}^{hd}}{\partial {Z}_{m, k}^{hd}}$

$\Delta {Z}_{m, k}^{hd} = \mathit{\boldsymbol{E}}\left[\sum \limits_{n}\begin{array}{c}\left({to}_{n}-{ao}_{n}^{hd}\right)\times {ao}_{k}^{hd}\left(1-{ao}_{n}^{hd}\right)\times {Z}_{k, n}\\ \end{array}\right]\times {ao}_{n}^{hd}\left(1-{ao}_{n}^{hd}\right)\times {(L}_{m, k})$

$\Delta {Z}_{m, k}^{hd} = \mathit{\boldsymbol{E}}\left[\sum \limits_{n}\begin{array}{c}\left({to}_{n}-{ao}_{n}^{hd}\right)\times {ao}_{k}^{hd}\left(1-{ao}_{n}^{hd}\right)\times {Z}_{k, n}\\ \end{array}\right]\times {ao}_{k}^{hd}\left(1-{ao}_{k}^{hd}\right)\times {(L}_{m, k})$

$\Delta {Z}_{m, k}^{hd} = \mathit{\boldsymbol{E}}\left[{\rho }_{k}{(L}_{m, k})\right]$

where,

${\rho }_{k} = \left[\sum \limits_{n}{\rho }_{n}\left({Z}_{k, n}^{hd}\right)\right]\times {ao}_{k}^{hd}\left(1-{ao}_{k}^{hd}\right)$

Modifying weight and bias among output & hidden layer is presented in Eq (23) where $\nabla {Z}_{m, n}$ represent the gradient descent w.r.t ${Z}_{m, n}$ ^{[77,79,80,81,82]}.

$\Delta {Z}_{m, n}^{hd = 6}\left(u\right) = \Delta {Z}_{m, n}^{hd = 6}\left(u\right)+\tau \nabla {Z}_{m, n}^{hd = 6}$

(23)

Modifying weight and bias among input & hidden layers is present in Eq (24) where $\nabla {Z}_{m, n}$ is gradient descent w.r.t ${Z}_{m, n}$ ^{[77,79,80,81,82]}.

$\Delta {Z}_{m, k}^{hd}\left(u\right) = \Delta {Z}_{m, k}^{hd}\left(u+1\right)+\tau \nabla {Z}_{m, n}^{hd}$

(24)

$\tau$ represent the key to finding local minima because it gives the step size for finding local minima.

4. Materials and methods

Materials and methods applied in this study are described below.

4.1. Dataset and feature selection

In this research, one dataset has been used for the experimentation. This dataset is accessible from the UCI Learning Repository. Apart from that, Cleveland data package was used for training, testing, and validating the prediction of breast cancer. Besides, the Wisconsin Breast Cancer Diagnostic (WBCD) dataset ^[66] is open to the public for analysis and research. This data set includes 32 human and biological characteristics. Further, the selection of features plays a vital role in the classification outcomes ^[67]. An increase in the performance and decrease in the time complexity of machine learning can be achieved through the appropriate selection of features ^[68]. Top 16 features have been selected using uni-variate and recursive feature selection strategies. Besides, data has been distributed among two classes (Positive and Negative). There are 355 healthy (Negative) samples and 214 diseased (Positive) samples. The selected features of data collection specifications are shown in the Table 1.

Table 1. Top 16 selected features.

Sr. No.	Attributes	Symbol	Type
1	Mean of the concave sections of the contour's severity	concavity_mean	Numeric
2	The average of distances between the center and the peripheral points	radius_mean	Numeric
3	The mean value for the severity of concave sections of the contour that is the worst or the greatest	concavity_worst	Numeric
4	Area	area_se	Numeric
5	Gray-scale standard deviation	Texture	Numeric
6	Worst symmetry	symmetry_worst	Numeric
7	arithmetic mean of the regional variance in radius lengths	smoothness_mean	Numeric
8	The standard error for the severity of concave contour segments	concavity_se	Numeric
9	The mean value that is the worst or the greatest for local variance in radius lengths	smoothness_worst	Numeric
10	The worst or biggest number for the mean of "coastline approximation"-1	fractal_dimension_worst	Numeric
11	The standard error for approximating the coastline-1	fractal_dimension_se	Numeric
12	Symmetry mean	symmetry_ mean	Numeric
13	arithmetic mean for "coastline approximation"-1	fractal_dimension_mean	Numeric
14	Symmetry se	symmetry_se	Numeric
15	standard inaccuracy in radius lengths due to local variation	smoothness_se	Numeric
16	standard inaccuracy for the standard deviation of gray-scale values	texture_se	Numeric

| Show Table

DownLoad: CSV

4.2. Performance evaluation metrics

An array of performance evaluation metrics has been developed to evaluate the performance of machine learning algorithms. Out of this array, frequently used metrics are accuracy (which evaluates accuracy rate (Acc)), specificity (Sp), precision (Pres), sensitivity (Sn), F-Measure, Negative Predicted Value (NPV), False Discovery Rate (FDR), False Positive Rate (FPR), False Negative Rate (FNR) and Mathew Co-relation Co-efficient (MCC), which assesses steadiness employing false positives, true negative, false negatives, and false positives values. These criteria are as follows:

Using Eq (25), accuracy can be calculated as given under ^[83]: -

$\mathrm{A}\mathrm{c}\mathrm{c}\mathrm{u}\mathrm{r}\mathrm{a}\mathrm{c}\mathrm{y}\;\left(\mathrm{A}\mathrm{c}\mathrm{c}\right) = \frac{T{r}_{P}+T{r}_{N}}{T{r}_{p}+T{r}_{N}+{Fa}_{p}+{Fa}_{N}}$

(25)

Using Eq (26), sensitivity/recall can be calculated as given under ^[83,84,85]: -

$\mathrm{S}\mathrm{e}\mathrm{n}\mathrm{s}\mathrm{i}\mathrm{t}\mathrm{i}\mathrm{v}\mathrm{i}\mathrm{t}\mathrm{y}/\mathrm{R}\mathrm{e}\mathrm{c}\mathrm{a}\mathrm{l}\mathrm{l}\;\left(\mathrm{S}\mathrm{n}\right) = \frac{T{r}_{P}}{T{r}_{P}+{Fa}_{N}}$

(26)

Using Eq (27), specificity can be calculated as given under ^[83,84]: -

$\mathrm{S}\mathrm{p}\mathrm{e}\mathrm{c}\mathrm{i}\mathrm{f}\mathrm{i}\mathrm{c}\mathrm{i}\mathrm{t}\mathrm{y}\;\left(\mathrm{S}\mathrm{p}\right) = \frac{T{r}_{N}}{T{r}_{N}+{Fa}_{P}}$

(27)

Using Eq (28), $\mathrm{P}\mathrm{r}\mathrm{e}\mathrm{c}\mathrm{i}\mathrm{s}\mathrm{i}\mathrm{o}\mathrm{n}\;\left(\mathrm{P}\mathrm{P}\mathrm{V}\right)$ can be calculated as given under ^[85]: -

$\mathrm{P}\mathrm{r}\mathrm{e}\mathrm{c}\mathrm{i}\mathrm{s}\mathrm{i}\mathrm{o}\mathrm{n}\;\left(\mathrm{P}\mathrm{P}\mathrm{V}\right)\; = \frac{T{r}_{P}}{T{r}_{P}+{Fa}_{P}}$

(28)

Using Eq (29), NPV can be calculated as given under ^[85]: -

$\mathrm{N}\mathrm{e}\mathrm{g}\mathrm{a}\mathrm{t}\mathrm{i}\mathrm{v}\mathrm{e}\;\mathrm{P}\mathrm{r}\mathrm{e}\mathrm{d}\mathrm{i}\mathrm{c}\mathrm{t}\mathrm{e}\mathrm{d}\;\mathrm{V}\mathrm{a}\mathrm{l}\mathrm{u}\mathrm{e}\;\left(\mathrm{N}\mathrm{P}\mathrm{V}\right)\; = \frac{T{r}_{N}}{T{r}_{N}+{Fa}_{N}}$

(29)

Using Eq (30), FPR can be calculated as given under ^[84,85]: -

$\mathrm{F}\mathrm{a}\mathrm{l}\mathrm{s}\mathrm{e}\;\mathrm{P}\mathrm{o}\mathrm{s}\mathrm{i}\mathrm{t}\mathrm{i}\mathrm{v}\mathrm{e}\;\mathrm{R}\mathrm{a}\mathrm{t}\mathrm{e}\;\left(\mathrm{F}\mathrm{P}\mathrm{R}\right) = \frac{{Fa}_{P}}{T{r}_{N}+{Fa}_{P}}$

(30)

Using Eq (31), FDR can be calculated as given under ^[83,84]: -

$\mathrm{F}\mathrm{a}\mathrm{l}\mathrm{s}\mathrm{e}\;\mathrm{D}\mathrm{i}\mathrm{s}\mathrm{c}\mathrm{o}\mathrm{v}\mathrm{e}\mathrm{r}\mathrm{y}\;\mathrm{R}\mathrm{a}\mathrm{t}\mathrm{e}\left(\mathrm{F}\mathrm{D}\mathrm{R}\right) = \frac{{Fa}_{P}}{T{r}_{P}+{Fa}_{P}}$

(31)

Using Eq (32), FNR can be calculated as given under ^[83,84]: -

$\mathrm{F}\mathrm{a}\mathrm{l}\mathrm{s}\mathrm{e}\;\mathrm{N}\mathrm{e}\mathrm{g}\mathrm{a}\mathrm{t}\mathrm{i}\mathrm{v}\mathrm{e}\;\mathrm{R}\mathrm{a}\mathrm{t}\mathrm{e}\;\left(\mathrm{F}\mathrm{N}\mathrm{R}\right) = \frac{{Fa}_{N}}{T{r}_{P}+{Fa}_{N}}$

(32)

Using Eq (33), F1-Score can be calculated as given under ^[83,84]: -

$\mathrm{F}1\; = \frac{2\times T{r}_{P}}{(2\times T{r}_{P}+{Fa}_{N}+{Fa}_{P})}$

(33)

Using Eq (34) MCC can be calculated as given under: -

$\mathrm{M}\mathrm{C}\mathrm{C}\; = \frac{\left(T{r}_{P}\times T{r}_{N}-{Fa}_{P}\times {Fa}_{N}\right)}{\mathrm{s}\mathrm{q}\mathrm{r}\mathrm{t}\left(\left(T{r}_{P}+{Fa}_{P}\right)\times \left(T{r}_{P}+{Fa}_{N}\right)\times \left(T{r}_{N}+{Fa}_{P}\right)\times \left(T{r}_{N}+{Fa}_{N}\right)\right)}$

(34)

4.3. Area Under the Curve (AUC) and ROC Curve

The receiver operating curve method is used to quantify and analyze the connection between a binary classifier's sensitivity and specificity. Sensitivity quantifies the percentage of properly classified positives; specificity quantifies the percentage of properly classified negatives ^[69,83,84].

AUC is the measurement of the area that is entirely covered by the ROC curve and it varies between 0 to 1. If a classification model produces a 100% accuracy rate then the AUC for that model comes out to be 1. In case, classification model gives 100% wrong classification results then the value of AUC calculates to be 0.

4.4. Split (Train-Test Ratio) and Cross-Validation

In split validation, data is divided into certain train and test ratios. In the proposed research, dataset has been divided into different sets of train-test ratios.

K-Fold cross-validation has also been employed to test the proposed model by plugging the different values in K. We have used K = 2 to K = 10 for K-Fold cross-validation to compute the average values but we have incorporated 10 folds cross-validation for the proposed model.

4.5. Mean Square Error (MSE), Root Mean Square Error (RMSE), and Absolute Mean Error Analysis (MAE)

The MSE analysis shows up to what extent the model has learned how much impact it is casting on the outcome. The machine's efficiency requires error minimization. The discrepancy between the intended and the actual output is measured as the mean square error. Besides, MSE, RMSR, MAE values for the training, testing, and validation phases were recorded against the different epochs.

5. Results and discussion

Varied experiments have been carried out to demonstrate the classification performance of the proposed model. Moreover, WBCD dataset has been used to train, test, and validate the model. Both the Split and K-Fold cross-validation techniques have been employed to validate the DEGDO base model. Besides, different train-test ratio groups have been set up and used like 50-50, 60-40, 70-30, and 80-20. Performance results produced with different train-test ratio groups are given in the Table 3. Apart from that, K-Fold cross-validation has also been carried out using the various values of K, K = 2 to K-10, for instance. Moreover, average performance produced by the different count of folds is shown in the Figure 3.

In the proposed model, multiple classifiers have been used to compare the performance of the proposed model DEGDO with other state-of-the-art methods. Moreover, multiple performance evaluation metrics have been used to check the proposed model for different classifiers like AUC-ROC, MSE, RMSE, MAE, Acc, Sp, PPV, Sn, F-Measure, NPV, FDR, FPR, FNR, and Matthew Co-relation Co-efficient (MCC). Table 2 shows the performance assessment of the Intelligent Breast Cancer Diagnostic System Empowered by Deep Extreme Gradient Descent Optimization with different train-test ratios. Besides, experiments are conducted on the selective features as well as on all the available features. Results produced with selective features are shown in the Figures 3 and 4. Apart from that, results with a complete set of attributes lowered the classification performance when it is compared to the performance carried out with the selective features. Results without selective features are shown in Table 2.

Table 2. Performance assessment of the intelligent breast cancer diagnostic system empowered by deep extreme gradient descent optimization with various train-test ratio.

Train-Test Ratio	Acc (%)	Sn (%)	Sp (%)	PPV (%)	NPV (%)	FPR	FDR	FNR	F1	MCC
50-50	96.66	95.28	97.48	95.73	97.21	0.0252	0.0427	0.0472	0.9551	0.9285
60-40	97.72	97.64	97.76	96.28	98.59	0.0224	0.0372	0.0236	0.9696	0.9513
70-30	98.95	98.58	99.16	98.58	99.16	0.0084	0.0142	0.0142	0.9858	0.9774
80-20	99.12	99.06	99.16	98.59	99.44	0.0084	0.0141	0.0094	0.9882	0.9812

| Show Table

DownLoad: CSV

Figure 3. Performance evaluation of intelligent breast cancer diagnostic system empowered by deep extreme gradient descent optimization, KNN, SVM, NB, ANN, and RF.

DownLoad: Full-Size Img PowerPoint

Figure 4. ROC curves for (4a) ANN, (4b) RF, (4c) SVM, (4d) NB, (4e) KNN and (4f) DEGDO.

The discrepancy between the intended and actual output is measured as the mean square error.

DownLoad: Full-Size Img PowerPoint

The accuracy, sensitivity, and specificity of the Intelligent Breast Cancer Diagnostic System Empowered by DEGDO were measured including classifiers like NB, SVM, K-NN, RF, and ANN. The performance results of the proposed Intelligent Breast Cancer Diagnostic System Empowered by Deep Extreme Gradient Descent Optimization are compared to the state-of-the-art methods like NB, SVM, K-NN, RF, and ANN (Figure 3).

It is found that the classification algorithm performed better with the deep extreme gradient descent optimization-based method. Besides, with the selective attributes for binary classification, the Intelligent Breast Cancer Diagnostic System empowered by DEGDO achieved a maximum accuracy of 98.73%. Apart from that, RF achieved the accuracy of 94.62% after the maximum accuracy of 98.73%. Moreover, Naive Bayes achieved an accuracy of 87.58% which sounds well but falls short of the target. SVM achieved just a 90.25% accuracy, which is better than the other algorithms like K-NN. Additionally, ANN and K-NN achieved accuracies of 85.29 and 83.81% respectively. Given these objective stats, we are justified to assert that the proposed model's accuracy got improved by using the selection of features technique. Moreover, Figure 4 illustrates a schematic comparison of the proposed DEGDO with various state-of-the-art machine learning techniques.

ROC curve generated by different classifiers used in this research is shown in the Figure 4. This figure vividly demonstrates that the proposed model rendered better results. Besides, AUC score for DEGDO, KNN, ANN, SVM, RF and NB are 0.989, 0.838, 0.867, 0.927, 0.948 and 0.876, respectively. X-axis represents the False Positive Rate (FPR) while Y-axis, the True Positive Rate (TPR).

Mean square error results for the training, testing, and validation phases measured by the number of epochs are displayed in Table 3. As the training iterations increase, a linear minimization in MSE, RMSE, and MAE is observed. Moreover, the lowest MSE is attained at 873 training epochs count for the proposed model which is recorded at 0.0569. Furthermore, in the training phase, the lowest obtained MSE was 0.0699 after 873 epochs.

Table 3. Mean square error (MSE), root mean square error (RMSE), and mean absolute error (MAE) of the intelligent breast cancer diagnostic system empowered by deep extreme gradient descent optimization with various train-test ratios.

Epochs Count	0	100	200	300	400	500	600	700	800	873
Training Phase
Mean Square Error (MSE)	1.665	0.2849	0.1724	0.1298	0.1012	0.0999	0.0842	0.0712	0.0645	0.0599
Root Mean Square Error (RMSE)	1.294	0.5464	0.4256	0.3652	0.3199	0.2994	0.2845	0.2689	0.2512	0.2502
Mean Absolute Error (MAE)	0.943	0.3815	0.3264	0.3054	0.2814	0.1985	0.1845	0.1542	0.1725	0.1311
Testing Phase
Mean Square Error (MSE)	1.754	0.3031	0.1852	0.1198	0.1426	0.1287	0.0984	0.0954	0.07425	0.0699
Root Mean Square Error (RMSE)	1.2854	0.5421	0.4157	0.3655	0.3451	0.3356	0.3158	0.2847	0.2485	0.0745
Mean Absolute Error (MAE)	0.948	0.3548	0.2954	0.2817	0.2465	0.2258	0.1956	0.1785	0.1688	0.1465
Validation Phase
MSE	1.841	0.3514	0.2785	0.1545	0.1348	0.1254	0.0998	0.0871	0.0785	0.0721
Root Mean Square Error (RMSE)	1.451	0.5812	0.5266	0.3863	0.3598	0.3421	0.3125	0.2706	0.2632	0.2415
Mean Absolute Error (MAE)	0.987	0.4487	0.4123	0.3458	0.2785	0.1859	0.1481	0.1399	0.1302	0.1298

| Show Table

DownLoad: CSV

In terms of performance, the proposed approach has been evaluated by comparing it to previously published experimental research models. It has been proven that the proposed approach is far more accurate than the ones published in the past. Table 4 gives a comparison of the accuracies between the proposed model and the other published works.

Table 4. Performance comparison of intelligent breast cancer diagnostic system empowered by deep extreme gradient descent optimization and literature models.

Reference Method	Model	Accuracy Results (%)^a
^[39]	Deep learning SMO, IBK	96.19, 95.90
^[43]	J48, Probabilistic NB	94.20, 82.60
^[44]	Deep learning ELU, Maxout, Tanh, ReLU Vote (NB + DT + SVM)	96.99, 96.56, 96.27, 96.55, 96.13
^[37]	K-SVM	97.38
^[48]	CSSFFS (10-FOLD), RBF Network	98.25, 93.60
^[70]	BIG-F	97.10
^[71]	DLA, EABA	97.20
^[72]	LDA & AE-DL	98.27
^[73]	DesneNet121 CNN	98.07
^[74]	EBL-RBFNN	98.40
^[75]	DL-CNN	95.00
^[76]	Boosting CN	98.27
	Proposed Model	98.73

| Show Table

DownLoad: CSV

6. Conclusions

This article demonstrated a marked rise in the breast cancer detection rates. A range of performance evaluation metrics like AUC-ROC, MSE, RMSE, MAE, Acc, Sp, PPV, Sn, F-Measure, NPV, FDR, FPR, FNR, and Matthew Co-relation Co-efficient (MCC) have been employed to evaluate the proposed model for different classifiers. The proposed model's accuracy, precision, sensitivity, and specificity are much better than many of the ones published in the literature, which, of course, makes this study more pronounced. Apart from that, both split and K-Fold cross-validation have been used to evaluate the performance. Besides, we have taken 10-folds cross-validation as a benchmark for results. The proposed model's accuracy, precision, sensitivity, and specificity rates came out to be 98.73, 99.48, 99.43 and 99.60% respectively. Additionally, the proposed model achieved a 0.989 AUC score. Numerous classifiers like ANN, KNN, NB, RF, and SVM were also applied to the same dataset but the proposed method outperformed all the afore-mentioned classifiers in terms of accuracy, precision, sensitivity, and specificity.

Paradoxically, some limitations plague the proposed model. Firstly, the model is trained and validated on a small dataset. Secondly, the diagnostic process consists of multiple stages ranging from the collection of relevant features to the medical laboratory test reports to feeding it to the proposed model in CSV format which, of course, delays the entire diagnosis.

In the future, this model can be exposed to multiple datasets like TCGA or NCBI GEO databases for better results. Moreover, the fusion technique can also be used to make the proposed model more reliable. Lastly, this model can be synergized with some other feature selections methods to boost its performance.

Acknowledgments

This research received no external funding.

Conflict of interest

The authors declare there is no conflict of interest.

References

[1]	G. Litjens, T. Kooi, B. E. Bejnordi, A. A. A. Setio, F. Ciompi, M. Ghafoorian, et al., A survey on deep learning in medical image analysis, Med. Image Anal., 42 (2017), 60–88. https://doi.org/10.1016/j.media.2017.07.005 doi: 10.1016/j.media.2017.07.005
[2]	A. Sáez, C. Serrano, B. Acha, Model-based classification methods of global patterns in dermoscopic images, IEEE Trans. Med. Imaging, 33 (2014), 1137–1147. https://doi.org/10.1109/TMI.2014.2305769 doi: 10.1109/TMI.2014.2305769
[3]	Y. Zhao, H. Li, S. Wan, A. Sekuboyina, X. Hu, G. Tetteh, et al., Knowledge-aided convolutional neural network for small organ segmentation, IEEE J. Biomed. Health Inf., 23 (2019), 1363–1373. https://doi.org/10.1109/JBHI.2019.2891526 doi: 10.1109/JBHI.2019.2891526
[4]	X. Fang, P. Yan, Multi-organ segmentation over partially labeled datasets with multi-scale feature abstraction, IEEE Trans. Med. Imaging, 39 (2020), 3619–3629. https://doi.org/10.1109/TMI.2020.3001036 doi: 10.1109/TMI.2020.3001036
[5]	Y. Fang, C. Chen, Y. Yuan, K. Y. Tong, Selective feature aggregation network with area-boundary constraints for polyp segmentation, in International Conference on Medical Image Computing and Computer Assisted Intervention, (2019), 302–310. https://doi.org/10.1007/978-3-030-32239-7_34
[6]	F. Milletari, N. Navab, S. A. Ahmadi, V-net: Fully convolutional neural networks for volumetric medical image segmentation, in 2016 Fourth International Conference on 3D Vision (3DV), (2016), 565–571. https://doi.org/10.1109/3DV.2016.79
[7]	T. Jin, X. Yang, Monotonicity theorem for the uncertain fractional differential equation and application to uncertain financial market, Math. Comput. Simul., 190 (2021), 203–221. https://doi.org/10.1016/j.matcom.2021.05.018 doi: 10.1016/j.matcom.2021.05.018
[8]	T. Jin, X. Yang, H. Xia, H. U. I. Ding, Reliability index and option pricing formulas of the first-hitting time model based on the uncertain fractional-order differential equation with Caputo type, Fractals., 29 (2021), 21500122. https://doi.org/10.1142/S0218348X21500122 doi: 10.1142/S0218348X21500122
[9]	C. Tian, T. Jin, X. Yang, Q. Liu, Reliability analysis of the uncertain heat conduction model, Comput. Math. Appl., 119 (2022), 131–140. https://doi.org/10.1016/j.camwa.2022.05.033.5 doi: 10.1016/j.camwa.2022.05.033.5
[10]	O. Ronneberger, P. Fischer, T. Brox, U-Net: Convolutional networks for biomedical image segmentation, in International Conference on Medical Image Computing and Computer-assisted Intervention, (2015), 234–241. https://doi.org/10.1007/978-3-319-24574-4_28
[11]	Ö. Çiçek, A. Abdulkadir, S. S. Lienkamp, T. Brox, O. Ronneberger, 3D U-Net: Learning dense volumetric segmentation from sparse annotation, in International Conference on Medical Image Computing and Computer-assisted Intervention, (2016), 424–432. https://doi.org/10.1007/978-3-319-46723-8_49
[12]	S. Zhang, H. Fu, Y. Yan, Y. Zhang, Q. Wu, M. Yang, et al., Attention guided network for retinal image segmentation, in Medical Image Computing and Computer Assisted Intervention, (2019), 797–805. https://doi.org/10.1007/978-3-030-32239-7_88
[13]	W. Bo, Y. Lei, S. Tian, T. Wang, Y. Liu, P. Patel, et al., Deeply supervised 3D fully convolutional networks with group dilated convolution for automatic MRI prostate segmentation, Med. Phys., 46 (2019), 1707–1718. https://doi.org/10.1002/mp.13416 doi: 10.1002/mp.13416
[14]	J. Chen, X. Wang, Z. Guo, X. Zhang, J. Sun, Dynamic region-aware convolution, in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), (2021), 8064–8073.
[15]	A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, et al., Attention is all you need, in Advances in Neural Information Processing Systems, (2017), 1–11.
[16]	M. Raghu, T. Unterthiner, S. Kornblith, C. Zhang, A. Dosovitskiy, Do vision transformers see like convolutional neural networks, preprint, arXiv: 2108.08810.
[17]	X. Zhai, A. Kolesnikov, N. Houlsby, L. Beyer, Scaling vision transformers, preprint, arXiv: 2106.04560.
[18]	B. Cheng, A. G. Schwing, A. Kirillov, Per-pixel classification is not all you need for semantic segmentation, preprint, arXiv: 2107.06278.
[19]	S. Zheng, J. Lu, H. Zhao, X. Zhu, Z. Luo, Y. Wang, et al., Rethinking semantic segmentation from a sequence-to-sequence perspective with transformers, in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), (2021), 6881–6890.
[20]	J. Chen, Y. Lu, Q. Yu, X. Luo, E. Adeli, Y. Wang, et al., TransUNet: Transformers make strong encoders for medical image segmentation, preprint, arXiv: 2102.04306.
[21]	H. Cao, Y. Wang, J. Chen, D. Jiang, X. Zhang, Q. Tian, et al., Swin-Unet: Unet-like pure transformer for medical image segmentation, preprint, arXiv: 2105.05537.
[22]	A. Dosovitskiy, L. Beyer, A. Kolesnikov, D. Weissenborn, X. Zhai, T. Unterthiner, et al., An image is worth 16x16 words: Transformers for image recognition at scale, preprint, arXiv: 2010.11929.
[23]	Z. Liu, Y. Lin, Y. Cao, H. Hu, Y. Wei, Z. Zhang, et al., Swin transformer: Hierarchical vision transformer using shifted windows, in Proceedings of the IEEE/CVF International Conference on Computer Vision, (2021), 10012–10022.
[24]	J. M. J. Valanarasu, P. Oza, I. Hacihaliloglu, V. M. Patel, Medical transformer: Gated axial-attention for medical image segmentation, in International Conference on Medical Image Computing and Computer-Assisted Intervention, (2021), 36–46. https://doi.org/10.1007/978-3-030-87193-2_4
[25]	Y. Zhang, H. Liu, Q. Hu, Transfuse: Fusing transformers and cnns for medical image segmentation, in International Conference on Medical Image Computing and Computer-Assisted Intervention, (2021), 14–24. https://doi.org/10.1007/978-3-030-87193-2_2
[26]	W. Wang, C. Chen, M. Ding, H. Yu, S. Zha, J. Li, Transbts: Multimodal brain tumor segmentation using transformer, in International Conference on Medical Image Computing and Computer-Assisted Intervention, (2021), 109–119. https://doi.org/10.1007/978-3-030-87193-2_11
[27]	Y. Xie, J. Zhang, C. Shen, Y. Xia, Cotr: Efficiently bridging cnn and transformer for 3d medical image segmentation, in International Conference on Medical Image Computing and Computer-Assisted Intervention, (2021), 171–180. https://doi.org/10.1007/978-3-030-87199-4_16
[28]	H. Wang, P. Cao, J. Wang, O. R. Zaiane, UCTransNet: Rethinking the skip connections in U-Net from a channel-wise perspective with transformer, preprint, arXiv: 2109.04335.
[29]	H. Y. Zhou, J. Guo, Y. Zhang, L. Yu, L. Wang, Y. Yu, nnformer: Interleaved transformer for volumetric segmentation, preprint, arXiv: 2109.03201.
[30]	B. Landman, Z. Xu, J. E. Igelsias, M. Styner, T. Langerak, A. Klein, Miccai multi-atlas labeling beyond the cranial vault–workshop and challenge, In Proc. MICCAI: Multi-Atlas Labeling Beyond Cranial Vault-Workshop Challenge, 5 (2015), 12.
[31]	O. Bernard, A. Lalande, C. Zotti, F. Cervenansky, X. Yang, P. A. Heng, et al., Deep learning techniques for automatic mri cardiac multi-structures segmentation and diagnosis: Is the problem solved, IEEE Trans. Med. Imaging, 37 (2018), 2514–2525. https://doi.org/10.1109/TMI.2018.2837502 doi: 10.1109/TMI.2018.2837502
[32]	Z. Zhou, M. M. R. Siddiquee, N. Tajbakhsh, J. Liang, Unet++: A nested u-net architecture for medical image segmentation, in Deep Learning in Medical Image Analysis and Multimodal Learning for Clinical Decision Support, (2018), 3–11. https://doi.org/10.1007/978-3-030-00889-5_1
[33]	H. Huang, L. Lin, R. Tong, H. Hu, Q. Zhang, Y. Iwamoto, et al., Unet 3+: A full-scale connected unet for medical image segmentation, in IEEE International Conference on Acoustics, Speech and Signal Processing, (2020), 1055–1059. https://doi.org/10.1109/ICASSP40776.2020.9053405
[34]	Z. Gu, J. Cheng, H. Fu, K. Zhou, H. Hao, Y. Zhao, et al., CE-Net: Context encoder network for 2D medical image segmentation, IEEE Trans. Med. Imaging, 38 (2019), 2281–2292. https://doi.org/10.1109/TMI.2019.2903562 doi: 10.1109/TMI.2019.2903562
[35]	I. Fabian, P. F. Jaeger, S. A. A. Kohl, J. Petersen, K. H. Maier-Hein, nnU-Net: A self-configuring method for deep learning-based biomedical image segmentation, Nat. Methods, 18 (2021), 203–211. https://doi.org/10.1038/s41592-020-01008-z doi: 10.1038/s41592-020-01008-z
[36]	A. Hatamizadeh, Y. Tang, V. Nath, D. Yang, A. Myronenko, B. Landman, et al., Unetr: Transformers for 3d medical image segmentation, in Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, (2022), 574–584.
[37]	O. Ozan, J. Schlemper, L. L. Folgoc, M. Lee, M. Heinrich, K. Misawa, et al., Attention u-net: Learning where to look for the pancreas, preprint, arXiv: 1804.03999.
[38]	R. Gu, G. Wang, T. Song, R. Huang, M. Aertsen, J. Deprest, et al., CA-Net: Comprehensive attention convolutional neural networks for explainable medical image segmentation, IEEE Trans. Med. Imaging, 40 (2020), 699–711. https://doi.org/10.1109/TMI.2020.3035253 doi: 10.1109/TMI.2020.3035253
[39]	S. Woo, J. Park, J. Y. Lee, I. S. Kweon, Cbam: Convolutional block attention module, in Proceedings of the European Conference on Computer Vision (ECCV), (2018), 3–19.
[40]	Y. Gao, M. Zhou, D. N. Metaxas, UTNet: A hybrid transformer architecture for medical image segmentation, in International Conference on Medical Image Computing and Computer-Assisted Intervention, (2021), 61–71. https://doi.org/10.1007/978-3-030-87199-4_6

Reader Comments

Your name:*

Email:*
© 2023 the Author(s), licensee AIMS Press. This is an open access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/4.0)