An ensemble-acute lymphoblastic leukemia model for acute lymphoblastic leukemia image classification

1.   Introduction

The recognition of medical diseases through image analysis relies heavily on the rapid advancement of artificial intelligence and machine learning technologies. Artificial intelligence, designed to replicate human cognitive functions, is ushering in a profound transformation in healthcare. This transformation is propelled by the growing abundance of healthcare data and the swift evolution of analytical capabilities, effectively reshaping conventional healthcare into an era of smart healthcare ^[1]. In recent years, the convergence of big data and deep learning technology has propelled medical disease image recognition to the forefront of research in the medical field. Historically, physicians heavily leaned on their experience and professional expertise for disease diagnosis and treatment. Nevertheless, given the immense diversity of diseases and their intricate presentations, relying solely on human judgment, doctors are susceptible to subjective biases and the risk of misdiagnosis. Furthermore, healthcare professionals must contend with a substantial volume of cases and a vast array of imaging data, resulting in an overwhelming workload. Consequently, there is a growing demand for swift and precise automated systems to support physicians in their diagnostic endeavors. Leveraging machine learning and deep learning techniques, these systems can autonomously scrutinize and detect anomalies in medical images, providing valuable aid to doctors in the process of diagnosing and treating diseases. Currently, medical disease image recognition technology has found extensive applications, with research conducted in various domains, such as brain tumors ^[2], diabetic retinopathy ^[3], lung diseases ^[4], and more. A meticulously crafted convolutional neural network (CNN) model not only offers valuable reference for medical professionals but also has the capability to automatically outline or pinpoint the specific areas of interest for observation through computer-assisted detection. This capability aids physicians in promptly interpreting medical images and taking the appropriate actions based on the findings.

One of the most prevalent form of cancer in children is acute lymphoblastic leukemia (ALL) ^[5,6,7], a condition that impacts white blood cells (WBCs). In ALL, patients typically exhibit an overabundance of immature WBCs within the bone marrow. These immature WBCs can potentially disseminate to other organs, including the spleen, liver, lymph nodes, central nervous system, and testicles. It is noteworthy that a substantial proportion of ALL cases, approximately 55% of the global total, are concentrated in the Asia-Pacific region ^[7].

The CNN stands as one of the prominent techniques in the realm of deep learning. It excels in training and categorizing extensive volumes of intricate data through its model. CNNs find application in a diverse range of domains, encompassing image recognition ^[8,9,10], and voice recognition ^{[11,12,13,14]}. The architecture of a CNN typically comprises one or more convolutional layers, a pooling layer, a rectified linear units layer (ReLU layer), a fully connected layer, and various hyperparameters, among other components.

Jawahar et al. ^[15] introduced a cutting-edge deep neural network model called ALNett, which leverages deep convolutions featuring varying dilation rates for the classification of microscopic leukocyte images. The model's performance was assessed against several pre-trained models, including VGG16, ResNet-50, GoogleNet, and AlexNet. The experimental findings revealed that the proposed ALNett model achieved an impressive classification accuracy of 91.13%, all while maintaining low computational complexity. Mondal et al. ^[7] also contributed to the field by deploying a deep CNN to identify ALL. Their study delved into the development of weighted ensembles of deep CNN with the aim of recommending improved classifiers for ALL diagnosis.

Das and Meher ^[16] introduced an effective deep CNN framework designed to enhance the accuracy of ALL detection. This framework incorporates innovative components, including depth-wise separable convolution, a linear bottleneck architecture, reverse residual structures, and skip connections. As a result, it not only achieves greater speed but also gains popularity in the field. A key feature of this method is the introduction of a novel probability-based weighting factor, skillfully combining the strengths of MobilenetV2 and ResNet18 while preserving the advantages of both approaches. Anilkumar et al. ^[17] have effectively classified ALL using deep CNN s, achieving an impressive accuracy rate of 94.12%. In a related study, Duggal et al. ^[18] introduced an innovative approach by incorporating a stain de-convolutional layer at the front of a CNN. They utilized this model to discriminate between malignant immature white blood cells and benign immature white blood cells, thereby enabling the detection of ALL. All three of these studies have contributed significantly to the field by proposing deep neural network models that leverage deep convolution and feature extraction techniques to successfully classify and identify ALL images.

In 2022, Ghaderzadeh et al. ^[19] assembled a dataset for ALL and conducted a comprehensive study. They employed ten different CNN models, including EfficientNet, MobileNetV3, VGG-19, Xception, InceptionV3, ResNet50V2, VGG-16, NASNetLarge, InceptionResNetV2, and DenseNet201, for various feature extraction tasks. Among these ten models, DenseNet201 emerged as the top-performing model, achieving exceptional classification results with an accuracy of 99.85%, sensitivity of 99.52%, and specificity of 99.89%. Panthakkan et al. ^[20] devised an innovative approach by integrating the Xception and ResNet50V2 models, ultimately creating a 4 × 4 × 2048 feature map in the final layer of feature extraction. They then concatenated and integrated the features extracted from Xception and ResNet50V2 into a 1 × 1 convolutional layer. This concatenated feature representation was subsequently combined with a classifier to form a unified deep network. The purpose of this proposed model is to provide valuable assistance to dermatologists and clinicians in the early detection of skin cancer within the clinical process.

Elashiri et al. ^[21] conducted a study in which they integrated features extracted by three different models: ResNet50, VGG16, and DeepLabv3. These integrated features were then passed to the feature transformation stage, employing weighted feature extraction facilitated by a hybrid squirrel butterfly search optimization technique. The transformed features were subsequently input to a modified long short-term memory (LSTM) for the final classification process. Remarkably, this approach yielded the highest accuracy rate, reaching an impressive 93%. Silva et al. ^[22] introduced an EnsembleDVX model, which combines the capabilities of DenseNet169, VGG16, and Xception in the context of COVID-19 detection. The ensemble EnsembleDVX model demonstrates superior performance when compared to individual single models. Additionally, Aurna et al. ^[23] introduced a two-phase ensemble model designed for the accurate and automated detection of brain tumors. This ensemble model effectively combines features extracted from five individual models and, through the use of principal component analysis, selects the most significant combinations of these features. The results obtained from the ensemble model surpass the performance of the single models, demonstrating its efficacy in precise brain tumor detection. Su et al. ^[24] utilized a combination of deep learning and ensemble learning techniques to tackle the challenging task of glioma grading, specifically focusing on the classification between glioma grades Ⅱ and Ⅲ. They constructed an ensemble model called LR-14, which is based on logistic regression and integrates the predictions from 14 different deep learning classifiers. This ensemble LR-14 model emerged as the top-performer, achieving the highest level of performance for this critical classification task.

As indicated by the examples mentioned, ensemble models have proven their effectiveness in the realm of medical classification. The outcomes from these integrated models have generally outperformed those from single models, underlining the value of combining different approaches for improved results. However, it is worth noting that many ensemble models tend to be complex with a significant number of parameters, which can lead to longer training times and increased computational demands. Hence, our primary objective of this study is to introduce a streamlined ensemble model with fewer parameters. The aim is to reduce training time while achieving enhanced stability and accuracy compared to single models. This approach seeks to strike a balance between computational efficiency and performance in the context of medical classification, potentially offering a more practical and accessible solution.

2.   Materials and methods

2.1.   ALL challenge dataset of ISBI 2019 (C-NMC 2019)

ALL accounts for around 25% of all pediatric cancers ^[25]. Identifying immature leukemic blasts from normal cells under a microscope is a challenging task because, morphologically, the images of these two cell types can be quite similar. The dataset used in this study was from the ALL Challenge dataset of ISBI 2019 (C-NMC 2019) ^[26]. The whole training set comprises a total of 73 subjects (10661 images), including 46 with ALL (7272 images) and 26 with Normal conditions (3,389 images). The training set is split into three folds including fold0, fold1, and fold2 (Table 1). Among the three folds, fold0, containing 2397 ALL images and 1,130 normal images, is split into training, validation, and test sets in this study. Table 2 provides an overview of example images and the number of training, test, and validation sets used in this investigation.

2.2.   Methodological framework

The proposed ensemble-ALL model is outlined and introduced as follows:

1) Data Splitting (Step 1): The initial dataset is divided into distinct subsets, including training, validation, and test sets.

2) Individual Model Training (Step 2): In additional to the well-known models, including InceptionV3, EfficientNetB4, ResNet50, and AlexNet, we have selected three models, ALL-CNN ^[27], CONV_POOL-CNN ^[27], and Network in Network (NIN-CNN) ^[28], with simple architecture in this study. The above seven models are independently applied to the C-NMC 2019 dataset.

3) Model Selection and Enhancement (Step 3): The top-performing four individual models are meticulously selected and integrated with the squeeze-and-excitation (SE) module to evaluate whether this fusion enhances performance.

4) Ensemble Model Formation (Step 4): The two most promising SE-embedded models are combined to create the proposed ensemble-ALL model.

5) Bayesian Optimization (Step 5): Bayesian optimization techniques are harnessed within the ensemble-ALL model to enhance its performance.

6) Cross-Validation (Step 6): A five-fold cross-validation strategy is employed to rigorously assess the model's performance.

7) Performance Evaluation (Step 7): Performance metrics including accuracy, precision, recall, F1-score, and kappa are recorded and evaluated for all models.

This structured approach shown in Figure 1 ensures the creation of a robust and optimized ensemble-ALL model for the task of medical image classification.

2.3.   Squeeze-and-excitation

Squeeze-and-Excitation Networks, known as SE Networks, secured the championship in the ImageNet 2017 classification competition. The SE module's concept is straightforward, easy to implement, and can be seamlessly integrated into existing network model frameworks ^[29]. Consequently, SE modules have been incorporated into models in numerous academic works ^{[29,30,31,32,33,34]}. It's evident that the addition of SE attention modules consistently leads to improved accuracy results in the studies, outperforming the original models. For instance, you can find examples such as SE-MobileNet ^[30], SE-ResNext ^[31], SE-ResNet-18 ^[32], and SE-DenseNet ^[33], all of which have demonstrated enhanced performance thanks to the inclusion of SE attention modules.

The decision to employ the SE module to enhance InceptionV3 in this study is rooted in the observation that many existing studies primarily focus on the SE-ResNet model, with only a limited number exploring SE-InceptionV3. Additionally, deeper network structures, such as the one in InceptionV3, are susceptible to feature distortion. To mitigate this, many studies have introduced attention modules into deeper network models ^[30,33,34]. Consequently, in this study, the SE module is introduced after the global average pooling layer of InceptionV3 to address these considerations. Figures 2–5. provide a visual representation of the architectures of the SE-InceptionV3, SE-CONV_POOL-CNN, SE-ALL-CNN, and SE-NIN-CNN models employed in the study.

2.4.   Region of interest

To streamline the feature extraction process and remove extraneous elements, the study employs a region of interest (ROI) feature extraction method within image processing. Specifically, the black background has a different proportion in each image, we can directly outline the ROI area through the pixel matrix [100:350,200:400] from the images. Figure 6(a). is an example of original image. The proportion of black background is greatly reduced in Figure 6(b). It's important to note that, given the variability in feature sizes in each picture, the elimination of the black background may not be entirely comprehensive in all cases.

2.5.   Bayesian optimization

Bayesian optimization is a frequently employed technique for optimizing highly complex or non-differentiable functions, often referred to as black-box functions ^[35]. Bayesian optimization algorithms have extensive application in areas such as automated hyperparameter tuning, machine learning, and deep learning. In the context of this study, Bayesian optimization is utilized to determine appropriate hyperparameters, including the number of epochs (ranging from 10 to 100), batch size (ranging from 8 to 32), and learning rate (ranging from 1 × 10⁻⁵ to 1 × 10⁻²). This approach enables the automatic selection of optimal hyperparameters for the task at hand. Due to the limitation of computer equipment, the execution of hyperparameter optimization is time consuming. It is not applied foron the training for seven individual CNN models in Section 2.2 Step 2, and is solely used for training the ensemble model in Section 2.2 Step 4 in this study.

2.6.   Five-fold cross-validation

The initial step involves partitioning all the available data into two distinct sets: a training set and a test set. Subsequently, the training set is further divided into five approximately equal and non-overlapping validation folds. During each iteration of the process, one of these folds is designated as the validation set, while the remaining folds are combined to form the training set. To illustrate, in Fold1, the training set is composed of the validation sets from Folds 2–5. The model's performance is assessed using the validation set (Val) in each iteration, and this process is repeated until every fold has been used as the validation set. This method, known as five-fold cross-validation, is visually depicted in Figure 7. to provide a clear overview of the process.

2.7.   Computational environment

The equipment used in the experiment is an Intel(R) Core(TM) i7-10700 3.80GHz 32GB CPU, NVIDIA GeForce RTX 3060 GPU. The whole experiment process is performed using Python 3.8 [Python Software Foundation, Fredericksburg, Virginia, USA], which contains Keras 2.6 and Tensorflow GPU 2.6.0.

3.   Results

3.1.   Performance of individual models

The results of individual models are recorded in Table 3. The top four highest accuracies are 90.78, 90.35, 89.21, and 84.82%, which are from CONV_POOL-CNN, ALL-CNN, NIN-CNN, and InceptionV3 models. The four best-performing models are chosen for further enhancement. These models are combined with the SE module to create four modified models, denoted as SE-InceptionV3, SE-CONV_POOL-CNN, SE-ALL-CNN, and SE-NIN-CNN. This integration with the SE module is intended to improve their performance and effectiveness.

3.2.   Performance of selected SE models

Performance of SE-InceptionV3, SE-CONV_POOL-CNN, SE-ALL-CNN, and SE-NIN-CNN models are listed in Table 4. Compared with Table 3, The SE module has a significant improvement on InceptionV3 model, but has less improvement on the other three models. The top two accuracies, precisions, recalls, F1-scores and kappas are from SE-InceptionV3 and SE-CONV_POOL-CNN models. The accuracy 84.82% and kappa 66.37% from InceptionV3 model were greatly increased to 95.03 and 88.69% for SE-InceptionV3 model. The accuracy 90.78% from CONV_POOL-CNN model was slightly increased to 91.34% for SE- CONV_POOL-CNN model.

3.3.   Performance of the proposed ensemble-ALL model

The two best-performing SE models, SE-InceptionV3 and SE-CONV_POOL-CNN, are carefully chosen and integrated into the proposed ensemble-ALL model, as depicted in Figure 8. In this architecture, the convolutional layer replaces the traditional fully connected layer to serve as the output of a single model. Subsequently, these outputs are merged and fed into the fully connected layer, effectively increasing the number of features. Following this, three fully connected layers are employed, and an SE module is introduced to preserve critical features. Finally, the output of the fully connected layer serves as the output of the ensemble model. This approach optimizes the model's architecture for the task of medical image classification.

In the study, the Bayesian optimization method is employed to identify the optimal hyperparameter combinations. Specifically, the chosen parameter values are as follows: Epoch = 50, Batch size = 15, and learning rate at 1.11 × 10⁻⁰⁵. The results of the five-fold cross-validation for the SE-InceptionV3, SE-CONV_POOL-CNN, and the proposed ensemble-ALL models are presented in Tables 5–7, respectively. These tables likely provide valuable insights into the performance and efficacy of these models in the context of medical image classification. The average of accuracy, precision, recall, F1-score and kappa from SE-InceptionV3 were 94.893, 95.054, 94.894, 94.877, and 88.229%, respectively. The accuracy, precision, recall, F1-score and kappa from SE-CONV_POOL-CNN were 89.985, 90.178, 89.986, 89.987, and 77.066%, respectively. The best accuracy, precision, recall, F1-score and kappa come from the proposed ensemble-ALL at 96.255, 96.259, 96.255, 96.246, and 91.362%, respectively. Figure 9 illustrates the performance of the three models, and it's clear that the proposed ensemble-ALL model outperforms the other two SE models across all major metrics. This highlights the effectiveness of the ensemble-ALL model in achieving superior results in the context of medical image classification.

The training curves for accuracy, loss, and the confusion matrix for SE-InceptionV3, SE-CONV_POOL-CNN, and the proposed ensemble-ALL models are presented in Figures 10–12. A comparison of these curves reveals that the proposed ensemble-ALL model exhibits higher accuracy, more stable and converged loss, and smoother training patterns in comparison to the two SE models. These observations further reinforce the superior performance of the ensemble-ALL model in the context of medical image classification.

4.   Discussion

4.1.   Selection process of the proposed ensemble-ALL model

The top-performing four out of seven individual models are selected and integrated with the SE module. Then, the two most promising SE-embedded models are combined to create the proposed ensemble-ALL model in this study. Definitely, the proposed model is not the only solution. We have completed many different selections among models in the experimental processes, and finally the proposed model stood out all combinations. Some of the results are presented in Table 8 as follows:

In addition, due to limitations of computer equipment, the author also needs to consider the amount of computer calculations. This is part of the reason that only two models are selected in the ensemble stage.

4.2.   Comparison with SOTA

Table 9 offers a comparative analysis of the results achieved by the proposed ensemble-ALL model with those obtained in related studies that used the same dataset. The comparison includes findings from Duggal et al. ^[18] and Mondal et al. ^[7], where some ensemble models were employed. Notably, the results highlight that the proposed ensemble-ALL model outperforms all other models considered in this analysis on the C-NMC 2019 dataset. This underscores the model's effectiveness and its capacity to yield superior performance in the domain of medical image classification.

5.   Conclusions

The ensemble-ALL model developed in this study represents a significant advancement in the field of medical image classification, particularly in the context of ALL. This ensemble model is designed to improve the early diagnosis and classification of ALL, enhancing the efficiency of medical professionals in the diagnostic and treatment process. When compared to related state-of-the-art studies, the proposed ensemble-ALL model stands out by achieving remarkable performance metrics, including accuracy, precision, recall, F1-score, and kappa, with values of 96.26, 96.26, 96.26, 96.25, and 91.36%, respectively. These results underscore the efficacy of the ensemble model in the accurate classification of ALL images. Key highlights and components of the ensemble-ALL model include:

1). Data Splitting: The original dataset is divided into training, validation, and test sets.

2). Individual Models: Several deep learning models, including InceptionV3, EfficientNetB4, ResNet50, ALL-CNN, CONV_POOL-CNN, Network in Network (NIN-CNN), and AlexNet, are individually applied to the C-NMC 2019 dataset.

3). Model Selection and Enhancement: The top-performing individual models are selected and enhanced with the SE module to boost performance.

4). Ensemble Formation: The two best-performing SE models, SE-InceptionV3 and SE-CONV_POOL-CNN, are combined to create the ensemble-ALL model.

5). Feature Extraction and Optimization: Feature extraction from ROI and Bayesian optimization techniques are utilized within the ensemble-ALL model to enhance its performance.

6). Cross-Validation: A five-fold cross-validation strategy is employed to rigorously assess the model's performance.

7). Performance Metrics: Performance indices, including accuracy, precision, recall, F1-score, and kappa, are recorded for evaluation.

The ensemble-ALL model, as demonstrated in the study, has shown remarkable results, outperforming existing state-of-the-art models in image classification for ALL.

While the proposed ensemble-ALL model demonstrates notable performance, there are several limitations to the study that should be acknowledged:

1) Limited datasets for ALL: Public datasets specifically tailored for ALL are relatively scarce compared to datasets for other diseases. Consequently, the ensemble-ALL model in this study is exclusively applied to a single ALL dataset, which may not capture the full spectrum of variability that exists across different datasets.

2) Hardware limitations: The study is constrained by hardware limitations, particularly in terms of graphics card capabilities and memory. Large datasets, extensive training times, and models with substantial parameters (e.g., EfficientNetB5 to EfficientNetB7, ConvNeXt, etc.) can strain hardware resources. This limitation may hinder the exploration of larger and more complex models.

3) Model parameter size vs. accuracy: Our focus of this study centers on optimizing model accuracy. However, the trade-off between model accuracy and the size of model parameters is not deeply explored. Future research could consider achieving high accuracy while minimizing the size of model parameters, which is crucial for real-world applications where resource constraints may be a concern.

These limitations provide valuable insights for future research endeavors, and they underscore the need for ongoing exploration and development in the field of medical image classification.

Use of AI tools declaration

The authors declare that they have not used Artificial Intelligence (AI) tools in the creation of this article.

Acknowledgments

The authors gratefully acknowledge the financial support of the Ministry of Science and Technology of Taiwan, R.O.C., through the grant MOST 111-2221-E-167-007-MY3.

Conflict of interest

The authors declare that there are no conflicts of interest.

Data and code availability

The data used to support the findings of this study are available from https://doi.org/10.7937/tcia.2019.dc64i46r. The code to generate the results and figures is available of the corresponding author upon request.