Lung adenocarcinoma identification based on hybrid feature selections and attentional convolutional neural networks

1.   Introduction

Global cancer statistics for 2020 projected approximately 19.29 million new cases and 9.96 million deaths ^[1]. Among cancers, lung cancer claimed the highest mortality rate worldwide, contributing to 18% of total cancer-related deaths. Over recent decades, lung adenocarcinoma, a prevalent form of non-small cell lung cancer, has exhibited an upward trajectory in both incidence and mortality across many nations ^[2]. Studies have shown that lung adenocarcinoma detected and diagnosed at an early stage can be treated by resection, and patients generally have a survival rate of about 90% within 5 years or even long-term survival ^[3]. However, sometimes hematogenous metastasis of lung adenocarcinoma occurs at an early stage, which requires combined chemotherapy and targeted drug therapy, and the survival period is shortened ^[4]. Therefore, early prevention of lung adenocarcinoma and the development of screening tools for this disease are crucial.

Clinical medicine and biomedical research have made tremendous progress with the development of gene chips and high-throughput sequencing technologies ^[5]. These technologies have revolutionized our understanding of the genome, proteome, and transcriptome, providing unprecedented opportunities for disease diagnosis, prevention, and treatment, and playing a key role in cancer research ^[6]. Advances in tumor genomics have helped scientists understand mutations in the genes that drive cancer, leading to the development of targeted therapies for specific mutations ^[7]. Common screening methods for lung adenocarcinoma include chest X-rays, blood biomarker tests, bronchoscopy, computed tomography (CT) scans, and genetic analysis. X-rays and blood biomarker tests are less effective in early screening, while bronchoscopy requires consideration of the patient's physical condition and is inconvenient to perform. On the other hand, CT scans and genetic analysis are widely employed in early detection, and with technological advances, they are also beginning to be integrated with artificial intelligence. Chen et al. developed a self-distillation training multitasking dense attention network based on CT scans ^[8]. They also proposed a deep learning-based protocol for CT image analysis that accurately identifies lung adenocarcinoma categories and high-risk tumor regions ^[9,10]. In a separate study, Gao ^[11] et al. utilized the particle swarm algorithm and the artificial bee colony algorithm to enhance support vector machine (SVM), achieving a prediction accuracy of 79.49% on lung adenocarcinoma gene expression data.

However, machine learning-based approaches to analyzing gene expression data for tumor prediction and key gene screening encounter several challenges. First, the categories of gene expression data typically display extreme imbalance, resulting in poor prediction performance for smaller categories, as classifiers tend to be more biased toward the dominant categories during training. Second, gene expression data often exhibit a large number of feature dimensions, which can easily lead to a dimensionality catastrophe, complicating the training and analysis of models, and increasing the difficulty of screening for key genes.

Many feature selection algorithms and classification prediction models have been applied to the study of gene expression profiling data ^[12]. Xie ^[13] et al. introduced a normalized mutual information method for screening key genes in unbalanced gene data, aiming to address the issue of traditional mutual information favoring multi-valued features. They approximated the gene subset size to 20–50 across multiple tumor gene datasets. Ye ^[14] et al. proposed a hybrid feature selection algorithm that combines FCBF and support vector machine recursive feature elimination (SVM-RFE). Through experiments on five public datasets, they reduced the gene subset size to approximately 10 and validated it using a k-nearest neighbor classifier, achieving an accuracy ranging from 83.87% to 100%. Ludwig ^[15] et al. suggested a fuzzy decision tree algorithm for analyzing gene expression in colon cancer data classification, achieving an accuracy of 80.28%. Zeebaree ^[16] et al. employed a simple CNN model to classify 10 cancer datasets for prediction and compared it with SVM and random forest. The results were excellent on most datasets, with an average classification accuracy of 94.74%. Nguyen ^[17] et al. proposed an improved hierarchical analysis method and incorporated the probabilistic neural network into it, achieving an accuracy of 88.89% in colon cancer data diagnosis. Xiao ^[18] et al. conducted experiments on lung adenocarcinoma, gastric adenocarcinoma, and breast cancer by integrating multiple machine learning models using deep learning. They achieved a significant improvement in accuracy compared to a single machine learning classifier, reaching 99.20%, 98.78%, and 98.41%, respectively.

In summary, the primary challenges in the analysis and research of tumor gene data include data category imbalance and the difficulty of gene screening arising from high feature dimensions. In addition, due to the specificity of genetic data, there may be complex dependencies between certain genes, which should be considered during model construction. To address these challenges, we propose the LATCNN model, which is based on a hybrid feature selection approach and an attentional convolutional neural network, for analyzing gene expression data to predict lung adenocarcinoma. The initial step involves utilizing the FCBF algorithm for rapid filtering in feature selection to eliminate a substantial number of redundant genes. Subsequently, the k-means-SMOTE ^[19] method is employed to address the impact of data category imbalance. Following this, the dynamic inertia weight particle swarm optimization algorithm is applied to further refine the features, aiming to filter out key genes. Finally, atCNN is constructed for the prediction of lung adenocarcinoma. Accuracy, recall, F1 score, and MCC were used to measure the model performance. Extensive experiments demonstrate that LATCNN exhibits exceptional performance. The main highlights are as follows:

1) We propose a hybrid feature selection algorithm that combines FCBF and an improved PSO algorithm.

2) We propose a new adaptive inertia weight PSO (AWPSO) algorithm that decays rapidly to dynamically adjust the inertia weights based on the number of iterations.

3) We use the k-means-SMOTE method to deal with the data category imbalance problem.

4) We propose that attentional convolutional neural networks can better capture the dependencies between features through the attention mechanism.

2.   Materials and methods

2.1.   Architecture and workflow of the LATCNN model

In this paper, we propose the LATCNN model for predicting lung adenocarcinoma and screening its key genes based on gene expression data. LATCNN consists of three main components: Data preprocessing, feature selection, and the construction of a classification model. To obtain gene expression data for lung adenocarcinoma, we performed data correction and variance analysis on gene microarray data. The FCBF algorithm was then applied to rapidly filter out irrelevant feature genes. Subsequently, the AWPSO algorithm containing dynamic inertia weights was utilized to further approximate the feature subset, reduce the data size and identify key genes. Additionally, the k-means-SMOTE method was employed to address the issue of data category imbalance. The preprocessed feature data were fed into the constructed atCNN for the prediction of lung adenocarcinoma. Finally, a series of experiments were conducted to assess the model's performance in both key gene screening and lung adenocarcinoma prediction. Figure 1 illustrates the overall structure and flow of LATCNN.

2.2.   Data collection and processing

The original lung adenocarcinoma data utilized in this study were obtained from the GEO database, with the gene chip number GSE31210 ^[20,21]. Gene chips are composed of thousands of biomolecular probes, and each probe point corresponds to a gene. However, there was a situation where a gene was represented by more than one probe, so the downloaded cancer tissues and paracancerous tissues were subjected to probe transformation. During this process, gene probe IDs are replaced by gene names, the data with the same gene names were merged by averaging, and the data of the genes that had no differentiation were deleted. The number of samples with lung adenocarcinoma in the integrated dataset was 226, and the number of normal samples was 20, and the number of genes was 20,248. A column of labels was added to the dataset to indicate whether the disease was present or not, with 1 label for lung adenocarcinoma samples and 0 for normal samples.

Gene expression data from gene chips or high-throughput sequencing typically exhibit right-skewed or left-skewed distributions, i.e., non-normal distributions, due to obeying different distributions. Logarithmic processing of the data can transform the data into a form that is closer to a normal distribution, which helps to satisfy the assumption of normality in statistical analysis and thus apply a wider range of statistical methods. Due to the variation introduced by experimental conditions or microarray differences, the abundance of gene expression is much different, and for the convenience of subsequent modeling, gene expression data need to be normalized. The specific steps of normalization are: First, calculate the quantile of each gene in each sample, and then adjust the data of each sample to a common distribution of quantiles (usually the median), which helps to make the distribution of data consistent among different samples. The specific formula is given below:

where $x'$ is the normalized gene data, $x$ is the original expression data, $q$ is the sample quartile, and $Q$ is the global quartile, and a comparison of the box-and-line plots before and after data correction is shown in Figures 2 and 3.

The fold change method is the most basic method for gene expression analysis, which calculates the fold difference between the diseased and normal gene data, and if the fold difference is greater than a specified threshold, the gene is identified as differentially expressed. If the multiplicity difference is greater than a specified threshold, the gene is identified as differentially expressed, and the expression formula is as follows:

where ${\bar x_1}(i)$ and ${\bar x_2}(i)$ denote the mean value of gene i in the two sets of data, respectively, and whether it is a differential gene is judged by comparing $FC(i)$ with the specified threshold. The multiplicative change method can effectively reduce the data dimension, but it is not precise enough, and the limitation of this method is that the threshold value is not well determined, which is likely to result in a high proportion of false-positive results for genes. There are many characterized genes after screening, and other characterization screening methods should be used subsequently to further narrow down the screening scope.

The fold change method specifies a fold threshold of 1 and a threshold of 0.05 for the parameter p. The genes were selected as up-regulated genes if the expression difference was greater than 1-fold, and down-regulated genes if the expression difference was less than 0.5-fold. A total of 3099 differential genes were screened by the fold change method of differential analysis.

Comparison of gene expression data before and after all pretreatment steps is shown in Tables 1 and 2.

In addition, in order to evaluate the performance of our proposed LATCNN model more comprehensively, we also chose three public tumor gene expression datasets for our experiments, namely, breast cancer, lung cancer, and DLBCL, which were obtained from the Kent Ridge Bio-medical dataset repository as mentioned in the literature ^[22]. It is worth noting that these datasets cover different types of cancers and are not limited to lung adenocarcinoma. By examining data from different cancer types, we aim to validate the generality and applicability of the LATCNN model, as well as its performance in handling diverse cancer expression data. Such a design allows us to gain a more comprehensive understanding of the potential broad application value of our proposed approach. Details of all datasets are shown in Table 3.

2.3.   Feature selection methods

2.3.1.   FCBF

FCBF ^[23] is a fast-filtering feature selection algorithm with symmetric uncertainty (SU) instead of information gain (IG) as a relevance measure.

The information entropy ^[24], is a measure of uncertainty of a random variable in information theory. Assuming that there is a random variable $X = \{ {x_1}, {\text{ }}{x_2}, {\text{ }}..., {\text{ }}{x_n}\}$, with probability $p(x)$, the information entropy $H(X)$ of $X$ is expressed as:

The greater the information entropy, the greater the amount of information it contains. Assuming that there is a random variable $Y$, the information entropy of $X$, i.e., the conditional entropy $H(X|Y)$, after observing the entropy value of $Y$, is expressed as:

where $p(y)$ is the probability of $y$, and $p(x|y)$ denotes the probability of event $x$ occurring under the given condition $y$.

The information gain $IG(X|Y)$, on the other hand, measures the extent to which the information entropy of the random variable $X$ is reduced, conditional on the information entropy of the random variable $Y$ being computed. In short, the information gain quantifies the amount of information provided by $Y$ for the prediction of $X$. The larger its value, the higher the correlation between $X$ and $Y$. The expression is:

The SU value is a measure that describes the degree of correlation between two nonlinearly correlated variables, which is calculated using information entropy, a normalized form of mutual information, capable of solving the problem of the tendency of mutual information to refer to more than one characteristic, and a normalized form of information gain, with symmetric uncertainty $SU(X, Y)$ expressed as:

From the above equation, when the value of $SU(X, Y)$ is 0, it indicates that there is no correlation between variables $X$ and $Y$. When the value of $SU(X, Y)$ is 1, it indicates that variables $X$ and $Y$ are perfectly correlated.

2.3.2.   ReliefF algorithm

Feature weight is a metric for evaluating the importance of features in the ReliefF algorithm and is used to update the weights of features in each iteration ^[25]. Feature weight $w[A]$ is an important metric used to measure the relationship between the distance value between the original feature set and feature $A$. The expression for $w[A]$ is:

$R$ represents a random sample drawn from the initial sample set. $class(R)$ represents the sample class containing sample $R$. $P(C)$ then denotes the ratio of the number of selected sample features to the total number of sample features among all sample classes. $H$ is the k-nearest neighbor sample among samples of the same class as $R$, and $M$ is the k-nearest neighbor sample among samples that are not of the same class as $R$. $diff(A, {R_1}, {R_2})$ denotes the difference between samples ${R_1}$ and ${R_2}$ on feature $A$, and ${M_j}(C)$ denotes the jth k-nearest neighbor sample of class $C$. The expression for $diff(A, {R_1}, {R_2})$ is given by:

The features are ranked according to the feature weights $w$ and the top-ranked features are selected as the final feature subset.

2.3.3.   SVM-RFE

The objective of SVM is to construct an optimal hyperplane to maximize the classification interval to achieve accurate classification of samples. When the samples are linearly differentiable and satisfy the constraints, the objective function of SVM can be expressed as:

where $w$ represents the weight vector of the hyperplane, $b$ represents the classification threshold, and $y$ represents the category labels.

In solving the nonlinear high-dimensional pattern recognition problem, SVM uses the kernel function $K({x_i} \cdot {x_j})$ to map the data into a high-dimensional space to find the optimal classification hyperplane to achieve linear differentiability. Ultimately, SVM computes the discriminant function by applying quadratic programming and Lagrange's dyadic theorem:

SVM-RFE uses a linear kernel function and the ordering coefficients can be defined as:

According to the above equation, the weight $w$ represents a linear combination of non-zero $\alpha$ support vectors, and an increase in weight indicates that the feature contains more categorical information.

2.3.4.   PSO

The PSO algorithm is a heuristic global optimization method that achieves an approximate optimal solution by simulating the social behavior of a flock of birds in a multidimensional space ^[26]. Each object in the PSO algorithm is referred to as a particle, and each particle has the attributes of position and velocity. Assuming that the algorithm runs in n-dimensional space, the pth iteration position and velocity of particle i can be denoted as $x_i^p = [x_{i1}^p, x_{i2}^p, \cdots, x_{in}^p]$, $v_i^p = [v_{i1}^p, v_{i2}^p, \cdots, v_{in}^p]$, respectively. In the iterative process, each particle judges the superiority of the current position according to the calculated value of the fitness function, and denotes the value of the best position of particle i in the previous p iterations in the dth dimension by $pbest_{id}^p$, and denotes the value of the best position of all particles in the previous p iterations in the dth dimension by $gbest_{id}^p$. The particle decides the next position and velocity by the current position, velocity and the above two best positions, and the update formula for particle i is as follows.

Here, ${c_1}$, ${c_2}$ are called learning coefficients, ${r_1}$ and ${r_2}$ take the random numbers from 0 to 1, $\omega$ are the inertia weights of the particles during the iteration ^[27], which are generally taken as constants, $U(0, 1)$denotes the random numbers from 0 to 1 obeying a uniform distribution. In addition, the velocity of each particle during iteration must be limited to$[{v_{\min }}, {v_{\max }}]$.

2.3.5.   Fast-decay adaptive inertia weights PSO algorithm (AWPSO)

Although the PSO algorithm possesses strong search ability, it often misses the optimal solution by falling into the local optimum when facing multi-polar problems. In order to further enhance the overall search ability of the particles, this paper introduces adaptive inertia weights to update the particle velocity:

Here, ${\omega }_{max}$ and ${\omega }_{min}$ are the maximum and minimum inertia weights, respectively; $\alpha$ and $\beta$ are the empirical value of 0.01 and 50; $iter$ is the current iteration number; $\max iter$ is the preset maximum iteration number. From Eq (15), the value of $iter$ eventually decreases from max to min with iterations. inertia weights have the function of balancing the global and local search of the particle swarm algorithm. When $iter$ is larger, it can improve the global search ability of PSO; when it is smaller, PSO possesses stronger local search ability.

The fitness function is an index for evaluating the quality of individual particles, and the selection of fitness function has a great influence on the convergence speed of PSO algorithm and whether it can find the optimal solution. The fitness function selected in this paper is adapted from the literature ^[28], and the specific formula is as follows:

where $\alpha$ is a hyperparameter that decides the tradeoff between the classifier performance $P$, and the size of the feature subset ${N_f}$ concerning the total number of features ${N_t}$. The classifier performance can be the accuracy, F-score, precision, and so on. In this paper, the accuracy of CART and the current number of features are used as the fitness function, because it has the ability of feature selection, which will lead to better verification of the feature subset and better evaluation of the quality of individual particles.

2.4.   K-means-SMOTE methods

In gene expression profiling data, category imbalance is an important issue. This means that the sample instances are unevenly distributed across different classes and hence the results of unbalanced data classification are biased towards the majority class. SMOTE is an improved algorithm for the random oversampling method, as the random oversampling method is a direct re-adoption of a few classes, which will result in a lot of duplicate samples in the training set, and is prone to cause overfitting problems in the resulting model ^[29]. The basic idea of the SMOTE algorithm is that for each minority class sample ${x_i}$, randomly select a sample ${\hat x_i}$ from its nearest neighbors (${\hat x_i}$ is a sample in the minority class and defaults to 1), and then randomly select a point on the line between ${x_i}$ and ${\hat x_i}$ as the newly synthesized minority class sample. The distance formula is:

For each randomly selected nearest neighbor b, respectively, a new sample is constructed according to the following formula:

The process of the SMOTE algorithm to synthesize few class samples is shown in Figure 4.

K-means-SMOTE is an enhanced version of the SMOTE algorithm proposed by Douzas et al. in 2018 ^[19], introducing a clustering operation on the basis of the SMOTE algorithm. Initially, the spatial domain is partitioned into k clusters using the K-means clustering method. Subsequently, based on the imbalance ratio, clusters requiring oversampling are selected, and additional samples are allocated to the sparser minority class clusters. For each selected cluster, the SMOTE algorithm is then applied to perform oversampling. The final output consists of a new dataset containing both the original samples and the synthesized samples.

2.5.   Improved convolutional neural networks based on attention mechanism (atCNN)

To predict lung adenocarcinoma more accurately, we propose a neural network classification model that becomes atCNN, which consists of an input layer, two convolutional layers, two pooling layers, two dropout layers, an attention module layer, two dense fully connected layers, a flat layer, and an output layer. AtCNN's network structure and the overall structure of the LATCNN model are shown in Figure 5.

The feature genes extracted by different feature selection algorithms are considered input features. In the convolutional layer, the convolutional kernel width is defined as 3. The convolutional layer's activation function employs the LeakyRelu function, a derivative of the Relu function that solves the Relu dead zone problem by introducing a small non-zero slope. When presented with an input sample, the convolutional layer functions in the subsequent manner:

Where$w$ denotes the convolution kernel size and $w_{pq}^k{x_{i + p, q}}$ denotes the weight matrix of the kth convolution kernel with matrix size $P \cdot Q$.

To achieve enhanced precision in results, the utilization of pooling layers is deemed essential for the reduction of output dimensionality derived from the convolutional layer. Typically, the pooling layer can be categorized into two types: average pooling and maximum pooling. In the context of this research, the choice has been made to employ the maximum pooling layer for processing. The maximum pooling operation involves the selection of the highest value within the filtering range. Two distinct pooling window sizes, namely 2 and 3, are applied respectively. The process of channeling the convolution-processed data into the maximum pooling layer is detailed as follows:

To prevent overfitting, a stochastic deactivation layer is attached after the pooling layer with the parameter set to 0.5.

An attention module is added before the second convolutional layer to introduce a self-attention mechanism to improve the performance of the model. The self-attention mechanism serves to introduce a kind of context-dependent weighted pooling in the intermediate layer of the model in order to capture long-distance dependencies in the input data. The relationships between different features can be complex, and the self-attention mechanism can help the model better understand these relationships. The core formula for the attention mechanism is:

where,

Here, ${X_i}$ is the ith element of the input sequence, ${W_Q}$, ${W_K}$, ${W_V}$ are the corresponding weight matrices, and ${d_k}$ is the dimension of the attention header. $Q$(Query) is a query vector that is used to query all other Key vectors to determine the weight given to a given Key in the attention mechanism, $K$(Key) is a key vector used to compare the query vectors, which denotes the importance of each element in the input sequence, and the $V$(Value) vectors are the values associated with each Key, which are used to generate the final output. In the self-attention mechanism, for each query vector, a weight is computed based on the Key vectors corresponding to it, and then the Value vectors associated with these Keys are weighted and summed using these weights. The final output is this weighted sum.

The input matrix transforms an output vector through the utilization of a flattening layer after the application of data filtering. Within this model, two dense fully connected layers with 128 and 64 neurons, respectively, are employed. The processing of these dense layers is segregated into two phases: Forward propagation and backpropagation. The former is employed for the computation of the desired output results, while the latter is utilized to iteratively optimize the parameters of the forward propagation process. The parameters, which are updated post-backpropagation, are subsequently reused for the computation of the output values. An activation function known as Relu is applied to each output. The mathematical expression representing the dense fully connected layer is presented below:

The output of the dense layer, denoted as ${a_i}$, is influenced by the weights, symbolized by ${w_{ij}}$. The input to the output of the pooling layer, which is referred to as ${x_j}$, contributes to this process. Additionally, a bias parameter, represented by ${b_i}$, is incorporated. The updated weights and bias parameter, denoted as ${w_{ij}}^\prime$ and ${b_i}^\prime$, respectively, play a crucial role in this context. It is imperative to highlight that the loss function quantifies the cumulative error.

In the classifier's ultimate output layer, the SoftMax activation function is incorporated, facilitating the assignment of a numerical range of 0 to 1 to each element within the vector. Furthermore, the cumulative sum of these elements invariably equals 1. This characteristic proves particularly advantageous in conveying the likelihood of the presence of lung adenocarcinoma. The mathematical expression for the SoftMax function is provided below:

2.6.   Model evaluation indicators

The most effective approach for illustrating the model's robustness and utility is regarded as the utilization of k-fold cross-validation ^[30]. In this research, a cross-validation technique is employed for the evaluation of the method's predictive capability. Furthermore, several potent assessment metrics have been chosen to validate the model's feasibility. These chosen metrics encompass accuracy (ACC), sensitivity (Sn), specificity (Sp), precision (Pre), the F1 index (F1), and Matthew's correlation coefficient (MCC).

A confusion matrix usually evaluates the model's output for a classification query. Table 4 shows the confusion matrix.

For the binary classification model, after the classifier classifies instances into positive and negative classes, there are four classification outcomes. True positive (TP): A positive class instance that is also classified as a positive class. True negative (TN): A negative class instance that is classified as a negative class. False positive (FP): An instance of a negative category, but is judged to be a positive example. False negative (FN): An instance of a positive category, but is judged to be a negative category. The relevant definitions and formulas for the six indicators are represented below:

Accuracy is the degree to which an instance is correctly described and is calculated as:

Specificity is the ability to correctly identify negative class samples and is calculated as:

Sensitivity is the proportion of correctly predicted positive class instances to the total number of actual positive class samples and is calculated as:

Precision is the accuracy of predicting positively classified samples and is the proportion of samples predicted to be positively classified that are correctly predicted, calculated by the formula:

Accuracy and sensitivity are conflicting indicators. In general, when accuracy is high, sensitivity tends to be low, while when accuracy is low, sensitivity tends to be high. The F1 index is a weighted average of accuracy and sensitivity, which enables the two indicators to be taken into account together, and is calculated using the following formula:

Matthews correlation coefficient is a comprehensive evaluation index that takes into account the sensitivity and specificity and takes the value between [0, 1], the larger the value of MCC, the better the classification effect of the model, and the calculation formula is:

In addition, the model was evaluated using the AUC value of the area under the curve of the ROC curve, which describes the performance of the classification model as a function of threshold, with no point on the curve reflecting the perceptibility of the same signal stimulus. It is a curve plotted with the true positive rate as the vertical coordinate and the false positive rate as the horizontal coordinate. When comparing two or more models, it is possible to visualize the strengths and weaknesses of the models. The closer the curve is to the upper left corner, the better the model performance. The formula for calculating the true positive rate is the same as for the sensitivity, and the formula for calculating the false positive rate is:

The larger the area under the curve, the better the performance of the corresponding model, and the AUC value takes the range.

2.7.   Algorithm description and pseudo-code

When using the hybrid feature selection algorithm proposed in this paper, the FCBF algorithm is first used to do the initial feature selection to quickly filter out a large number of irrelevant features using symmetric uncertainty as a metric, next we propose a PSO optimization algorithm that dynamically adjusts the inertia weights to do the second stage of the feature selection using the performance of the CART and the number of features together to form the fitness function in order to further narrow down the the optimal gene subset. The following is the pseudo-code for this description.

Input: Tumor gene expression data $(X, C)$, where $X = \{ {x_1}, {x_2}, \ldots, {x_m}\}$, $C = {\{ {c_1}, {c_2}, \ldots, {c_n}\} ^T}$.

Output: The best subset of features G.

1) G = {} # Initialize G as an empty set

2) for i in range(1, n+1):

3) Calculate C-correlation value $SU({c_i}, d)$

4) G = G + {${c_i}$}

5) end

6) Sort G in descending order based on SU values

7) G = Top 100 elements of G

8) For each particle i:

9) Calculate initial fitness value fit[i]

10) If fit[i] is better than individual best fitP[i], update fitP[i] and individual best position Xpb[i]

11) If fitP[i] is better than global best fitG, update fitG and global best position Xgb

12) For each particle i:

13) Update particle velocity V[i], including the update for adaptive inertia weight

14) Update particle position X[i]

15) Repeat steps 8−14 until reaching the maximum iteration count max_iter

16) Return the global best feature subset Xgb

17) G = Xgb

3.   Result and discussion

In order to verify the effectiveness of LATCNN, the dataset made from gene chip GSE31210 and three publicly available datasets are chosen for experiments in this paper. The experiments in this paper were all conducted at Google Colab with an Intel Xeon Dual Core 2.2 GHz CPU and NVIDIA Tesla K80 GPU.

3.1.   Comparison of feature selection methods

For the comparison of feature selection algorithms, we used SVM as a classifier and performed ten trials on the best subset of features screened by each dataset. Five-fold cross-validation was used for each experiment, and the average result of the ten trials was used as the final accuracy. In order to better evaluate the hybrid feature selection algorithm combining FCBF and AWPSO proposed in this paper, this paper also conducts comparative experiments with the following algorithms, including FCBF-PSO, ReliefF-AWPSO, ReliefF-SVMRFE, and FCBF-SVMRFE used in the literature ^[14]. Among them, the filtering algorithms FCBF and ReliefF's are both selected to rank the top 100 features. Table 5, Figures 6 and 7 show the experimental results for the four datasets under the five algorithms, where ats denotes the size of the subset of optimal genes and acc denotes the final accuracy.

In summary, it can be seen that the FCBF-AWPSO algorithm proposed in this paper has the smallest size of the best gene subset selected on each dataset, and none of the classification accuracies are lower than the other combinations. This indicates that the combination of FCBF and AWPSO is a better feature selection algorithm. It can also be seen that compared to ReliefF, FCBF is more suitable for gene priming in the first stage of the hybrid feature selection algorithm.

As shown in Figure 8, we also recorded the time taken to run these algorithms, and it can be seen that the algorithms in this paper took the least amount of time on each dataset.

Based on the above analysis, the hybrid feature selection algorithm using FCBF combined with AWPSO can quickly and accurately capture the feature information of tumor gene data. Therefore, we adopt this method to construct the LATCNN model.

3.1.1.   Analysis of key genes

The 12 key genes of GSE31210 after screening were COL10A1, AFAP1-AS1, MNX1, ENTREP2, GALNT7, LGI3, RCC1, B3GNT3, ST6GALNAC3, KDELR3, STIL, and TACC1. Among these genes, COL10A1, AFAP1-AS1, MNX1, B3GNT3, and TACC1 have been documented to be associated with lung adenocarcinoma [31−35]. GO functional enrichment analysis of these genes was performed, and the results are shown in Figure 9, in which RCC1, STIL, and TACC1 belong to the same 3 pathways: GO:0007052, GO:1902850, GO:0007051. GALNT7, B3GNT3, and ST6GALNAC3 belong to the same 5 pathways: GO:0006486, GO:0043413, GO:0070085, GO:0009101, GO:0009100, and GO:0016757. The function of each gene is described in Table 6.

3.1.2.   K-means-SMOTE algorithm result

The K-means-SMOTE algorithm is used after the initial screening of features for FCBF to balance the minority samples. Since the second stage of feature selection is influenced by the classifier, it is affected by the unbalanced data. The balancing of the dataset is achieved by adding new minority instances and making them equal to the majority instances. Table 7 compares the results before and after using the SMOTE algorithm on the original dataset.

3.2.   Comparison of multiple methods

In order to objectively evaluate LATCNN, we launched a series of experiments on the independent dataset GSE31210 and three public datasets. Each method was subjected to 5 experiments on each dataset, and 10-fold cross-validation was used each time, with the most average value as the final result. For classifier comparison, we chose three machine learning and three deep learning baseline methods, namely SVM, Random Forest, XGBOOST, CNN, DNN, and LSTM, and accuracy, recall, F1 score, and MCC are used as evaluation metrics, respectively. Table 8 shows the results of these methods on each dataset.

As can be seen from Table 8, atCNN performs better on all datasets compared to other classifiers. On the lung adenocarcinoma independent dataset GSE31210, the accuracy, recall, F1 score, and MCC reached 99.70%, 99.33%, 99.98%, and 98.67%, respectively.

We also compare LATCNN with some of the existing models in the literature, including the proposed FCBF-PA-SVM in literature ^[11], the FCBF-SVMRFE feature selection using FCBF-SVMRFE features and prediction using four machine learning classifiers as mentioned in literature ^[14], and a CNN model as mentioned in literature ^[16].

Table 9 shows the accuracy comparison between LATCNN and the above three models on each dataset, and the model in this paper achieves the best performance on all four datasets, reaching 99.64, 97.85, 99.71, and 98.53, respectively. We also modified the models in the above three literatures by replacing their original methods with the feature selection method proposed in this paper, and Figure 10 shows the comparison between before and after replacement. It can be seen that after the replacement, the accuracy of all these models is improved, proving that the feature selection method proposed in this paper can indeed improve the model performance.

Compared with other models, LATCNN has two advantages: First, the hybrid feature selection algorithm can remove irrelevant and redundant features and accurately select the optimal subset of features, which on the one hand facilitates the subsequent classification prediction of the model, and on the other hand, it also filters out the key genes at the same time. Second, the convolutional neural network with the introduction of the attention mechanism performs better than traditional machine learning classifiers and also performs better than other deep learning models. However, our model has some shortcomings, compared with machine learning, deep learning models take longer to train and are more susceptible to hardware factors. Thus, finding better hyperparameters will be the key to optimize the model in the future. In summary, LATCNN can be applied to tumor genetic data for prediction and key gene screening.

4.   Conclusions

Tumors are extremely harmful to humans, and early identification and screening of key genes are particularly critical. We take lung adenocarcinoma as an example and propose the LATCNN model, which aims to accurately predict lung adenocarcinoma and screen key genes. We face the problem of how to better downsize the gene data to narrow the scope and more accurately make predictions.

Initial steps include processing the gene chip data to prepare a usable tumor gene expression dataset. The next step is feature extraction using a hybrid feature selection algorithm, which includes fast filtering using FCBF, unbalanced categories using K-means-SMOTE, and further feature reduction using AWPSO to achieve optimal feature extraction. Finally, atCNN was constructed for prediction, achieving an accuracy of 99.70 and the best results on the other three public datasets. Comparison with existing methods shows that LATCNN performs better in lung adenocarcinoma prediction and key gene screening, and has the potential to analyze other oncogene data.

Our aim of this study is to explore better methods for oncogene screening as well as prediction. In the future, better integration with clinical data is suggested to further improve prediction accuracy. In addition, for gene chips prepared in different ways, a suitable batch correction method is searched for to obtain a better dataset through joint analysis of multi-chip data. By solving these problems, it will eventually help our research on tumors.

Use of AI tools declaration

The authors declare that they have not used Artificial Intelligence (AI) tools in the creation of this article.

Acknowledgments

This study is financially supported by the Gansu Provincial Science and Technology Program Project of China (grant no. 21JR1RA272) and the Gansu Provincial Department of Education: University Teachers Innovation Fund Project (grant no. 2023B-105).

Conflict of interest

The authors declare that there are no conflicts of interest.