A new approach to generating virtual samples to enhance classification accuracy with small data—a case of bladder cancer

1.   Introduction

Bladder cancer is a common cancer in the human urinary system. Some bladder cancer studies have reported that this cancer is related to tumor suppressor genes and oncogene genes, such as multidrug resistance (MDR), topoisomerase II (Topo II), Rb, epidermal growth factor receptor (EGFR), HER-2 (Neu), c-ErbB-3, c-ErbB-4, Cyclin A, Cyclin D1, P16, Cdc 2, Bcl-2, Bax, etc. ^{[1,2,3,4,5,6,7]}. These genes can be used to diagnose bladder cancer ^[1,4,6] and analyze the effects of X-ray treatment among bladder cancer patients ^[5,7]. For example, higher Cyclin D1 values indicate that patients are in the early stage of bladder cancer ^[2]. The over-expression values of EGFR, Neu, c-ErbB-3, c-ErbB-4 may reflect the treatment outcomes of Cobalt-60 (Co-60) radiation therapy on bladder cancer ^[3]. These studies often suffer from a small sample problem because it is difficult to obtain a sufficient number of gene expression profiles from patients due to the need for a costly genome sequencing procedure ^[8,9]. However, for researchers who adopt data-driven models as forecasting tools to predict cancer, it is necessary to collect sufficient data and find representative samples from which to build data-driven models with good learning performance ^[10,11,12]. With a limited number of gene profiles, traditional machine learning methods cannot construct data-driven models that will provide accurate predictions of bladder cancer. For this reason, small data learning has become an important challenge to bladder cancer prediction with small datasets.

1.1.   Small data learning

Previous research approaches to the problem of small data learning can be divided into two types: The first proposes a better learning model for different data sets. For example, Mao et al. ^[13] proposed a modified Mahalanobis-Taguchi System to extract important information for the purpose of improving classification accuracy for a high-dimensional small dataset. Izonin et al. ^[14] proposed the input-doubling method to improve prediction performance of the compressive strength of trabecular bone with only 77 observations. Due to data diversity, it is difficult to apply the same model to different datasets. Researchers must fine-tune the parameters of data-driven models to improve the learning of models using small data. The second approach is a virtual sample generation (VSG) method used to expand the quantity of data. VSG methods have been developed in various fields, such as diagnostic classification of tumors ^[1,4,6,15], prediction of the treatment effects of radiotherapy on tumors ^[5,7], and engineering applications ^[16,17,18]. These methods based on the concept of VSG are typically classified into three types: resampling-based VSG, model-based VSG, and diffusion-based VSG. The bootstrap method and the Synthetic Minority Over-sampling Technique (SMOTE) ^[19] typify resampling-based VSG methods. The bootstrap method replaces the original sample with a synthetic sample by re-sampling the original samples ^[20]. The SMOTE method draws out a neighboring example of the original data as a new example to fill gaps between data. The neighboring example is calculated as shown in the following equation: ${x_{synthetic}} =$ ${x_i} + random(0, 1) \cdot ({x_j} - {x_i}), \forall i \ne j$, where ${x_{synthetic}}$ is a synthetic virtual sample different from ${x_i}$, and ${x_i}$ is resampled data and ${x_j}$ is the neighboring data of ${x_i}$, e.g., Figure 1.

Many studies have combined bootstrap methods with data-driven models to achieve good learning results ^[21,22,23]. For example, Lee et al. ^[21] created samples using the combined bootstrap method to construct a support vector regression (SVR), when only a small number of samples from the health care system were used to predict oscillometric measurement of blood pressure (BP). La Rocca and Perna ^[23] proposed a combination of extreme learning machine (ELM) models with bootstrap samples to improve the learning accuracy of small data sets. These experimental results showed that the proposed method led to a significant improvement in the performance of small data learning. The problem with bootstrapping, however, is that it can repeatedly overlearn the same sample, thus creating an overfitting problem. In this regard, Lee et al. ^[21] applied a bootstrap to a SVR to reduce the probability of overfitting. Unfortunately, most machine learning models are not immune to this problem. From the perspective of augmenting data diversity, some scholars suggest that by treating data that approximates the original sample as a virtual sample, the model can learn diverse data and reduce the incidence of overfitting.

Model-based VSG methods depend on machine learning models or artificial neural networks to find the relevance between input attributes and the output targets of small data in high dimensional space. The characteristics of original data can be learned by generating virtual samples in high dimensional space. Cho et al. ^[24] randomly generated virtual samples and selected virtual samples that were similar to the original examples using a multilayer perceptron neural network. Huang and Moraga ^[25] suggested the use of the diffusion-neural-network (DNN), which utilizes information diffusion and fuzzy theory to derive artificial samples in DNNs. In the field of image recognition, the generative adversarial net (GAN) model ^[26], which is specifically designed to generate synthetic images. The concept of the GAN is to generate a fake image that approximates a true image to increase the amount of training data. However, the GAN model is computationally intensive and may suffer from mode collapses due to low image variability when the true image data has an extremely skewed distribution ^[27].

Diffusion-based VSG methods are represented based on information derived from original data. The method usually assumes that data comes from a mathematical function or a specific probability distribution to generate virtual samples. The most representative method is mega-trend-diffusion (MTD) method ^[28]. Based on a triangular membership function (MF), the MTD method is to estimate domain of small data and generate virtual samples within the domain. Methods based on MTD method have been developed to improve the prediction performance of data-driven models with small datasets in practical applications ^{[29,30,31,32]}. For example, Majid et al. ^[30] applied the MTD technique to perform estimation of the data range in minority class data. Their experimental results showed that the generated virtual samples improved the classification accuracy of four data-driven models (k-nearest neighbor, support vector machines, Naïve Bayes, and random forest) for the prediction of human breast and colon cancers.

As an alternative to the MTD method, Yang et al. ^[33] proposed the VSG method based on a Gaussian distribution (GD), and virtual samples were generated within the estimated data domain. However, in the real world, data may be fitted using multiple distributions. The MTD and GD methods did not consider generating virtual samples based on multiple distributions. For this reason, based on multiple triangular MFs, Zhu et al. ^[31] proposed the multi-distribution MTD (MD-MTD) to generate virtual samples intended to improve the learning accuracies of models of purified terephthalic acid production. In addition, the MD-MTD method has been applied to enhance the prediction of an applicant risk assessment of an early online lending platform ^[32]. In addition to the assumption of multiple triangular MFs, Wang et al. ^[34] assumed that the data came from a mixture of normal distributions to create a large amount of data from a similar process and fuse data using a transfer learning method. The distribution of the fused data can be inferred using a nonparametric method. The estimated distribution is used to generate virtual samples to improve prediction in other processes. Although their method achieved good prediction results, it is more complex than the MD-MTD method. In addition, virtual samples generated using their method follow symmetric normal distributions. However, in fact, it is more appropriate to consider using asymmetric distributions to generate virtual samples for small datasets ^[31,32]. Therefore, based on the assumption of multi-distributions, in this paper, a distance-based mega-trend-diffusion (DB-MTD) technique is proposed to avoid over-diffusion of data due to extreme values and estimate the central locations (CLs) of multi-distributions. The detailed motivation for this work is explained in the following section.

1.2.   Motivation

The traditional MTD method uses the midpoint of the data as an estimation of the CL of the data and estimates the skewness of the data by the amount of data greater or less than CL. The concept of data diffusion in MTD is that when the data is more skewed, the data domain is more diffused. However, when estimating the skewness of data in terms of data quantity, there is a problem: a small data set with about the same amount of data on both sides of the CL is estimated to be an un-skewed distribution for the reasons illustrated using Scenario A and Scenario B in Figure 2(a). Scenario A considers the amount of data used to estimate the degree of skewness of the data. As a result, the degree of skewness on the left side of the data is not significant enough to increase the spread of the data range. Scenario B considers the distance between the data and the CL to estimate the degree of skewness, and the data on the left is farther away from the CL, which means that the skewness between the left and the CL is greater, and the data distribution is close to the left skewed distribution. Therefore, the data in the left-hand area has a larger diffusion of data, meaning that more potential information is hidden in this region. Thus, more virtual samples can be explored in that domain. On the other hand, the MTD method assumes the data is fitted in a unimodal triangular MF, and it ignores the possibility that the data may come from a multi-modal triangular MF. This is illustrated in Scenarios C and D in Figure 2(b). The data in Scenario C is fitted in two triangular MFs, where the distance between the two MFs is relatively large. Therefore, it is regarded as multiple triangular MFs. The data in Scenario D indicates that when two triangular MFs have overlapping regions, this means that the similarity between the data is high, and it also means that potential information is hidden in the overlapping region. Therefore, in this paper, the two triangular MFs are fused into a trapezoidal MF.

As mentioned above, data diffusion depends on the degree of data skewness. When extreme values exist in a small dataset, the estimated skewness is excessively left or right, which results in over-diffusion of data. Most virtual samples generated in an over-extended domain are unsuitable and cannot effectively improve the learning of models. To address this problem, the MD-MTD method can be used to modify the estimation of data skewness in order to avoid excessive data diffusion. However, because data skewness in MD-MTD is still determined by the amount of data, the problem where different small datasets result in the same data diffusion shown in Scenario A and Scenario B still exists and thus needs to be solved. In addition, although the MD-MTD method estimates the CL of multi-distributions, it does not consider multiple CLs for multi-distributions. Therefore, in this paper, a distance-based mega-trend-diffusion (DB-MTD) method is proposed to extend the data range and reduce the influences of extreme values when there are small datasets. In our method, data skewness is determined by the distances between examples. When most of the examples are farther from the CL, this means that a significant amount of information is hidden between these examples and the CL, and vice versa. In addition, in this paper, a plausibility assessment mechanism (PAM) is proposed to select appropriate virtual samples generated by multiple triangular MFs or one trapezoidal MF.

This study validates the proposed DB-MTD approach using one bladder cancer classification case from Liao ^[6] to diagnose bladder cancer and a regression case from Tsai et al. ^[7] to predict the effects of treatment of bladder cancer patients with radiotherapy. In this study, a data-driven model using a back-propagation neural network (BPNN) was used to compare the learning performance between the proposed DB-MTD method and other VSG methods. The experimental results of the two bladder cancer datasets showed that the learning performance of the proposed method outperformed the other VSG methods. A paired-test was also used to verify whether there were statistically significant differences in the BPNN model for the two bladder cancer datasets.

The remainder of this study is organized as follows: Section 2 is a review of the literature on VSG methods, as well as a brief introduction to the learning tools used in the experimental validation. Section 3 explains the details of the proposed DB-MTD. Section 4 is the description of the two bladder cancer datasets and provides the experimental results. The conclusions are given in Section 5.

2.   Background

Many studies have attempted to improve the learning performance of small datasets by generating virtual samples that been added into the original dataset to expand the amount of training data ^{[24,25,28,33,31]}. The generation of virtual samples has been verified as an effective strategy by which to extract meaningful information from a small dataset and increase the forecasting accuracy of models. Three related studies on virtual sample generation are reviewed in this section.

2.1.   Diffusion neural network (DNN)

Huang and Moraga ^[25] proposed the DNN method, which applies a diffusion function to generate virtual samples on both sides of a given data point. In their approach, the data point is designated as the midpoint of a fuzzy normal function. After determining the possibility of n midpoints, the DNN approach then carried out symmetric diffusion on both sides of the midpoint to derive 2n virtual samples from Eqs (1)–(5). In Eqs (1) and (2), u and v are virtual samples of the input and output derived by using the original data; ${h_{set}}$ is the diffusion coefficient obtained from Eq (3); "r" represents the linear correlation coefficient of the input x and the output y, and $\psi \left(r \right)$ represents the membership function value of the linear correlation coefficient r, as shown in Eq (5). After obtaining the corresponding values of the fuzzy normal membership function using Eqs (4), (1) and (2) are applied to produce virtual samples, u and v, after which the possibilities of the virtual samples are entered into a neural network for training, as shown in Figure 3.

2.2.   The mega-trend-diffusion (MTD) approach

Li et al. ^[28] proposed the mega-trend-diffusion (MTD) method to estimate the triangular membership function used to generate virtual samples. This method assumes that data follows the assumptions below:

1) Each variable is independent of the other.

2) Each variable is fitted into a unimodal distribution.

The virtual sample generation of data range can be estimated using Eqs (6) and (7).

where ${u_{set}} = (\max + \min)/2$ is the midpoint, and the coefficients of skewness are defined as $Ske{w_L} = \frac{{{N_L}}}{{{N_L} + {N_U}}}$ and $Ske{w_U} = \frac{{{N_U}}}{{{N_L} + {N_U}}}$, which refers to the degree of diffusion on two sides of ${u_{set}}$; $\sqrt { - {\text{2 }}\times\frac{{\hat s_x^2}}{{{N_L}}}\times{\text{ ln}}\left(\varphi \right)}$ and $\sqrt { - {\text{2 }}\times\frac{{\hat s_x^2}}{{{N_U}}}\times{\text{ ln}}\left(\varphi \right)}$ are defined as the proportion of the degree of diffusion, and $\varphi = {10^{ - 20}}$ and $\hat s_x^2$ represent the sample variance, where the virtual sample is generated randomly between interval [a, b], as shown in Figure 4(a).

2.3.   The Multi-distribution MTD (MD-MTD) approach

Zhu et al. ^[31] presented the multi-distributions MTD method to avoid over-diffusion of small data. They modified the MTD method to avoid extreme values to influence estimations of data skewness and CL. In their paper, the data skewness and CL are modified as:

where m is a shape parameter used to adjust the degree of data skewness, and ${x}_{[·]}$ is an order statistic. The m is set to one in their method. The ${N_L}$ (${N_U}$) is the number of values smaller (greater) than ${u_{set}}$. The range estimation of MD-MTD method is defined by Eqs (6) and (7). Due to the adjusted skewness on the left and right side of CL, the data diffusion is reduced into a narrower domain [a, b], as shown in Figure 4(b).

2.4.   Back-propagation neural network (BPNN)

The back-propagation neural network (BPNN) was adopted in the present study as the data-driven model, which is a supervised learning model applicable for classification or forecasting problems. The input and output of the training data were used to build a neural network, and the weight of each node could be adjusted based on the difference between the degree of the expected output value and the actual observed value between each layer. A mapping function was built to process the weights in the different layers, as shown in Eq (11).

The output result for the input can be expressed as Eq (11), where ${X_i}$ and $Y$ are ith input variable and one output variable, respectively; $W$ is the weighting value between the node and other nodes in the layer; $f(\cdot)$ is an activate function for nodes; b represents the product-sum of the weights, which must be greater than an error value, so the input nodes can be transferred into other nodes between linking layers, and p is the number of inputs from other nodes. In this paper, $f(\cdot)$ is set as the sigmoid activation function to update the weights of the nodes in the jth hidden layer as follows:

In a basic BPNN scheme, nodes are composed of so-called "layers, " as shown in Figure 5, where the input and output layers are, respectively, the attributes and target of data. A hidden layer is mainly used to process the data, and the number of layers is determined according to the data complexity. Generally speaking, a higher number of hidden layers and more nodes produce fewer learning errors, but the network scheme in the training process is more complex and has a longer convergence time.

3.   Methodology

In this paper, we propose a unique distance-based mega-trend-diffusion method, including the skewness degree, the diffusion coefficient, the selection of ${u_{set}}$, and the number of ${u_{set}}$. This section illustrates the proposed method in detail, including the plausibility assessment mechanism (PAM) in Subsection 3.8 used to select suitable virtual samples.

3.1.   Symbol definitions

Assume that a training dataset has n examples with m-1 input attributes $\{ {X_j}|j = 1, 2, ..., m - 1\}$, and one output target ${X_m}$, denoted as $T = \{ ({\vec x_1}, {\vec y_1}), {\text{ }}({\vec x_2}, {\vec y_2}), ..., ({\vec x_i}, {\vec y_i})\}$, $i = 1, 2, \cdots, n$, where ${\vec x_i} = \{ {x_{i, 1}}, {x_{i, 2}}, ..., {x_{i, m - 1}}\}$ has m-1 variables and ${\vec y_i}$ is the target of ${\overrightarrow x _i}$. The elements of the variables are $\{ {x_{i, j}}|$ $i = 1, 2, ..., n;j = 1, 2, ..., m\}$ in the training dataset, as shown in Table 1.

3.2.   Data pre-processing

This subsection introduces the Min-Max data normalization method and k-nearest neighbor (kNN) algorithm used for imputation of missing data.

3.2.1.   Min-Max data normalization

Because the proposed method is based on the distances between examples to measure the degree of data diffusion, before performing our proposed method, a data normalization process is necessary to avoid the effects from different variable scales that affect the data diffusion measurement. In this paper, we applied the Min-Max data normalization process to transform the domain of original data ${x_{i, j}}$ into [0, 1]. The transforming formula is expressed as:

where $x_{i, j}^*$ is transformed value and ${\max _j}$ (${\min _j}$) is the maximum(minimum) value of the ${X_j}$ variable. Min-Max normalization is a linear transformation intended to maintain the relative distances between the original examples. As a result, the proposed method can be effectively performed with transformed examples for small datasets with different variable scales.

3.2.2.   kNN in missing data imputation

When missing values occur in the original dataset, they increase the amount of data diffusion measurement error due to an incomplete data structure. There are two types of methods that can be used to deal with missing values. One is the ignoring missing value method in which examples with missing values are deleted, and the other is the data imputation method, where missing values are replaced with plausible values. In this paper, we adopt the data imputation method to fill in missing values instead of deleting them due to the very limited amount of data. Common data imputation methods include mean imputation, imputation with distributions, kNN imputation, etc. Some studies have suggested that kNN has better imputation performance than other methods ^{[35,36,37,38]}. For example, Jadhav et al. ^[38] compared kNN with other methods in terms of data imputation, and kNN significantly outperformed the other methods for numerical datasets. Because the proposed method is applicable to numerical variables, we use the kNN imputation method to handle data with missing values.

The kNN is an instance-based algorithm proposed by Cove and Hart ^[39]. This algorithm classifies examples based on similarity measurements that are made between examples. For data imputation, the kNN is used to impute missing values by selecting the nearest neighbors of examples with missing values. In this paper, we use the Euclidean distance as the similarity measurement to select the nearest neighbors of the example, as follows:

where k is the number of nearest neighbors, and j represents jth data variable. We illustrate a simple example of kNN for data imputation in Figure 6. When we set k as three, the original data with a missing value has two neighbors, where the missing value can be imputed using the nearest neighbors.

3.3.   Measurement of the degree of skewness

In the original MTD, the number of samples occurring on two sides of ${u_{set}}$ is used to determine the degree to which the data is skewed. When the number of samples is significantly larger on one side, the MTD formula is not suitable for estimating the data distribution. In this paper, a unique MTD method is proposed based on the distance between samples. The idea of this paper is that when the sum of the distances between the samples on one side is less than that on the other side, this indicates that the sample distribution is skewed to one side.

The distance between the samples is defined by using the distance equation derived from the Minkowski Distance, as noted by Cha ^[40], where ${\text{Distance}} = {\left({\sum\limits_{i = 1}^d {{{\left| {{P_i} - {Q_i}} \right|}^h}} } \right)^{\frac{1}{h}}}$. Since in this paper, it is assumed that each variable is independent, only a 1-D data distribution is considered. That is, ${\text{Distance}} = \sum\limits_{i = 1}^d {\left| {{P_i} - {Q_i}} \right|}$, where ${P_i}$ represents the ith sample point ${x_i}$, and ${Q_i}$ is the ${u_{set}}$ of variables. The degree to which the data is skewed is considered in Eqs (18) and (19).

where ${G_L}$ (or ${G_U}$) represents the sum of the distance between the smaller (or greater) sample, as compared to ${u_{set}}$ and ${u_{set}}$. Figure 7(a) (or (b)) shows that if $Ske{w_L}$ is smaller (or greater) than $Ske{w_U}$, the data distribution is inferred as being a right (or left) skewed distribution.

3.4.   Diffusion coefficient

In the MTD, the diffusion coefficient is set as $\frac{{\hat s_x^2}}{{{N_L}}}$ or $\frac{{\hat s_x^2}}{{{N_U}}}$ to fine-tune the degree of diffusion on two sides of ${u_{set}}$. When extreme values exist in the dataset, then the variations among the samples are significant, so the sample variance $\hat s_x^2$ leads to excessive effects on the estimate of the diffusion coefficient ${h_{set}}$. In order to reduce this effect, ${h_{set}}$ is set as $\frac{{\left| {{{\hat s}_x}} \right|}}{{{G_L}}}$ or $\frac{{\left| {{{\hat s}_x}} \right|}}{{{G_U}}}$. Therefore, in order to reduce the influence of outliers by using the distance function G on two sides of ${u_{set}}$, we adjust the diffusion coefficient as follows: $\sqrt { - {\text{2 }}\times\frac{{\left| {{{\hat s}_x}} \right|}}{{{G_L}}}\times{\text{ ln}}\left(\varphi \right)}$ or $\sqrt { - {\text{2 }}\times\frac{{\left| {{{\hat s}_x}} \right|}}{{{G_U}}}\times{\text{ ln}}\left(\varphi \right)}$. If ${G_L}$ is greater than ${G_U}$, then it will be skewed to the left as a small ${h_{set}}$, and if there are potential outliers, then the G value in this direction will be increased. Thus, the proposed ${h_{set}}$ can be used to mitigate the increasingly excessive diffusion.

3.5.   Selection of ${u_{set}}$

This selection of ${u_{set}}$ is often seriously affected when there are many extreme values in a dataset, and thus, the midpoint ${u_{set}}$ that is obtained may not represent the tendency of the dataset although ${u_{set}}$ can be set as the mean of the samples, and the mean can then be used as the estimation of the data tendency when the number of samples is sufficient. However, setting the mean of ${u_{set}}$ is unstable in terms of estimating the data tendency in small samples. Thus, when considering the estimation of the tendency of the dataset for a small dataset, the median and mode are considered to be slightly affected by extreme values. k modes can be used to discover the frequency of repeated values in the dataset to predict the tendency of the dataset. Therefore, the proposed setting of ${u_{set}}$ can be defined as follows:

Given a dataset $X = \left\{ {{x_1}, {x_2}, ..., {x_n}} \right\}$, n represents the number of samples, and k is the number of modes, where the mode is defined as identical data that occurs more than twice in a dataset. If no mode exists in a dataset, then the ${u_{set}}$ is the median.

3.6.   Estimation of the data range

Assuming a dataset $X = \left\{ {{x_1}, {x_2}, ..., {x_n}} \right\}$, the forms of Eqs (6), (7) and Eqs (16), (17) can be modified with k ${u_{set}}$ as follows:

Eqs (25) and (26) represent the sum of the distances between ${x_i}$ and ${u_{set, k}}$. ${G_{L, k}}$ and ${G_{U, k}}$ represent the sum of the distances between the sample points fewer and greater than kth ${u_{set}}$, respectively. Therefore, in this case, we use the distance function on two sides of kth ${u_{set}}$ as an expression of skewness, as shown in Eqs (27) and (28).

In Eqs (23) and (24), considering the diffusion coefficient presented in Section 3.4, two specific cases are discussed: If the kth mode is one of the extreme values, this will cause ${G_{L, k}}$ or ${G_{U, k}}$ to be close to 0 because ${x_i} \approx {u_{set}} \to \left| {{x_i} - {u_{set}}} \right| \to 0$; then, the diffusion in the direction of the extreme value cannot be defined, and the lower bound ${a_k}$ or the upper bound ${b_k}$ of the kth mode is set as ${u_{set, k}}$. In addition, when the total number of samples is one, both ${G_{L, k}}$ and ${G_{U, k}}$ are 0.

3.7.   Virtual sample generation with the proposed method

Since the previous MTD method set ${u_{set}}$ as the midpoint, the method is susceptible to extreme values. In addition, the method assumes that the data is fitted into a unimodal distribution, but the data may come from a multi-modal distribution. For these two reasons, the new DB-MTD method proposed in this paper generates virtual samples from a unimodal distribution or a multi-modal distribution, where the median or mode is set as ${u_{set}}$ to reduce the influence of outliers. The process proposed in the DB-MTD used to generate virtual samples is shown in Figure 8.

When using the proposed method to estimate the data range, some possible scenarios are illustrated as follows:

1) When a dataset does not have a mode, the median is taken as the ${u_{set}}$, as shown in Figure 9, Scenario A. Or, when the mode exists in the dataset, then ${u_{set}}$ is the mode, as shown in Figure 9, Scenario B.

2) When there is only one mode, the ${u_{set}}$ is the extreme minimum, as shown in Figure 9, Scenario C. If ${u_{set}}$ is the extreme maximum, the estimation of membership function is as shown in Figure 9, Scenario D.

3) When there is more than one mode, then it is necessary to generate more than one triangular fuzzy membership function, and we thus need to consider whether multiple triangular membership functions will overlap or not.

a. When there is no overlapping area between any two modes, there are two triangular membership functions, as shown in Figure 9, Scenario E.

b. When these two triangular fuzzy membership functions overlap, there is one trapezoidal membership function, as shown in Figure 9, Scenario F.

Since the overlapping of two triangular fuzzy membership functions in Scenarios E and F mean that important data may be located in the overlapping area, the value of the membership function of the sample point is set as 1. In other words, we thus set the possibility of the virtual sample being located in the interval as 1.

After estimating the data range given by Eqs (23) and (24), the virtual sample is randomly generated within the specified range with a uniform distribution.

As mentioned above, we discuss possible scenarios of the membership functions, as shown in Figure 9, where the peak of the membership function is set at 1, and the membership function value of the upper bound and lower bound is set at 0. When k ${u_{set}}$ exists, and sample x lies in the interval [ ${a_k}$, ${b_k}$ ], the value of the membership function of x can be calculated when multiple triangular membership functions do not overlap each other, using Eq (29). When multiple triangular membership functions overlap each other, trapezoidal membership functions are used, as shown in Eq (30).

3.8.   Plausibility assessment mechanism (PAM)

According to the virtual sample generation method, the values of the fuzzy membership functions are used to randomly generate virtual samples in order to fill the gaps in the data in the case of small sample learning. In this paper, a procedure was developed for a PAM that could be used to select the virtual samples based on an inferred data distribution so as to increase learning robustness. This mechanism can be used to examine whether the randomly generated virtual values are qualified or not, and it is described as follows in detail: Firstly, a tv (temporary value) is randomly generated from the estimated interval [a, b], after which the corresponding membership function values, MF (tv), are discovered, as shown in Figure 10.

Three possible scenarios are illustrated, as follows: In Figure 10, Scenario A only has one ${u_{set}}$; Scenario B is two triangular membership functions, and Scenario C is a trapezoid fuzzy membership function with two ${u_{set}}$. When determining whether tv can be kept as a virtual sample or not, we additionally generate a rs (random seed) ranging between [0, 1], as shown in Figure 10. This random seed is defined based on a uniform distribution. In the testing process, if the random seed is less than the membership function value of tv, then this tv will be kept as a suitable virtual sample v; if not, this tv will be abandoned, and a new virtual value will be generated. When a tv is close to the ${u_{set}}$, then the membership function value of tv is close to 1, and thus, it will have a higher likelihood of being be kept as a qualified virtual sample, as shown in Eq (31).

4.   Experiments

This section illustrates the experimental procedures designed for the purposes of this study. We compare the proposed method with other VSG methods with two bladder cancer cases. The experiments were implemented with Python 3.8.10. The experimental results are discussed in the following section.

4.1.   Case description

Two bladder cancer cases are examined in our experiments. One case provided by Liao ^[6] is used to classify whether a patient suffers from bladder cancer, and the other case was obtained in ^[7] to predict a bladder cancer patient's resistance to radiotherapy.

4.1.1.   Bladder cancer (BC) case

The BC dataset comprises nine bladder cancer patients and nine healthy persons, as listed in Table 2. The first nine examples are bladder cancer patients defined as class 1, and the last nine samples are healthy persons defined as class 0. Thirteen gene proteins MDR, Topo II, Rb, EGFR, Neu, c-ErbB-3, c-ErbB-4, Cyclin A, Cyclin D1, P16, Cdc 2, Bcl-2, and Bax extracted in bladder cell lines are set as the input attributes.

4.1.2.   Radiotherapy of bladder cancer (RBC) case

The RBC dataset has 36 examples used to predict the resistance to Co-60 radiotherapy for bladder cancer cells. Each example has thirteen gene proteins, as mentioned above, and one additional attribute, the energy of gamma radiation from the Co-60 isotope, as listed in Table 3. The example output represents the patients' resistance to radiotherapy treatment.

4.2.   An example using the proposed DB-MTD method

We randomly drew 10 samples in the RBC dataset as an example to explain the different scenarios in the proposed DB-MTD method. The drawn samples were set as a training dataset, as listed in Table 4. The implementation procedure for the DB-MTD method is explained in the following discussion.

We use the Min-Max data normalization process from Eq (13) to transform the training data domain into [0, 1]. The lower bound a and upper bound b of the input attributes can be obtained from Eqs (23) and (24), as shown in Table 5. The Rb, Cyclin D1, Bax attributes are determined to be the trapezoidal MF based on our method, where we could obtain two CLs. The remaining attributes were defined as the triangular MF.

To explain the PAM used to select the virtual samples, we use the MDR and Rb attributes as examples. The virtual sample (vs) could be created based on different MFs in the two attributes, as shown in Figure 11. Then, the vs was randomly generated within [a, b] to calculate the possibility MF(vs). When the MF(vs) was greater than rs~Uniform(0, 1), then vs could be regarded as an appropriate virtual sample.

After performing the PAM, the virtual samples could be generated, as shown in Table 6. The virtual sample output was derived using the built BPNN model with small datasets. Then, we added the generated virtual examples into the original training dataset to build a new training dataset.

4.3.   Experimental procedure

In our experiment, we used the random sampling method to create different scenarios for small sample sizes. The different training datasets were created by sampling from the original datasets with the BC dataset ranging in size from 3 to 17 and training data sizes {5, 10, 15, 20, 25, 30, 35} for the RBC dataset, and the remaining data was set as a testing dataset. Then, we used Eqs (23) and (24) to calculate the lower and upper bounds of the attributes of the training dataset. The virtual sample could be randomly generated from the evaluated range. In addition, we calculated the MF values of the virtual samples to select virtual samples using the PAM. The virtual data size was set as 100. The f-k-l BPNN network was set as a learning model in the experiment, where f represents f attributes of the training dataset as the number of nodes in the input layer; k is the number of nodes in the hidden layer, and l is the number of nodes in the output layer. The testing dataset was inputted into the learned BPNN model with 100 nodes in the hidden layer and 50 epochs in the training process. In the BC dataset, we used classification accuracy as the evaluation metric for the BPNN model. In the RBC dataset, the prediction performance of the BPNN model was evaluated using the root mean squared error (RMSE), which was defined as:

where $tes{t_N}$ is the number of testing samples; ${y_i}$ is the actual value, and ${\hat y_i}$ is the predicted values of the ith testing sample. A total of 100 iterations were used in this experiment. After performing the experiments, we calculated the average and standard deviation of the classification accuracy for the BC dataset and the average and standard deviation of the RMSE for the RBC dataset.

4.4.   Experimental results

To verify the effectiveness of the proposed DB-MTD method, we compared it using four methods: RAW (using raw training samples), MTD (generating virtual samples based on the triangular MF), GD (generating virtual samples base on a normal distribution), and MD-MTD (generating virtual samples based on multiple distributions). For the BC dataset, the average (Avg-accuracy) and the standard deviation (SD-accuracy) of the classification accuracies are respectively shown in Figures 12 and 13. The improvements (%) in classification accuracy using the proposed DB-MTD method are listed in Table 7. In Figure 12, it can be seen that the results for most of the training subsets indicated superior classification accuracies as compared to those for the other methods under consideration. The SD-accuracy in these methods had smaller variances that were close to each other with different training data sizes. For example, when the training sample size was 3, the Avg-accuracy using the DB-MTD method was improved from 86.2 to 89.9%, where the improvement was approximately 3.7%, as shown in Table 7. For the RBC dataset, the average (Avg-RMSE) and the standard deviation (SD-RMSE) of RMSE are respectively shown in Figures 14 and 15. In Figure 14, the Avg-RMSE using DB-MTD method was improved from 24.79 to 24.47 at a training sample size of 5, where the improvement was approximately 0.32. The other improvements are shown in Table 8.

4.5.   The statistical test confirming the experimental results

A paired t-test was used to verify the significance of the learning improvements using the proposed DB-MTD method in BC and RBC datasets. The p-values of the paired t-test for the four methods and the DB-MTD method are listed in Tables 9 and 10, where the symbol "*" indicates that the prediction results demonstrated a statistically significant difference (p-value < 0.05). In these tables, the results show statistical significance in terms of the difference in the Avg-accuracy for the BC dataset and the Avg-RMSE for the RBC dataset in some training data sizes. For example, when the training data size was set at 10 in the BC dataset, "0.02*" represents that the prediction results based on the DB-MTD demonstrated a significant difference in learning performance for the RAW method. Although the DB-MTD method was only statistically superior to the other methods in some training sizes, the Avg-accuracy of the DB-MTD method was greater than that when using the methods for BC dataset. It is the same results based on the Avg-RMSE were obtained for the RBC dataset, as shown in Table 10.

5.   Conclusions

In many fields, the virtual sample generation (VSG) approach has been regarded as an effective method to improve the learning performance of machine learning models with small sample sizes. The popular MTD method has been widely applied in many VSG studies to generate virtual samples within the estimated data range to extend the amount of the original training data. The problem with the MTD method is that extreme values have serious effects, including making it difficult to estimate the central location and data skewness. To deal with this problem, a new distance-based MTD (DB-MTD) method based on a defined distance function between data was proposed in the present work to improve the degree of data diffusion. The distance became the basis of the coefficient of skewness, which made the inferred distribution close to the pattern in the existing samples. The proposed DB-MTD coefficient could more effectively reduce the excessive diffusion problem that existed in the original MTD method based on a premise where only the triangular membership function was considered.

As to the limitations of this method, the proposed DB-MTD method can be used for small data with continuous variables, but it is not suitable for categorical or discrete variables. In our future research, two research directions can be considered: One is finding discrete density functions to create virtual samples for categorical or discrete attributes. The other is validating the proposed method on practical medical datasets.

Acknowledgments

This study is supported by the Ministry of Science and Technology and National Taipei University of Nursing and Health Sciences, Taiwan. This research was funded by the Ministry of Science and Technology Taiwan under MOST 110-2222-E-227-001-MY2.

Conflict of interest

The authors declare that there are no conflicts of interest.