Figure 1.
Venn diagram presents the feature selection scenario where the two candidate features $\bar{F_1}$ and $\bar{F_2}$ have the same new classification information.
Citation: Amanda Swearingen, Jessica Price, Janet Silbernagel, Randy Swaty, Nicholas Miller. State-and-transition simulation modeling to compare outcomes of alternative management scenarios under two natural disturbance regimes in a forested landscape in northeastern Wisconsin, USA[J]. AIMS Environmental Science, 2015, 2(3): 737-763. doi: 10.3934/environsci.2015.3.737
| [1] | Ming-Zhen Xin, Bin-Guo Wang, Yashi Wang . Stationary distribution and extinction of a stochastic influenza virus model with disease resistance. Mathematical Biosciences and Engineering, 2022, 19(9): 9125-9146. doi: 10.3934/mbe.2022424 |
| [2] | Tingting Xue, Xiaolin Fan, Zhiguo Chang . Dynamics of a stochastic SIRS epidemic model with standard incidence and vaccination. Mathematical Biosciences and Engineering, 2022, 19(10): 10618-10636. doi: 10.3934/mbe.2022496 |
| [3] | Fathalla A. Rihan, Hebatallah J. Alsakaji . Analysis of a stochastic HBV infection model with delayed immune response. Mathematical Biosciences and Engineering, 2021, 18(5): 5194-5220. doi: 10.3934/mbe.2021264 |
| [4] | Kangbo Bao, Qimin Zhang, Xining Li . A model of HBV infection with intervention strategies: dynamics analysis and numerical simulations. Mathematical Biosciences and Engineering, 2019, 16(4): 2562-2586. doi: 10.3934/mbe.2019129 |
| [5] | Yang Chen, Wencai Zhao . Dynamical analysis of a stochastic SIRS epidemic model with saturating contact rate. Mathematical Biosciences and Engineering, 2020, 17(5): 5925-5943. doi: 10.3934/mbe.2020316 |
| [6] | Helong Liu, Xinyu Song . Stationary distribution and extinction of a stochastic HIV/AIDS model with nonlinear incidence rate. Mathematical Biosciences and Engineering, 2024, 21(1): 1650-1671. doi: 10.3934/mbe.2024072 |
| [7] | H. J. Alsakaji, F. A. Rihan, K. Udhayakumar, F. El Ktaibi . Stochastic tumor-immune interaction model with external treatments and time delays: An optimal control problem. Mathematical Biosciences and Engineering, 2023, 20(11): 19270-19299. doi: 10.3934/mbe.2023852 |
| [8] | Tianfang Hou, Guijie Lan, Sanling Yuan, Tonghua Zhang . Threshold dynamics of a stochastic SIHR epidemic model of COVID-19 with general population-size dependent contact rate. Mathematical Biosciences and Engineering, 2022, 19(4): 4217-4236. doi: 10.3934/mbe.2022195 |
| [9] | Divine Wanduku . The stochastic extinction and stability conditions for nonlinear malaria epidemics. Mathematical Biosciences and Engineering, 2019, 16(5): 3771-3806. doi: 10.3934/mbe.2019187 |
| [10] | Xiaomeng Wang, Xue Wang, Xinzhu Guan, Yun Xu, Kangwei Xu, Qiang Gao, Rong Cai, Yongli Cai . The impact of ambient air pollution on an influenza model with partial immunity and vaccination. Mathematical Biosciences and Engineering, 2023, 20(6): 10284-10303. doi: 10.3934/mbe.2023451 |
Recently, classification systems have a wide range in many fields such as text classification, intrusion detection, bio-informatics, and image retrieval [1]. Unfortunately, a huge amount of data which may include irrelevant or redundant features is one of the main challenges of these systems. The negative effect of these undesirable features reduces the classification performance [2]. For this reason, reducing the number of features by finding an effective subset of features is an important task in classification systems [2]. Feature reduction has two techniques: feature selection and feature extraction [3]. Both reduce a high-dimensional dataset into a representative feature subset of low-dimensional. Feature extraction is effective when the original features fail to discriminate the classes [4], but it requires extra computation. Moreover, it changes the true meaning of the original features. In contrast, the feature selection preserves the true meaning of the selected features, which is important for some classification systems [5]. Furthermore, the result of FS is more understandable for domain experts [6].
FS tries to find the best feature subset which represents the dataset well and improves the performance of classification systems [7]. It can be classified into three approaches [6]: wrapper, embedded, and filter. According to an evaluation strategy, wrapper and embedded are called classifier-dependent approaches, while filter is called classifier-independent approach [8]. In this paper, we use the filter approach according to its advantages over wrapper or embedded approaches in terms of efficiency, simplicity, scalability, practicality, and classifier-independently [6,9]. Filter approach is a pre-processing task which finds the highly ranked features to be the input of classification systems [7,10]. There are two criteria to rank features: feature relevance, and feature redundancy [11]. Feature relevance is related to how features discriminate different classes, while feature redundancy is related to how features share the same information of each other [12]. To define these criteria, filter approach uses many weighting functions which rank features based on their significance [10] such as correlation [13], mutual information (MI) [14]. MI overcomes the weakness of correlation, whereas, correlation is suitable only for linear relationship and numerical features [1]. MI is suitable for any kind of relationship such as linear and non-linear. Moreover, MI deals with both numerical and categorical features [1].
Although MI has been widely used in many methods to find the best feature subset that maximizes the relevancy between the candidate feature and class label, and minimizes the redundancy between the candidate feature and pre-selected features [15]. The main limitations of these methods are: (1) difficult to indicate the best candidate features with the same new classification information [16], (2) difficult to deal with continuous features without information loss [17], and (3) consider the inner-class information only [18]. In this paper, we integrate fuzzy concept with mutual information to propose a new FS method called Fuzzy Joint Mutual Information Maximization (FJMIM). The fuzzy concept helps the proposed method to exploit all possible information of data where it can deal with any numerical data and extract the inner and outer-class information. Moreover, the objective function of FJMIM can overcome the feature overestimation problem which happens when the candidate feature be completely correlated with some of pre-selected features and does not depend on the majority of the subset at the same time [8].
The rest of this paper is organized as follows: Section 2 presents the basic measures of fuzzy information theory. Then, we present the proposed method in section 3. After that, the experiment design was presented in section 4, followed by the results and discussion in section 5. Finally, section 6 concludes the paper.
For the purpose of measuring the significance of features, information theory introduced many information measures such as entropy, and mutual information. To enhance these measures, fuzzy concept is used to estimate new extensions of information measures based on fuzzy equivalence relations such as fuzzy entropy, and fuzzy mutual information [19,20]. Fuzzy entropy measures the average amount of uncertainty of fuzzy relation in order to estimate its discriminative power, while fuzzy mutual information measures the shared amount of information between two fuzzy relations. In the following, we present the basic measures of fuzzy information theory:
Given a dataset $ D = F\cup C $, where $ F $ is a set of $ n $ features, and $ C $ is the class label. Let $ {\bar{F}} = \{a_1, a_2, ….., a_m\} $ be a feature of $ m $ samples, where $ {\bar{F}} \in F $. Let $ S $ is the feature subset with $ d $ of selected features, and the remaining set is $ \{F-S\} $, where $ \bar{F}_f \in F-S $ and $ \bar{F}_s \in S $. Based on the fuzzy equivalence relation $ R_{\bar{F}} $ on $ \bar{F} $, the feature $ \bar{F} $ can be represented by the relation matrix $ M(R_{\bar{F}}) $.
|
$ M(RˉF)=(r11r12…r1mr21……r2m…………rm1rm2…rmm) $
|
(2.1) |
where $ r_{ij} = R_{\bar{F}}(a_i, a_j) $ is the fuzzy equivalence relation between two samples $ a_i $ and $ a_j $.
In this paper, the used fuzzy equivalence relation between two elements $ a_i $ and $ a_j $ is defined as [21]:
|
$ RˉF(ai,aj)=exp−‖ai−aj‖ $
|
(2.2) |
Fuzzy equivalence class of sample $ a_i $ on $ R_{\bar{F}} $ can be defined as:
|
$ [ai]RˉF=[ri1]a1+[ri2]a2+⋯+[rim]am $
|
(2.3) |
Fuzzy entropy of feature $ \bar{F_1} $ based on fuzzy equivalence relation is defined as:
|
$ H(ˉF)=1mm∑i=1logm|[ai]RˉF| $
|
(2.4) |
where $ \lvert[a_i]_{R_{\bar{F}}}\rvert = \sum_{i = 1}^{m} r_{ij} $.
Let $ \bar{F_1} $ and $ \bar{F_2} $ be two features of $ F $, fuzzy joint entropy of $ \bar{F_1} $ and $ \bar{F_2} $ is defined as:
|
$ H(¯F1,¯F2)=H(R¯F1,R¯F2)=1mm∑i=1logm|[ai]R¯F1∩[ai]R¯F2| $
|
(2.5) |
Fuzzy conditional entropy of $ \bar{F_1} $ given $ \bar{F_2} $ is defined as
|
$ H(¯F1|¯F2)=H(R¯F1|R¯F2)=1mm∑i=1log|[ai]R¯F2||[ai]R¯F1∩[ai]R¯F2| $
|
(2.6) |
Fuzzy Mutual information between two features $ \bar{F_1} $ and $ \bar{F_2} $ is defined as:
|
$ I(¯F1;¯F2)=I(R¯F1;R¯F2)=1mm∑i=1logm|[ai]R¯F1∩[ai]R¯F2||[ai]R¯F1.[ai]R¯F2| $
|
(2.7) |
Fuzzy conditional mutual information between feature $ \bar{F_1} $ and $ \bar{F_2} $ given class $ C $ is defined as:
|
$ I(¯F1;¯F2|C)=H(¯F1|C)+H(¯F2|C)−H(¯F1,¯F2|C) $
|
(2.8) |
Fuzzy joint mutual information between two features $ \bar{F_1} $, $ \bar{F_2} $ and class $ C $ is defined as:
|
$ I(¯F1,¯F2;C)=I(¯F1;C)+I(¯F2;C|¯F1) $
|
(2.9) |
Fuzzy interaction information between among $ \bar{F_1} $, $ \bar{F_2} $ and $ C $ is defined as:
|
$ I(¯F1;¯F2;C)=I(¯F1;C)+I(¯F2;C)−I(¯F1,¯F2;C) $
|
(2.10) |
In this section, we presented the general theoretical frameworks of different feature selection methods based on mutual information. Then, we studied the limitation of previous work. Finally, we introduced the proposed method.
Brown et al. [22] studied the exist feature selection methods based on MI and analyzed the different criteria to propose the following theoretical framework of these methods.
|
$ J(ˉFf)=I(ˉFf;C)−β∑ˉFs∈SI(ˉFf;ˉFs)+γ∑ˉFs∈SI(ˉFf;ˉFs|C) $
|
(3.1) |
This framework is a linear combination of three terms: relevance, redundancy, and conditional that measures the individual predictive power of the feature, the unconditional relation, and the class-conditional relation, respectively. The criteria of different feature selection based on MI depends on the value of $ \beta $ and $ \gamma $. MIM ($ \beta = \gamma = 0 $) [23] is the simplest FS method based on MI. It considers only the relevance relation only. However, It may suffer from the redundant features. MIFS ($ \gamma = 0 $) [24] introduced two criteria to estimate the feature relevance and redundancy. An extension of MIFS, called MIFS-U [24] is proposed to improve the redundancy term of MIFS by considering the uniform distribution of the information. However, Both MIFS and MIFS-U still require an input parameter $ \beta $. To avoid this limitation, MRMR ($ \beta = \frac{1}{\lvert S\rvert} $, $ \gamma = 0 $) [25] introduced the mean of the redundancy term as automatic value to the input parameter ($ \beta $). JMI ($ \beta = \gamma = \frac{1}{\lvert S\rvert} $) [26] extended MRMR to extract the benefit of conditional term. In addition, Brown et al. [22] introduced also a similar non-linear framework to represent some methods as CMIM method [27]. According to [22], CMIM can be written as:
|
$ Jcmim=minˉFs∈S[I(ˉFf;C|ˉFs)]=I(ˉFf;C)−maxˉFs∈S[I(ˉFf;ˉFs)−I(ˉFf;ˉFs|C)] $
|
(3.2) |
The reason of the non-linear relation on CMIM returns to the using of $ max $ operation. Similar to CMIM, JMIM [8] introduces a non-linear relation as follows:
|
$ Jjmim=minˉFs∈S[I(ˉFs;C)+I(ˉFf;C|ˉFs)]=I(ˉFf;C)−maxˉFs∈S[I(ˉFf;ˉFs)−I(ˉFf;ˉFs|C)−I(ˉFs;C)] $
|
(3.3) |
Although MI has been widely used in many feature selection methods such as MIFS [24], JMI [26], mRMR [25], DISR [28], IGFS [29], NMIFS [30] and MIFS-ND [31]. These methods suffer from the overestimation of the feature significance problem [8]. For this reason, Bennasar et al. [8] proposed JMIM method to address the overestimation of the feature significance problem. However, it may fail to select the best candidate features if they have the same new classification information. To illustrate this problem, Figure 1 shows the FS scenario, where $ \bar{F_1} $ and $ \bar{F_2} $ are two candidate features, $ \bar{F_s} $ is the pre-selected feature subset, and $ C $ is the class label. $ \bar{F_1} $ is partially redundant with $ \bar{F_s} $, while $ \bar{F_2} $ is independent to $ \bar{F_s} $. Suppose that $ \bar{F_1} $ and $ \bar{F_2} $ have the same new classification information $ I(\bar{F_1};C|\bar{F_s}) $ = (area 3) and $ I(\bar{F_2};C|\bar{F_s}) $ = (area 5) respectively. In this case, JMIM may fail to indicate the best feature where $ I(\bar{F_1}, \bar{F_s}; C) $ and $ I(\bar{F_1}, \bar{F_s}; C) $ are equal.
Unfortunately, JMIM also shares some limitations with the previous methods. Firstly, it can not directly estimate MI between continuous features [17]. To address this limitation, there are two methods were introduced. One is to estimate MI based on Parzen window [18], but it is inefficient in high-dimensional feature spaces with spare samples [17]. Moreover, its performance depends on the used window function which requires a window width parameter [32]. The other one is to discretize continuous features before estimating MI [33], but it may cause information loss [34]. Secondly, JMIM depends only on inner-class information without considering outer-class information [18].
Motivated by the previous limitation of JMIM, we proposed a new FS method, called Fuzzy Joint Mutual Information Maximization (FJMIM). Both of FJMIM and JMIM depends on "maximum of the minimum" approach. The main difference is that JMIM maximize the joint mutual information of the candidate feature and pre-selected feature subset with class, whereas FJMIM maximizes the joint mutual information of the candidate feature and pre-selected feature subset with class without considering the class-relevant redundancy. To illustrate the difference in Figure 1, JMIM depends on the union of areas 2, 4 and 3, while FJMIM depends on the union of areas 2 and 3. The proposed method discarded the class-relevant redundancy (area 4) because it can reduce the predictive ability of the feature subset when a candidate feature is selected [15]. On the other hand, integrating fuzzy concept with MI has many benefits. Firstly, using a fuzzy concept helps to deal directly with continuous features. Furthermore, it enables MI to take the advantages of inner and outer-class information [35]. Moreover, FS methods based on fuzzy concept are more robust toward any change of the data than methods based on probability concept [36].
According to FJMIM, the candidate feature $ \bar{F}_f $ must satisfy the following condition
|
$ ˉFf=argmaxˉFf∈F−S(minˉFs∈S(I(ˉFf,ˉFs;C)−I(ˉFf;ˉFs;C))) $
|
(3.4) |
FJMIM also can be written according to the non-linear framework as follows:
|
$ ˉFf=2∗I(ˉFf;C)−maxˉFs∈S[I(ˉFf;ˉFs)−I(C;ˉFs)−I(ˉFf;ˉFs|C)−I(ˉFf;C|ˉFs)] $
|
(3.5) |
The proposed method can be summarized to find the best feature subset of size $ d $ as follows:
Input: $ F $ is a set of $ n $ features, $ C $ is the class label, and $ d $ is the number of selected features.
Step 1: Initialize the empty selected feature subset $ S $.
Step 2: Update the selected feature set $ S $ and the feature set $ F $.
Step 2.1: Compute $ I(\bar{F}; C) $ for all $ \bar{F} $ in the feature set $ F $.
Step 2.2: Add the feature $ \bar{F} $ that maximizes $ I(\bar{F}; C) $ to the selected feature set $ S $.
Step 2.3: Remove the feature $ \bar{F} $ from the feature set $ F $.
Step 3: Repeat until $ \lvert S \lvert = d $
Step 3.1: Add the feature $ \bar{F} $ that satisfies
$arg \max_{\bar{F}_f \in F-S} (\min_{\bar{F}_s \in S} (I(\bar{F}_f, \bar{F}_s; C)-I(\bar{F}_f; \bar{F}_s; C))) $
to the selected feature set $ S $.
Step 3.2: Remove the feature $ \bar{F} $ from the feature set $ F $.
Output: Return the selected feature set $ S $.
Success of FS methods depends on different criteria such as classification performance, and stability [11]. Consequently, we design the experiment based on these criteria (Figure 2). To clarify our improvement, we compared our proposed method FJMIM, with four conventional methods (CMIM [27], JMI [26], QPFS [37], Relief [38]) and five state-of-the-art methods (CMIM3 [39], JMI3 [39], JMIM [8], MIGM [40], and WRFS [41]). The compared methods can be divided into two groups: FS based on fuzzy concept and FS based on probability concept. For the methods which depend on probability concept, data discretization is required as a pre-processing step prior to the FS process. So, the continuous features are transformed into ten bins using EqualWidth discretization [42]. Then, we selected feature subset from all methods based on threshold which is defined as the median position of the ranked features (or the nearest integer position when the number of ranked features is even).
FS plays an important role to improve the classification performance. There are many measures to evaluate classification performance such as classification accuracy, precision, F-measure, area-under-ROC (AUC), and area-under-PRC (AUCPR) [43]. To clarify our improvement, popular classifiers were used in this study such as Naive Bayes (NB), Support Vector Machine (SVM), and 3-Nearest Neighbors (KNN) [44]. The average classification performance measures were computed by 10-fold cross-validation approach [45].
Another important evaluation criterion for FS is stability. FS stability measures the impact of any change in the input data on FS result [46]. In this study, we measure the impact of noise on the selected feature subset. Firstly, we produce the noise using standard deviation and the normal distribution of each feature [47]. Then, we injected 10% of the data by adding noise. After that, we repeated this step ten times. Each time produces a different sequence of selected features. Finally, we computed the stability of each method using Kuncheva stability index [48].
Our experiment was conducted using 13 datasets from UCI machine learning repository [49]. Table 1 presents a brief description about these datasets.
| No. | Datasets | Instances | Features | Classes |
| 1 | Acute Inflammations | 120 | 6 | 2 |
| 2 | Arrhythmia | 452 | 279 | 13 |
| 3 | Blogger | 100 | 6 | 2 |
| 4 | Diabetic Retinopathy Debrecen (DRD) | 1151 | 20 | 2 |
| 5 | Hayes-Roth | 160 | 5 | 3 |
| 6 | Indian Liver Patient Dataset (ILPD) | 583 | 10 | 2 |
| 7 | Lenses | 24 | 4 | 3 |
| 8 | Lymphography | 148 | 18 | 4 |
| 9 | Congressional Voting Records (CVR) | 435 | 16 | 2 |
| 10 | Sonar | 208 | 60 | 2 |
| 11 | Thoracic Surgery | 470 | 17 | 2 |
| 12 | Wilt | 4889 | 6 | 2 |
| 13 | Zoo | 101 | 17 | 7 |
DownLoad:
CSV
1) Accuracy: A paired two-tailed t-test is employed between FJMIM and other compared methods. The notations ([=], [+], and [-]) indicate the statistically significant (5%) that the proposed method (equals, wins, and losses) other methods. According to NB classifier, FJMIM achieved the maximum average accuracy with score 78.02%, while Relief achieved the minimum average accuracy with score 75.34% (Table 2). The proposed method outperformed compared methods in the range from 0.06 to 2.68%. In SVM classifier, FJMIM outperformed other methods with score 80.03%, while Relief achieved the minimum average accuracy with score 77.33% (Table 3). The proposed method outperformed compared methods in the range from 0.69 to 2.7%. Similarly with KNN classifier, FJMIM kept the maximum average accuracy by 81.45%, while Relief achieved the minimum average accuracy with score 77.98% (Table 4). The proposed method outperformed compared methods in the range from 0.57 to 3.47%. Across all datasets, Figure 3(a) shows the distribution of the average accuracy values of all used classifiers. In a detailed box-plot, the box represents upper and lower quartiles, while the black circle represents the median. The box-plot confirms that FJMIM is more consistent and outperformed other compared methods. Figure 3(b) shows the average accuracy of the three used classifiers. FJMIM achieved the best accuracy, followed by QPFS, JMI3, both of JMI and CMIM, both of CMIM3 and JMIM, MIGM, WRFS, and Relief respectively. The proposed method outperformed compared methods in the range from 0.6 to 2.9%.
| Dataset | CMIM | CMIM3 | JMI | JMI3 | JMIM | MIGM | QPFS | Relief | WRFS | FJMIM |
| Acute Inflammations | 100±0[=] | 100±0[=] | 100±0[=] | 100±0[=] | 100±0[=] | 99.75±2.5[=] | 99.75±2.5[=] | 100±0[=] | 90.83±8.5[+] | 100±0 |
| Arrhythmia | 60.64±7.01[=] | 59.22±6.17[=] | 59.42±6.49[=] | 59.58±6.58[=] | 60.62±6.93[=] | 58.98±6.39[=] | 63.7±6.57[-] | 64.94±6.33[-] | 59.31±6.3[=] | 59.74±6.43 |
| Blogger | 61.9±10.7[=] | 61.9±10.7[=] | 61.9±10.7[=] | 64.9±10.2[=] | 61.9±10.7[=] | 61.9±10.7[=] | 61.9±10.7[=] | 65.4±7.84[=] | 66.1±9.09[=] | 65.9±10.26 |
| DRD | 57.59±4.95[=] | 60.66±4.69[-] | 60.3±5.04[-] | 60.6±4.58[-] | 57.59±4.95[=] | 60.3±5.04[-] | 61.19±4.49[-] | 60.61±5.06[-] | 57.47±4.98[=] | 57.56±4.83 |
| Hayes-Roth | 45.81±10.77[=] | 45.81±10.77[=] | 45.81±10.77[=] | 45.81±10.77[=] | 45.81±10.77[=] | 45.81±10.77[=] | 49.37±10.53[=] | 46.31±11.48[=] | 45.81±10.77[=] | 49.37±10.53 |
| ILPD | 68.99±5.68[=] | 67.33±5.87[=] | 68.99±5.68[=] | 67.33±5.87[=] | 68.99±5.68[=] | 68.99±5.68[=] | 68.73±5.64[=] | 64.96±6.45[+] | 70.11±6.74[=] | 69.54±5.75 |
| Lenses | 86.83±21.88[=] | 86.83±21.88[=] | 86.83±21.88[=] | 86.83±21.88[=] | 86.83±21.88[=] | 86.83±21.88[=] | 86.83±21.88[=] | 54.83±30.73[+] | 86.83±21.88[=] | 86.83±21.88 |
| Lymphography | 79.02±10.57[=] | 82.05±9.59[=] | 79.02±10.57[=] | 77.93±10.87[=] | 78.27±10.98[=] | 77.25±11.38[=] | 79.02±10.57[=] | 82.61±9.77[=] | 76.91±11.64[+] | 81.5±10.29 |
| CVR | 94.3±3.15[=] | 94.32±3.12[=] | 93.75±3.57[=] | 93.75±3.57[=] | 93.75±3.57[=] | 93.75±3.57[=] | 94.32±3.26[=] | 93.82±3.28[=] | 93.52±3.18[=] | 93.75±3.57 |
| Sonar | 75.05±8.33[=] | 75.42±9.08[=] | 75.75±8.85[=] | 76.23±9.99[=] | 72.2±9.93[=] | 72.14±9.54[+] | 76.42±8.69[=] | 74.25±8.33[=] | 77.91±8.74[=] | 77±8.98 |
| Thoracic Surgery | 83.83±3.1[=] | 83.38±3.09[=] | 83.38±3.09[=] | 83.85±2.24[=] | 83.38±3.09[=] | 84.64±1.83[=] | 83.55±3.08[=] | 83.06±2.77[=] | 83.85±2.74[=] | 83.38±3.09 |
| Wilt | 94.61±0.06[=] | 94.61±0.06[=] | 94.61±0.06[=] | 94.61±0.06[=] | 94.61±0.06[=] | 94.61±0.06[=] | 94.61±0.06[=] | 94.61±0.06[=] | 94.61±0.06[=] | 94.61±0.06 |
| Zoo | 95.15±5.74[=] | 96.05±5.6[=] | 96.03±5.66[=] | 96.03±5.66[=] | 96.03±5.66[=] | 96.93±5.42[=] | 94.08±6.6[=] | 94.08±6.6[=] | 92.96±7.05[=] | 95.05±5.87 |
| Average | 77.21 | 77.51 | 77.37 | 77.50 | 76.92 | 77.07 | 77.96 | 75.34 | 76.63 | 78.02 |
DownLoad:
CSV
| Dataset | CMIM | CMIM3 | JMI | JMI3 | JMIM | MIGM | QPFS | Relief | WRFS | FJMIM |
| Acute Inflammations | 98.33±3.35[=] | 99.67±1.64[=] | 98.33±3.35[=] | 98.33±3.35[=] | 98.33±3.35[=] | 99.42±2.44[=] | 99.42±2.44[=] | 100±0[+] | 89.92±9.5[+] | 99.42±2.44 |
| Arrhythmia | 66.09±6.81[=] | 64.08±6.11[=] | 65.01±5.96[=] | 65.7±6.23[=] | 66.14±6.61[=] | 65.23±6.39[=] | 66.82±6.2[=] | 67.22±6.05[=] | 65.23±6.15[=] | 67.11±6.7 |
| Blogger | 68±4.02[=] | 67.7±4.89[=] | 68±4.02[=] | 67.4±5.62[=] | 68±4.02[=] | 68±4.02[=] | 67.7±4.89[=] | 67.7±4.89[=] | 68±4.02[=] | 67.7±4.89 |
| DRD | 68.03±3.92[=] | 67.55±4.07[=] | 67.15±3.85[=] | 67.03±3.95[=] | 68.03±3.92[=] | 67.19±3.8[=] | 65.1±4.33[+] | 61.91±4.51[+] | 67.7±4[=] | 67.96±4.23 |
| Hayes-Roth | 49.94±12.83[=] | 49.94±12.83[=] | 49.94±12.83[=] | 49.94±12.83[=] | 49.94±12.83[=] | 49.94±12.83[=] | 52±11.61[=] | 48.75±13.21[=] | 49.94±12.83[=] | 52±11.61 |
| ILPD | 71.36±0.73[=] | 71.36±0.73[=] | 71.36±0.73[=] | 71.36±0.73[=] | 71.36±0.73[=] | 71.36±0.73[=] | 71.36±0.73[=] | 71.36±0.73[=] | 71.36±0.73[=] | 71.36±0.73 |
| Lenses | 76.17±22.13[=] | 76.17±22.13[=] | 76.17±22.13[=] | 76.17±22.13[=] | 76.17±22.13[=] | 76.17±22.13[=] | 76.17±22.13[=] | 54.5±30.42[+] | 76.17±22.13[=] | 76.17±22.13 |
| Lymphography | 81.57±10.06[=] | 80.56±9.09[=] | 81.42±10.15[=] | 84.55±8.84[=] | 82.02±10.48[=] | 79.06±10.44[+] | 81.57±10.11[=] | 80.3±9[=] | 74.25±11.45[+] | 85.5±8.3 |
| CVR | 94.09±3.33[=] | 94.46±3.37[=] | 94.3±3.34[=] | 94.32±3.32[=] | 94.3±3.34[=] | 94.3±3.34[=] | 94±3.44[=] | 94.43±3.41[=] | 94.37±3.41[=] | 94.3±3.34 |
| Sonar | 79.34±8.37[=] | 80.48±8.05[=] | 80.18±8.33[=] | 78.46±8.44[=] | 80.54±8.43[=] | 76.93±8.71[=] | 79.91±7.73[=] | 82.1±8.07[=] | 77.41±8.41[=] | 80.67±7.88 |
| Thoracic Surgery | 85.11±0[=] | 85.11±0[=] | 85.11±0[=] | 85.11±0[=] | 85.11±0[=] | 85.11±0[=] | 85.11±0[=] | 85.11±0[=] | 85.11±0[=] | 85.11±0 |
| Wilt | 97.62±0.57[+] | 97.97±0.49[=] | 97.97±0.49[=] | 97.97±0.49[=] | 97.62±0.57[+] | 97.97±0.49[=] | 94.61±0.06[+] | 97.97±0.49[=] | 97.62±0.57[+] | 97.98±0.49 |
| Zoo | 95.75±5.12[=] | 93.88±6.72[=] | 89.39±5.72[+] | 89.49±5.64[+] | 89.49±5.64[+] | 94.06±6.9[=] | 95.27±5.85[=] | 93.89±6.96[=] | 90.49±7.96[+] | 95.06±6.2 |
| Average | 79.34 | 79.15 | 78.79 | 78.91 | 79.00 | 78.83 | 79.16 | 77.33 | 77.51 | 80.03 |
DownLoad:
CSV
| Dataset | CMIM | CMIM3 | JMI | JMI3 | JMIM | MIGM | QPFS | Relief | WRFS | FJMIM |
| Acute Inflammations | 99.5±1.99[=] | 100±0[=] | 99.5±1.99[=] | 99.5±1.99[=] | 99.5±1.99[=] | 100±0[=] | 100±0[=] | 100±0[=] | 99.67±1.64[=] | 100±0 |
| Arrhythmia | 58.27±5.17[+] | 57.63±5.17[+] | 57.79±5.34[+] | 58.32±5.08[+] | 58.94±5.13[=] | 57.67±5.21[+] | 60.73±4.64[=] | 65.13±4.43[-] | 58.39±5.22[+] | 61.11±5.23 |
| Blogger | 72.1±11.31[=] | 72.1±11.31[=] | 72.1±11.31[=] | 78.8±10.18[=] | 72.1±11.31[=] | 72.1±11.31[=] | 72.1±11.31[=] | 70.3±11.41[=] | 76.1±12.05[=] | 78.6±11.37 |
| DRD | 64.3±4.19[=] | 62.62±5.29[=] | 63.69±4.58[=] | 61.96±4.68[=] | 64.3±4.19[=] | 63.58±4.56[=] | 60.47±4.64[+] | 63.51±4.63[=] | 64.06±4.4[=] | 64.7±4.23 |
| Hayes-Roth | 59.19±10.41[=] | 59.19±10.41[=] | 59.19±10.41[=] | 59.19±10.41[=] | 59.19±10.41[=] | 59.19±10.41[=] | 63.12±11.32[=] | 57.56±10.82[=] | 59.19±10.41[=] | 63.12±11.32 |
| ILPD | 67.16±5.41[=] | 68.22±5.06[=] | 67.54±5.44[=] | 68.22±5.06[=] | 67.16±5.41[=] | 67.3±5.42[=] | 68.74±5.01[=] | 67.84±5.56[=] | 66.66±5.25[=] | 69.15±5.62 |
| Lenses | 86.83±21.88[=] | 86.83±21.88[=] | 86.83±21.88[=] | 86.83±21.88[=] | 86.83±21.88[=] | 86.83±21.88[=] | 86.83±21.88[=] | 54.17±30.46[+] | 86.83±21.88[=] | 86.83±21.88 |
| Lymphography | 84.61±7.58[=] | 72.61±9.14[+] | 84.61±7.58[=] | 84.16±9.46[=] | 80.99±9.49[=] | 76.62±8.87[=] | 84.61±7.58[=] | 81.44±9.42[=] | 79.74±9.36[=] | 78.87±8.35 |
| CVR | 93.63±3.43[=] | 94.09±3.33[=] | 95.26±3.01[=] | 95.26±3.01[=] | 95.26±3.01[=] | 95.26±3.01[=] | 94.14±3.37[=] | 93.67±3.5[=] | 94.09±2.95[=] | 95.26±3.01 |
| Sonar | 85.09±7.43[=] | 83.51±7.68[=] | 87.74±7.51[=] | 85.53±8.83[=] | 82.7±9.06[=] | 82.4±7.72[+] | 86.15±7.24[=] | 87.95±6.78[=] | 84.17±7.73[=] | 87.65±7.27 |
| Thoracic Surgery | 80.96±3.45[=] | 81.38±3.53[=] | 81.38±3.53[=] | 83.45±3.05[=] | 81.38±3.53[=] | 82.64±3.28[=] | 83.89±2.95[=] | 82.68±3.08[=] | 84.85±0.97[-] | 81.38±3.53 |
| Wilt | 97.69±0.57[+] | 98.12±0.55[=] | 98.12±0.55[=] | 98.12±0.55[=] | 97.69±0.57[+] | 98.12±0.55[=] | 94.44±0.52[+] | 98.12±0.55[=] | 97.69±0.57[+] | 98.12±0.55 |
| Zoo | 91.9±7.21[=] | 92.06±7.35[=] | 92.05±7.24[=] | 92.05±7.24[=] | 92.05±7.24[=] | 91.16±7.48[=] | 91.42±6.94[=] | 91.42±6.94[=] | 90.29±7.45[+] | 94.08±6.6 |
| Average | 80.09 | 79.10 | 80.45 | 80.88 | 79.85 | 79.45 | 80.51 | 77.98 | 80.13 | 81.45 |
DownLoad:
CSV
2) Precision: Figure 4 shows the precision results of NB, SVM, KNN, and their average. FJMIM achieved the highest precision, while Relief achieved the lowest precision. The proposed method outperformed other methods in the range from 0.6 to 12.7% based on NB, from 0.3 to 3.3% based on SVM, and from 2 to 8.2% based on KNN. According to the average of all classifiers, FJMIM achieved the highest precision by 71.2%, while Relief kept the lowest precision by 63.1%. The second-best method achieved by CMIM3, followed by JMI3, QPFS, JMI, JMIM, MIGM, CMIM and WRFS. The results of precision distribution are inconsistent as shown in Figure 5. Both FJMIM and QPFS achieved the best distribution. FJMIM achieved the highest upper quartile and median, while QPFS achieved the highest lower quartile and median. In addition, FJMIM shares the highest upper quartile with JMI, JMIM and MIGM.
3) F-measure: Figure 6 shows the F-measure results of the three used classifiers and their average. FJMIM achieved the highest F-measure by 79.8, 71.5 and 84% on NB, SVM and KNN respectively. Relief achieved the lowest F-measure on NB and SVM, while WRFS achieved the lowest score on SVM. The proposed method outperformed other methods in the range from 0.3 to 16.6% based on NB, from 0.1 to 1.4% based on SVM, and from 0.6 to 15.2% based on KNN. According to the average of all classifiers, FJMIM achieved the highest precision by 78.4% and outperformed other methods in the range from 2.5 to 10.8%. The second-best method achieved by JMI3, followed by QPFS, CMIM3, JMI, JMIM, CMIM, MIGM, WRFS and Relief. Figure 7 shows the distribution of F-measure across all datasets. The box-plot confirms the outperformance of FJMIM compared to other methods.
4) AUC: Figure 8 shows the AUC results of the used classifiers and their average. FJMIM achieved the highest AUC on NB and KNN by 83.9 and 85.2%, while it achieved the second-best score on SVM by 74.8%. On the other hand, Relief achieved the lowest AUC on all classifiers. Although MIGM achieved the best AUC on SVM, FJMIM outperformed on the average of all classifiers. The proposed method outperformed other methods in the range from 0.4 to 4.2%. As shown in the box-plot (Figure 9), the proposed method achieved the highest lower quartile and median values. On the other hand, JMI achieved also the highest lower quartile, while CMIM achieved the highest upper quartile.
5) AUCPR: The highest AUCPR was achieved by both FJMIM and CMIM3 using NB, MIGM using SVM, and FJMIM using KNN (Figure 10). On the other hand, Relief achieved the lowest AUCPR using all classifiers. According to the average of all classifiers, FJMIM achieved the best AUCPR by 81.6%, while Relief kept the lowest AUCPR by 77.2%. The proposed method outperformed other methods in the range from 0.3 to 4.4%. The second-best AUCPR was achieved by CMIM3, followed by MIGM, both QPFS and JMIM, both CMIM and JMI, and both JMI3 and WRFS. Figure 11 shows the distribution of AUCPR across all datasets. It's obvious that CMIM achieved the highest median, lower, and upper quartiles, while FJMIM achieved the highest median, upper quartile.
Figure 12(a) shows the stability of the used FS methods on all datasets. It's obvious that FJMIM is more consistent and stable compared to all other methods. Figure 12(b) confirms the stability of the compared method. FJMIM achieved the highest average stability by 87.8%. The proposed method outperformed other methods in the range from 6.6 to 43%. JMI achieved the second-best position by 81.2%, while Relief achieved the lowest stability by 44.3%.
More detailed results are presented in the appendix section. According to previous results, It is obvious that FJMIM achieves the best results in most measures. This is expected because our proposed method addresses the feature overestimation problem and handle the candidate feature problem well. Moreover, it avoids the discretization step. Another reason is the advantages of both inner and outer class information which FJMIM depends on it. This information helps the proposed method to be more robust toward the noise. On the other hand, the other compared methods are close to FJMIM than Relief that achieved the lowest result. This is because all compared method except Relief depends on mutual information as the proposed method to estimate the significant of features.
In this paper, we propose a new FS method, called, Fuzzy Joint Mutual Information Maximization (FJMIM). The proposed method depends on integrating an improved JMIM objective function with fuzzy concept. The benefits of our proposed method include: 1) The ability to deal directly with discrete and continuous features. 2) The suitability to handle any kind of relations between features such as linear and non-linear relation. 3) The ability to take the advantages of inner and outer class information. 4) The robustness toward the noise. 5) The ability to select the most significant feature subset and avoiding the undesirable features.
To confirm the effectiveness of FJMIM, 13 benchmark datasets have been used to evaluate the proposed method in the term of classification performance (accuracy, precision, F-measure, AUC, and AUCPR) and feature selection stability. According to nine conventional and state-of-the-art feature selection methods, the proposed method achieved promising improvement on feature selection in the terms of classification performance and stability.
In future work, we plan to extend the proposed method to cover multi-label classification problem. Moreover, we plan to study the effectiveness of imbalanced data on the proposed method.
This research has been supported by the National Natural Science Foundation (61572368).
The authors declare no conflict of interest.
Tables A1–A6 show some numerical results of Figures 3(b), 4, 6, 8, 10 and 12(b), while Figures A1–A4 show the statistical results (mean ± standard deviation) of some datasets (DRD, Sonar, Wilt and Zoo) by precision, F-measure, AUC and AUCPR, respectively.
| FS method | Average |
| CMIM | 78.9 |
| CMIM3 | 78.6 |
| JMI | 78.9 |
| JMI3 | 79.1 |
| JMIM | 78.6 |
| MIGM | 78.4 |
| QPFS | 79.2 |
| Relief | 76.9 |
| WRFS | 78.1 |
| FJMIM | 79.8 |
DownLoad:
CSV
| FS method | NB | SVM | KNN | Average |
| CMIM | 0.702 | 0.641 | 0.704 | 0.682 |
| CMIM3 | 0.742 | 0.641 | 0.705 | 0.696 |
| JMI | 0.705 | 0.640 | 0.713 | 0.686 |
| JMI3 | 0.712 | 0.636 | 0.723 | 0.690 |
| JMIM | 0.701 | 0.642 | 0.707 | 0.683 |
| MIGM | 0.701 | 0.637 | 0.708 | 0.682 |
| QPFS | 0.712 | 0.636 | 0.711 | 0.686 |
| Relief | 0.621 | 0.612 | 0.661 | 0.631 |
| WRFS | 0.698 | 0.632 | 0.708 | 0.679 |
| FJMIM | 0.748 | 0.645 | 0.743 | 0.712 |
DownLoad:
CSV
| FS method | NB | SVM | KNN | Average |
| CMIM | 0.722 | 0.712 | 0.818 | 0.750 |
| CMIM3 | 0.795 | 0.712 | 0.752 | 0.753 |
| JMI | 0.724 | 0.712 | 0.823 | 0.753 |
| JMI3 | 0.729 | 0.712 | 0.834 | 0.759 |
| JMIM | 0.719 | 0.714 | 0.821 | 0.751 |
| MIGM | 0.722 | 0.710 | 0.818 | 0.750 |
| QPFS | 0.732 | 0.712 | 0.825 | 0.756 |
| Relief | 0.632 | 0.708 | 0.688 | 0.676 |
| WRFS | 0.716 | 0.701 | 0.826 | 0.748 |
| FJMIM | 0.798 | 0.715 | 0.840 | 0.784 |
DownLoad:
CSV
| FS method | NB | SVM | KNN | Average |
| CMIM | 0.835 | 0.733 | 0.812 | 0.793 |
| CMIM3 | 0.836 | 0.748 | 0.844 | 0.809 |
| JMI | 0.837 | 0.734 | 0.815 | 0.795 |
| JMI3 | 0.831 | 0.729 | 0.805 | 0.788 |
| JMIM | 0.836 | 0.742 | 0.812 | 0.796 |
| MIGM | 0.832 | 0.754 | 0.807 | 0.798 |
| QPFS | 0.838 | 0.710 | 0.807 | 0.785 |
| Relief | 0.804 | 0.707 | 0.801 | 0.771 |
| WRFS | 0.831 | 0.732 | 0.806 | 0.789 |
| FJMIM | 0.839 | 0.748 | 0.852 | 0.813 |
DownLoad:
CSV
| FS method | NB | SVM | KNN | Average |
| CMIM | 0.848 | 0.708 | 0.798 | 0.785 |
| CMIM3 | 0.862 | 0.715 | 0.862 | 0.813 |
| JMI | 0.850 | 0.708 | 0.797 | 0.785 |
| JMI3 | 0.837 | 0.705 | 0.800 | 0.781 |
| JMIM | 0.852 | 0.713 | 0.797 | 0.787 |
| MIGM | 0.857 | 0.724 | 0.827 | 0.803 |
| QPFS | 0.857 | 0.708 | 0.797 | 0.787 |
| Relief | 0.842 | 0.685 | 0.791 | 0.772 |
| WRFS | 0.850 | 0.697 | 0.795 | 0.781 |
| FJMIM | 0.862 | 0.718 | 0.869 | 0.816 |
DownLoad:
CSV
| FS method | Stability |
| CMIM | 72.6 |
| CMIM3 | 67.3 |
| JMI | 81.2 |
| JMI3 | 75.2 |
| JMIM | 75.9 |
| MIGM | 66.2 |
| QPFS | 71.7 |
| Relief | 44.3 |
| WRFS | 66.6 |
| FJMIM | 87.8 |
DownLoad:
CSV
| [1] | Fairfax S, Gwin L, King M, et al. (2005) Buying nature: the limits of land acquisition as a conservation strategy, 1780-2004. Cambridge, Massachusetts: MIT Press, 379. |
| [2] |
Hobbs R (2007) Setting effective and realistic restoration goals: key directions for research. Restor Ecol 15: 354-357. doi: 10.1111/j.1526-100X.2007.00225.x
|
| [3] | Turner M, Gardner R, O'Neill R (2001) Landscape ecology in theory and practice: pattern and process. New York: Springer. |
| [4] | Boutin S, Herbert D (2002) Landscape ecology and forest management: developing an effective partnership. Ecol Appl 12: 390-397. |
| [5] | Cissel B, Swanson F, Mckee W, et al. (1994) Using the past to plan the future in the Pacific Northwest. J Forest 92: 30-31. |
| [6] |
Franklin J, Forman R (1987) Creating landscape patterns by forest cutting: ecological consequences and principles. Landsc Ecol 1: 5-18. doi: 10.1007/BF02275261
|
| [7] | Li H, Franklin J, Swanson F, et al. (1993). Developing alternative forest cutting patterns: a simulation approach. Landscape Ecol 8: 63-75. |
| [8] | Silbernagel J, Price J, Swaty R, et al. (2011) The next frontier: projecting the effectiveness of broad-scale forest conservation strategies. In C. Li, R. Lafortezza, & J. Chen Eds., Landscape Ecology in Forest Management and Conservation: Challenges and Solutions in a Changing Globe, New York: Springer, 209-230. |
| [9] |
Coreau A, Pinay G, Thompson J, et al. (2009) The rise of research on futures in ecology: rebalancing scenarios and predictions. Ecol Lett 12: 1277-1286. doi: 10.1111/j.1461-0248.2009.01392.x
|
| [10] |
Wollenberg E, Edmunds D, Buck L (2000) Using scenarios to make decisions about the future: anticipatory learning for the adaptive co-management of community forests. Landscape Urban Plan 47: 65-77. doi: 10.1016/S0169-2046(99)00071-7
|
| [11] | Gustafson E, Lytle D, Swaty R, et al. (2006) Simulating the cumulative effects of multiple forest management strategies on landscape measures of forest sustainability. Landsc Ecol 22: 141-156. |
| [12] |
Provencher L, Forbis T, Frid L, et al. (2007) Comparing alternative management strategies of fire, grazing, and weed control using spatial modeling. Ecol Model 209: 249-263. doi: 10.1016/j.ecolmodel.2007.06.030
|
| [13] |
Frid L, Holcombe T, Morisette J, et al. (2013) Using state-and-transition modeling to account for imperfect detection in invasive species management. Invas Plant Sci Mana 6: 36-47 doi: 10.1614/IPSM-D-11-00065.1
|
| [14] |
Kurz W, Beukema B, Klenner W, et al. (2000) TELSA: the tool for exploratory landscape scenario analyses. Comput Electron Agr 27: 227-242. doi: 10.1016/S0168-1699(00)00109-5
|
| [15] | Beukema S, Kurz W, Pinkham C, et al. (2003) Vegetation Dynamics Development Tool: user's guide version 4.4c. Vancouver: ESSA Technologies Ltd. |
| [16] |
Evers L, Miller R, Doescher P, et al. (2013) Simulating Current Successional Trajectories in Sagebrush Ecosystems With Multiple Disturbances Using a State-and-Transition Modeling Framework. Rangel Ecol Manag 66: 313-329. doi: 10.2111/REM-D-11-00220.1
|
| [17] | Provencher L, Anderson T (2011) Climate Change Revisions to Nevada's Wildlife Action Plan: Vegetation Mapping and Modeling Report to the Nevada Department of Wildlife. |
| [18] |
Halofsky J, Hemstrom M, Conklin D, et al. (2013) Assessing potential climate change effects on vegetation using a linked model approach. Ecol Modell 266: 131-143. doi: 10.1016/j.ecolmodel.2013.07.003
|
| [19] |
Yospin G, Bridgham S, Neilson R, et al. (2015) A new model to simulate climate-change impacts on forest succession for local land management. Ecol Appl 25: 226-242. doi: 10.1890/13-0906.1
|
| [20] | Nowacki G, Abrams M (2014) Is climate an important driver of post-European vegetation change in the Eastern United States? Glob Change Biol 21: 314-334. |
| [21] |
Costanza J, Terando A, McKerrow A, et al. (2015) Modeling climate change, urbanization, and fire effects on Pinus palustris ecosystems of the southeastern U.S. J Environ Manage 151: 186-199. doi: 10.1016/j.jenvman.2014.12.032
|
| [22] |
Price J, Silbernagel J, Miller N, et al. (2012) Eliciting expert knowledge to inform landscape modeling of conservation scenarios. Ecol Modell 229: 76-87. doi: 10.1016/j.ecolmodel.2011.09.010
|
| [23] | Janowiak M, Iverson L, Mladenoff D, et al. (2014) Forest ecosystem vulnerability assessment and synthesis for northern Wisconsin and western Upper Michigan: a report from the northwoods climate change response framework project. Gen Tech Rep NRS-136. |
| [24] | WICCI, Forestry Working Group Report. Madison, Wisconsin: Nelson Institute for Environmental Studies, University of Wisconsin-Madisonand the Wisconsin Department of Natural Resources, 2011. Available from: http://www.wicci.wisc.edu/report/Forestry.pdf. |
| [25] |
Turner M (1989) Landscape Ecology: The Effect of Pattern on Process. Annu Rev Ecol Syst 20:171-197. doi: 10.1146/annurev.es.20.110189.001131
|
| [26] | LANDFIRE, LANDFIRE National Vegetation Dynamics Models. U.S. Department of Agriculture, Forest Service; U.S. Department of Interior, 2007. Available from http://www.landfire.gov/index.php. |
| [27] | TNC, Adapting LANDFIRE vegetation dynamics models manual v.1. Tallahassee, FL: TNC Global Fire Initiative, 2009. Available from: http://www.tncfire.org/documents/Aug12pluslinksFINAL_003.pdf. |
| [28] |
Ryan KC, Opperman TS (2013) LANDFIRE-A national vegetation/fuels data base for use in fuels treatment, restoration, and suppression planning. Forest Ecol Manage 294: 208-216. doi: 10.1016/j.foreco.2012.11.003
|
| [29] |
Rollins M (2009) LANDFIRE: A nationally consistent vegetation, wildland fire, and fuel assessment. Int J Wildl Fire 18: 235-249. doi: 10.1071/WF08088
|
| [30] | Wisconin Department of Natural Resources, Wisconsin's managed forest law: a program summary. Wisconsin Department of Natural Resources. PUB-FR-295, 2013. Available from: http://dnr.wi.gov/files/pdf/pubs/fr/FR0295.pdf |
| [31] |
Cleland D, Crow T, Saunders S, et al. (2004) Characterizing historical and modern fire regimes in Michigan (USA): a landscape ecosystem approach. Landsc Ecol 19: 311-325. doi: 10.1023/B:LAND.0000030437.29258.3c
|
| [32] | Schulte L, Mladenoff D, Crow T, et al. (2007) Homogenization of northern U.S. Great Lakes forests due to land use. Landsc Ecol 22: 1089-1103. |
| [33] |
Drever C, Bergeron Y, Drever M, et al. (2009) Effects of climate on occurrence and size of large fires in a northern hardwood landscape: historical trends, forecasts, and implications for climate change in Temiscamingue, Quebec. Appl Veg Sci 12: 261-272. doi: 10.1111/j.1654-109X.2009.01035.x
|
| [34] |
Drobyshev I, Goebel P, Bergeron Y, et al. (2012) Detecting changes in climate forcing on the fire regime of a North American mixed-pine forest: A case study of Seney National Wildlife Refuge, Upper Michigan. Dendrochronologia 30: 137-145. doi: 10.1016/j.dendro.2011.07.002
|
| [35] |
Flannigan M, Stocks B, Turetsky M, et al. (2009) Impacts of climate change on fire activity and fire management in the circumboreal forest. Glob Chang Bio 15: 549-560. doi: 10.1111/j.1365-2486.2008.01660.x
|
| [36] |
Overpeck J, Rind D, Goldberg R (1990) Climate-induced changes in forest disturbance and vegetation. Nature 343: 51-53. doi: 10.1038/343051a0
|
| [37] | Woolford D, Cao J, Dean C, et al. (1999) Signals in a region of the Canadian boreal forest. Environmetrics 21: 789-800. |
| [38] |
Guyette RP, Thompson FR, Whittier J, et al. (2014). Future Fire Probability Modeling with Climate Change Data and Physical Chemistry. Forest Sci 60: 862-870. doi: 10.5849/forsci.13-108
|
| [39] | Andresen J, Hilberg S, Kunkel K (2012) Historical Climate and Climate Trends in the Midwestern USA. U.S. Natl Clim Assess Midwest Tech Input Rep. 3-16. |
| [40] |
Peterson C (2000) Catastrophic wind damage to North American forests and the potential impact of climate change. Sci Total Environ 262: 287-31. doi: 10.1016/S0048-9697(00)00529-5
|
| [41] | Coniglio M, Stensrud D (2004) Interpreting the Climatology of Derechos. Weather Forecast 19: 95-605. |
| [42] |
Dale V, Joyce L, McNulty S, et al. (2001) Climate Change and Forest Disturbances. Bioscience 51: 723-734. doi: 10.1641/0006-3568(2001)051[0723:CCAFD]2.0.CO;2
|
| [43] | LANDFIRE, LANDFIRE National Vegetation Dynamics Models. U.S. Department of Agriculture, Forest Service; U.S. Department of Interior, 2007. Availble from: http://www.landfire.gov/national_veg_models_op2.php |
| [44] | ESRI (Environmental Systems Research Institute) (2008) ArcGIS: Release 9.3 software. Environmental Systems Research Institute, Redlands, California |
| [45] | McGarigal K, Cushman S, Neel M, et al. FRAGSTATS: Spatial pattern analysis program for categorical maps. University of Massachusetts, Amherst, 2002. Available from: http://www.umass.edu/landeco/research/fragstats/fragstats. |
| [46] |
Li B, Archer S (1997) Weighted mean patch size: A robust index for quantifying landscape structure. Ecol Modell 102: 353-361. doi: 10.1016/S0304-3800(97)00071-9
|
| [47] |
Riitters K, O'Neill R, Hunsaker C, et al. (1995) A factor analysis of landscape pattern and structure metrics. Landsc Ecol 10: 23-39. doi: 10.1007/BF00158551
|
| [48] | R Core Team, R: A language environment for statistical computing. R Foundation for Statistical Computing, Vienna, Austira. 2013. Available from: http://www.R-project.org/. |
| [49] |
Merenlender A, Huntsinger L, Guthey G, et al. (2004) Land trusts and conservation easements: Who is conserving what for whom? Conserv Biol 18: 65-75. doi: 10.1111/j.1523-1739.2004.00401.x
|
| [50] |
Scheller R, Mladenoff D (2007) An ecological classification of forest landscape simulation models: tools and strategies for understanding broad-scale forested ecosystems. Landsc Ecol 22: 491-505. doi: 10.1007/s10980-006-9048-4
|
| [51] |
Wear D, Turner M, Flamm R, et al. (1996) Ecosystem management with multiple owners: landscape dynamics in a Southern Appalachian watershed. Ecol Appl 6: 1173-1188. doi: 10.2307/2269600
|
| [52] |
Zollner P, Roberts L, Gustafson E, et al. (2008) Influence of forest planning alternatives on landscape pattern and ecosystem processes in northern Wisconsin, USA. Forest Ecol Management 254: 429-444. doi: 10.1016/j.foreco.2007.07.038
|
| [53] | Levins R (1966) The strategy of model building in population biology. Ameri Scientist 54: 421-431. |
| [54] |
Radeloff V, Mladenoff D, Gustafson E, et al. (2006) Modeling forest harvesting effects on landscape pattern in the Northwest Wisconsin Pine Barrens. Forest Ecol Management 236: 113-126. doi: 10.1016/j.foreco.2006.09.007
|
| [55] |
Peterson G (2002) Contagious disturbance, ecological memory, and the emergence of landscape pattern. Ecosystems 5: 329-338.56. Drescher M, Perera A, Buse L, et al. (2008) Uncertainty in expert knowledge of forest succession: A case study from boreal Ontario. For Chron 84: 194-209. doi: 10.5558/tfc84194-2
|
| 1. | Lin Sun, Shanshan Si, Jing Zhao, Jiucheng Xu, Yaojin Lin, Zhiying Lv, Feature selection using binary monarch butterfly optimization, 2023, 53, 0924-669X, 706, 10.1007/s10489-022-03554-9 | |
| 2. | Amjad Qtaish, Malik Braik, Dheeb Albashish, Mohammad T. Alshammari, Abdulrahman Alreshidi, Eissa Jaber Alreshidi, Enhanced coati optimization algorithm using elite opposition-based learning and adaptive search mechanism for feature selection, 2024, 1868-8071, 10.1007/s13042-024-02222-3 | |
| 3. | Xi-Ao Ma, Hao Xu, Yi Liu, Justin Zuopeng Zhang, Class-specific feature selection using fuzzy information-theoretic metrics, 2024, 136, 09521976, 109035, 10.1016/j.engappai.2024.109035 | |
| 4. | Lei Xiao, 2024, Evaluation Model Construction Based on 'Three-dimensional' Data Fusion in Cross-cultural Blended Course, 9798400710360, 11, 10.1145/3686424.3686426 | |
| 5. | Pedro J. Gutiérrez-Diez, Jorge Alves-Antunes, Stock market uncertainty determination with news headlines: A digital twin approach, 2023, 9, 2473-6988, 1683, 10.3934/math.2024083 | |
| 6. | Ah. E. Hegazy, B. Hafiz, M. A. Makhlouf, Omar A. M. Salem, Optimizing medical data classification: integrating hybrid fuzzy joint mutual information with binary Cheetah optimizer algorithm, 2025, 28, 1386-7857, 10.1007/s10586-025-05102-9 |
Amanda Swearingen, Jessica Price, Janet Silbernagel, Randy Swaty, Nicholas Miller. State-and-transition simulation modeling to compare outcomes of alternative management scenarios under two natural disturbance regimes in a forested landscape in northeastern Wisconsin, USA[J]. AIMS Environmental Science, 2015, 2(3): 737-763. doi: 10.3934/environsci.2015.3.737
| No. | Datasets | Instances | Features | Classes |
| 1 | Acute Inflammations | 120 | 6 | 2 |
| 2 | Arrhythmia | 452 | 279 | 13 |
| 3 | Blogger | 100 | 6 | 2 |
| 4 | Diabetic Retinopathy Debrecen (DRD) | 1151 | 20 | 2 |
| 5 | Hayes-Roth | 160 | 5 | 3 |
| 6 | Indian Liver Patient Dataset (ILPD) | 583 | 10 | 2 |
| 7 | Lenses | 24 | 4 | 3 |
| 8 | Lymphography | 148 | 18 | 4 |
| 9 | Congressional Voting Records (CVR) | 435 | 16 | 2 |
| 10 | Sonar | 208 | 60 | 2 |
| 11 | Thoracic Surgery | 470 | 17 | 2 |
| 12 | Wilt | 4889 | 6 | 2 |
| 13 | Zoo | 101 | 17 | 7 |
DownLoad:
CSV
| Dataset | CMIM | CMIM3 | JMI | JMI3 | JMIM | MIGM | QPFS | Relief | WRFS | FJMIM |
| Acute Inflammations | 100±0[=] | 100±0[=] | 100±0[=] | 100±0[=] | 100±0[=] | 99.75±2.5[=] | 99.75±2.5[=] | 100±0[=] | 90.83±8.5[+] | 100±0 |
| Arrhythmia | 60.64±7.01[=] | 59.22±6.17[=] | 59.42±6.49[=] | 59.58±6.58[=] | 60.62±6.93[=] | 58.98±6.39[=] | 63.7±6.57[-] | 64.94±6.33[-] | 59.31±6.3[=] | 59.74±6.43 |
| Blogger | 61.9±10.7[=] | 61.9±10.7[=] | 61.9±10.7[=] | 64.9±10.2[=] | 61.9±10.7[=] | 61.9±10.7[=] | 61.9±10.7[=] | 65.4±7.84[=] | 66.1±9.09[=] | 65.9±10.26 |
| DRD | 57.59±4.95[=] | 60.66±4.69[-] | 60.3±5.04[-] | 60.6±4.58[-] | 57.59±4.95[=] | 60.3±5.04[-] | 61.19±4.49[-] | 60.61±5.06[-] | 57.47±4.98[=] | 57.56±4.83 |
| Hayes-Roth | 45.81±10.77[=] | 45.81±10.77[=] | 45.81±10.77[=] | 45.81±10.77[=] | 45.81±10.77[=] | 45.81±10.77[=] | 49.37±10.53[=] | 46.31±11.48[=] | 45.81±10.77[=] | 49.37±10.53 |
| ILPD | 68.99±5.68[=] | 67.33±5.87[=] | 68.99±5.68[=] | 67.33±5.87[=] | 68.99±5.68[=] | 68.99±5.68[=] | 68.73±5.64[=] | 64.96±6.45[+] | 70.11±6.74[=] | 69.54±5.75 |
| Lenses | 86.83±21.88[=] | 86.83±21.88[=] | 86.83±21.88[=] | 86.83±21.88[=] | 86.83±21.88[=] | 86.83±21.88[=] | 86.83±21.88[=] | 54.83±30.73[+] | 86.83±21.88[=] | 86.83±21.88 |
| Lymphography | 79.02±10.57[=] | 82.05±9.59[=] | 79.02±10.57[=] | 77.93±10.87[=] | 78.27±10.98[=] | 77.25±11.38[=] | 79.02±10.57[=] | 82.61±9.77[=] | 76.91±11.64[+] | 81.5±10.29 |
| CVR | 94.3±3.15[=] | 94.32±3.12[=] | 93.75±3.57[=] | 93.75±3.57[=] | 93.75±3.57[=] | 93.75±3.57[=] | 94.32±3.26[=] | 93.82±3.28[=] | 93.52±3.18[=] | 93.75±3.57 |
| Sonar | 75.05±8.33[=] | 75.42±9.08[=] | 75.75±8.85[=] | 76.23±9.99[=] | 72.2±9.93[=] | 72.14±9.54[+] | 76.42±8.69[=] | 74.25±8.33[=] | 77.91±8.74[=] | 77±8.98 |
| Thoracic Surgery | 83.83±3.1[=] | 83.38±3.09[=] | 83.38±3.09[=] | 83.85±2.24[=] | 83.38±3.09[=] | 84.64±1.83[=] | 83.55±3.08[=] | 83.06±2.77[=] | 83.85±2.74[=] | 83.38±3.09 |
| Wilt | 94.61±0.06[=] | 94.61±0.06[=] | 94.61±0.06[=] | 94.61±0.06[=] | 94.61±0.06[=] | 94.61±0.06[=] | 94.61±0.06[=] | 94.61±0.06[=] | 94.61±0.06[=] | 94.61±0.06 |
| Zoo | 95.15±5.74[=] | 96.05±5.6[=] | 96.03±5.66[=] | 96.03±5.66[=] | 96.03±5.66[=] | 96.93±5.42[=] | 94.08±6.6[=] | 94.08±6.6[=] | 92.96±7.05[=] | 95.05±5.87 |
| Average | 77.21 | 77.51 | 77.37 | 77.50 | 76.92 | 77.07 | 77.96 | 75.34 | 76.63 | 78.02 |
DownLoad:
CSV
| Dataset | CMIM | CMIM3 | JMI | JMI3 | JMIM | MIGM | QPFS | Relief | WRFS | FJMIM |
| Acute Inflammations | 98.33±3.35[=] | 99.67±1.64[=] | 98.33±3.35[=] | 98.33±3.35[=] | 98.33±3.35[=] | 99.42±2.44[=] | 99.42±2.44[=] | 100±0[+] | 89.92±9.5[+] | 99.42±2.44 |
| Arrhythmia | 66.09±6.81[=] | 64.08±6.11[=] | 65.01±5.96[=] | 65.7±6.23[=] | 66.14±6.61[=] | 65.23±6.39[=] | 66.82±6.2[=] | 67.22±6.05[=] | 65.23±6.15[=] | 67.11±6.7 |
| Blogger | 68±4.02[=] | 67.7±4.89[=] | 68±4.02[=] | 67.4±5.62[=] | 68±4.02[=] | 68±4.02[=] | 67.7±4.89[=] | 67.7±4.89[=] | 68±4.02[=] | 67.7±4.89 |
| DRD | 68.03±3.92[=] | 67.55±4.07[=] | 67.15±3.85[=] | 67.03±3.95[=] | 68.03±3.92[=] | 67.19±3.8[=] | 65.1±4.33[+] | 61.91±4.51[+] | 67.7±4[=] | 67.96±4.23 |
| Hayes-Roth | 49.94±12.83[=] | 49.94±12.83[=] | 49.94±12.83[=] | 49.94±12.83[=] | 49.94±12.83[=] | 49.94±12.83[=] | 52±11.61[=] | 48.75±13.21[=] | 49.94±12.83[=] | 52±11.61 |
| ILPD | 71.36±0.73[=] | 71.36±0.73[=] | 71.36±0.73[=] | 71.36±0.73[=] | 71.36±0.73[=] | 71.36±0.73[=] | 71.36±0.73[=] | 71.36±0.73[=] | 71.36±0.73[=] | 71.36±0.73 |
| Lenses | 76.17±22.13[=] | 76.17±22.13[=] | 76.17±22.13[=] | 76.17±22.13[=] | 76.17±22.13[=] | 76.17±22.13[=] | 76.17±22.13[=] | 54.5±30.42[+] | 76.17±22.13[=] | 76.17±22.13 |
| Lymphography | 81.57±10.06[=] | 80.56±9.09[=] | 81.42±10.15[=] | 84.55±8.84[=] | 82.02±10.48[=] | 79.06±10.44[+] | 81.57±10.11[=] | 80.3±9[=] | 74.25±11.45[+] | 85.5±8.3 |
| CVR | 94.09±3.33[=] | 94.46±3.37[=] | 94.3±3.34[=] | 94.32±3.32[=] | 94.3±3.34[=] | 94.3±3.34[=] | 94±3.44[=] | 94.43±3.41[=] | 94.37±3.41[=] | 94.3±3.34 |
| Sonar | 79.34±8.37[=] | 80.48±8.05[=] | 80.18±8.33[=] | 78.46±8.44[=] | 80.54±8.43[=] | 76.93±8.71[=] | 79.91±7.73[=] | 82.1±8.07[=] | 77.41±8.41[=] | 80.67±7.88 |
| Thoracic Surgery | 85.11±0[=] | 85.11±0[=] | 85.11±0[=] | 85.11±0[=] | 85.11±0[=] | 85.11±0[=] | 85.11±0[=] | 85.11±0[=] | 85.11±0[=] | 85.11±0 |
| Wilt | 97.62±0.57[+] | 97.97±0.49[=] | 97.97±0.49[=] | 97.97±0.49[=] | 97.62±0.57[+] | 97.97±0.49[=] | 94.61±0.06[+] | 97.97±0.49[=] | 97.62±0.57[+] | 97.98±0.49 |
| Zoo | 95.75±5.12[=] | 93.88±6.72[=] | 89.39±5.72[+] | 89.49±5.64[+] | 89.49±5.64[+] | 94.06±6.9[=] | 95.27±5.85[=] | 93.89±6.96[=] | 90.49±7.96[+] | 95.06±6.2 |
| Average | 79.34 | 79.15 | 78.79 | 78.91 | 79.00 | 78.83 | 79.16 | 77.33 | 77.51 | 80.03 |
DownLoad:
CSV
| Dataset | CMIM | CMIM3 | JMI | JMI3 | JMIM | MIGM | QPFS | Relief | WRFS | FJMIM |
| Acute Inflammations | 99.5±1.99[=] | 100±0[=] | 99.5±1.99[=] | 99.5±1.99[=] | 99.5±1.99[=] | 100±0[=] | 100±0[=] | 100±0[=] | 99.67±1.64[=] | 100±0 |
| Arrhythmia | 58.27±5.17[+] | 57.63±5.17[+] | 57.79±5.34[+] | 58.32±5.08[+] | 58.94±5.13[=] | 57.67±5.21[+] | 60.73±4.64[=] | 65.13±4.43[-] | 58.39±5.22[+] | 61.11±5.23 |
| Blogger | 72.1±11.31[=] | 72.1±11.31[=] | 72.1±11.31[=] | 78.8±10.18[=] | 72.1±11.31[=] | 72.1±11.31[=] | 72.1±11.31[=] | 70.3±11.41[=] | 76.1±12.05[=] | 78.6±11.37 |
| DRD | 64.3±4.19[=] | 62.62±5.29[=] | 63.69±4.58[=] | 61.96±4.68[=] | 64.3±4.19[=] | 63.58±4.56[=] | 60.47±4.64[+] | 63.51±4.63[=] | 64.06±4.4[=] | 64.7±4.23 |
| Hayes-Roth | 59.19±10.41[=] | 59.19±10.41[=] | 59.19±10.41[=] | 59.19±10.41[=] | 59.19±10.41[=] | 59.19±10.41[=] | 63.12±11.32[=] | 57.56±10.82[=] | 59.19±10.41[=] | 63.12±11.32 |
| ILPD | 67.16±5.41[=] | 68.22±5.06[=] | 67.54±5.44[=] | 68.22±5.06[=] | 67.16±5.41[=] | 67.3±5.42[=] | 68.74±5.01[=] | 67.84±5.56[=] | 66.66±5.25[=] | 69.15±5.62 |
| Lenses | 86.83±21.88[=] | 86.83±21.88[=] | 86.83±21.88[=] | 86.83±21.88[=] | 86.83±21.88[=] | 86.83±21.88[=] | 86.83±21.88[=] | 54.17±30.46[+] | 86.83±21.88[=] | 86.83±21.88 |
| Lymphography | 84.61±7.58[=] | 72.61±9.14[+] | 84.61±7.58[=] | 84.16±9.46[=] | 80.99±9.49[=] | 76.62±8.87[=] | 84.61±7.58[=] | 81.44±9.42[=] | 79.74±9.36[=] | 78.87±8.35 |
| CVR | 93.63±3.43[=] | 94.09±3.33[=] | 95.26±3.01[=] | 95.26±3.01[=] | 95.26±3.01[=] | 95.26±3.01[=] | 94.14±3.37[=] | 93.67±3.5[=] | 94.09±2.95[=] | 95.26±3.01 |
| Sonar | 85.09±7.43[=] | 83.51±7.68[=] | 87.74±7.51[=] | 85.53±8.83[=] | 82.7±9.06[=] | 82.4±7.72[+] | 86.15±7.24[=] | 87.95±6.78[=] | 84.17±7.73[=] | 87.65±7.27 |
| Thoracic Surgery | 80.96±3.45[=] | 81.38±3.53[=] | 81.38±3.53[=] | 83.45±3.05[=] | 81.38±3.53[=] | 82.64±3.28[=] | 83.89±2.95[=] | 82.68±3.08[=] | 84.85±0.97[-] | 81.38±3.53 |
| Wilt | 97.69±0.57[+] | 98.12±0.55[=] | 98.12±0.55[=] | 98.12±0.55[=] | 97.69±0.57[+] | 98.12±0.55[=] | 94.44±0.52[+] | 98.12±0.55[=] | 97.69±0.57[+] | 98.12±0.55 |
| Zoo | 91.9±7.21[=] | 92.06±7.35[=] | 92.05±7.24[=] | 92.05±7.24[=] | 92.05±7.24[=] | 91.16±7.48[=] | 91.42±6.94[=] | 91.42±6.94[=] | 90.29±7.45[+] | 94.08±6.6 |
| Average | 80.09 | 79.10 | 80.45 | 80.88 | 79.85 | 79.45 | 80.51 | 77.98 | 80.13 | 81.45 |
DownLoad:
CSV
| FS method | Average |
| CMIM | 78.9 |
| CMIM3 | 78.6 |
| JMI | 78.9 |
| JMI3 | 79.1 |
| JMIM | 78.6 |
| MIGM | 78.4 |
| QPFS | 79.2 |
| Relief | 76.9 |
| WRFS | 78.1 |
| FJMIM | 79.8 |
DownLoad:
CSV
| FS method | NB | SVM | KNN | Average |
| CMIM | 0.702 | 0.641 | 0.704 | 0.682 |
| CMIM3 | 0.742 | 0.641 | 0.705 | 0.696 |
| JMI | 0.705 | 0.640 | 0.713 | 0.686 |
| JMI3 | 0.712 | 0.636 | 0.723 | 0.690 |
| JMIM | 0.701 | 0.642 | 0.707 | 0.683 |
| MIGM | 0.701 | 0.637 | 0.708 | 0.682 |
| QPFS | 0.712 | 0.636 | 0.711 | 0.686 |
| Relief | 0.621 | 0.612 | 0.661 | 0.631 |
| WRFS | 0.698 | 0.632 | 0.708 | 0.679 |
| FJMIM | 0.748 | 0.645 | 0.743 | 0.712 |
DownLoad:
CSV
| FS method | NB | SVM | KNN | Average |
| CMIM | 0.722 | 0.712 | 0.818 | 0.750 |
| CMIM3 | 0.795 | 0.712 | 0.752 | 0.753 |
| JMI | 0.724 | 0.712 | 0.823 | 0.753 |
| JMI3 | 0.729 | 0.712 | 0.834 | 0.759 |
| JMIM | 0.719 | 0.714 | 0.821 | 0.751 |
| MIGM | 0.722 | 0.710 | 0.818 | 0.750 |
| QPFS | 0.732 | 0.712 | 0.825 | 0.756 |
| Relief | 0.632 | 0.708 | 0.688 | 0.676 |
| WRFS | 0.716 | 0.701 | 0.826 | 0.748 |
| FJMIM | 0.798 | 0.715 | 0.840 | 0.784 |
DownLoad:
CSV
| FS method | NB | SVM | KNN | Average |
| CMIM | 0.835 | 0.733 | 0.812 | 0.793 |
| CMIM3 | 0.836 | 0.748 | 0.844 | 0.809 |
| JMI | 0.837 | 0.734 | 0.815 | 0.795 |
| JMI3 | 0.831 | 0.729 | 0.805 | 0.788 |
| JMIM | 0.836 | 0.742 | 0.812 | 0.796 |
| MIGM | 0.832 | 0.754 | 0.807 | 0.798 |
| QPFS | 0.838 | 0.710 | 0.807 | 0.785 |
| Relief | 0.804 | 0.707 | 0.801 | 0.771 |
| WRFS | 0.831 | 0.732 | 0.806 | 0.789 |
| FJMIM | 0.839 | 0.748 | 0.852 | 0.813 |
DownLoad:
CSV
| FS method | NB | SVM | KNN | Average |
| CMIM | 0.848 | 0.708 | 0.798 | 0.785 |
| CMIM3 | 0.862 | 0.715 | 0.862 | 0.813 |
| JMI | 0.850 | 0.708 | 0.797 | 0.785 |
| JMI3 | 0.837 | 0.705 | 0.800 | 0.781 |
| JMIM | 0.852 | 0.713 | 0.797 | 0.787 |
| MIGM | 0.857 | 0.724 | 0.827 | 0.803 |
| QPFS | 0.857 | 0.708 | 0.797 | 0.787 |
| Relief | 0.842 | 0.685 | 0.791 | 0.772 |
| WRFS | 0.850 | 0.697 | 0.795 | 0.781 |
| FJMIM | 0.862 | 0.718 | 0.869 | 0.816 |
DownLoad:
CSV
| FS method | Stability |
| CMIM | 72.6 |
| CMIM3 | 67.3 |
| JMI | 81.2 |
| JMI3 | 75.2 |
| JMIM | 75.9 |
| MIGM | 66.2 |
| QPFS | 71.7 |
| Relief | 44.3 |
| WRFS | 66.6 |
| FJMIM | 87.8 |
DownLoad:
CSV
| No. | Datasets | Instances | Features | Classes |
| 1 | Acute Inflammations | 120 | 6 | 2 |
| 2 | Arrhythmia | 452 | 279 | 13 |
| 3 | Blogger | 100 | 6 | 2 |
| 4 | Diabetic Retinopathy Debrecen (DRD) | 1151 | 20 | 2 |
| 5 | Hayes-Roth | 160 | 5 | 3 |
| 6 | Indian Liver Patient Dataset (ILPD) | 583 | 10 | 2 |
| 7 | Lenses | 24 | 4 | 3 |
| 8 | Lymphography | 148 | 18 | 4 |
| 9 | Congressional Voting Records (CVR) | 435 | 16 | 2 |
| 10 | Sonar | 208 | 60 | 2 |
| 11 | Thoracic Surgery | 470 | 17 | 2 |
| 12 | Wilt | 4889 | 6 | 2 |
| 13 | Zoo | 101 | 17 | 7 |
| Dataset | CMIM | CMIM3 | JMI | JMI3 | JMIM | MIGM | QPFS | Relief | WRFS | FJMIM |
| Acute Inflammations | 100±0[=] | 100±0[=] | 100±0[=] | 100±0[=] | 100±0[=] | 99.75±2.5[=] | 99.75±2.5[=] | 100±0[=] | 90.83±8.5[+] | 100±0 |
| Arrhythmia | 60.64±7.01[=] | 59.22±6.17[=] | 59.42±6.49[=] | 59.58±6.58[=] | 60.62±6.93[=] | 58.98±6.39[=] | 63.7±6.57[-] | 64.94±6.33[-] | 59.31±6.3[=] | 59.74±6.43 |
| Blogger | 61.9±10.7[=] | 61.9±10.7[=] | 61.9±10.7[=] | 64.9±10.2[=] | 61.9±10.7[=] | 61.9±10.7[=] | 61.9±10.7[=] | 65.4±7.84[=] | 66.1±9.09[=] | 65.9±10.26 |
| DRD | 57.59±4.95[=] | 60.66±4.69[-] | 60.3±5.04[-] | 60.6±4.58[-] | 57.59±4.95[=] | 60.3±5.04[-] | 61.19±4.49[-] | 60.61±5.06[-] | 57.47±4.98[=] | 57.56±4.83 |
| Hayes-Roth | 45.81±10.77[=] | 45.81±10.77[=] | 45.81±10.77[=] | 45.81±10.77[=] | 45.81±10.77[=] | 45.81±10.77[=] | 49.37±10.53[=] | 46.31±11.48[=] | 45.81±10.77[=] | 49.37±10.53 |
| ILPD | 68.99±5.68[=] | 67.33±5.87[=] | 68.99±5.68[=] | 67.33±5.87[=] | 68.99±5.68[=] | 68.99±5.68[=] | 68.73±5.64[=] | 64.96±6.45[+] | 70.11±6.74[=] | 69.54±5.75 |
| Lenses | 86.83±21.88[=] | 86.83±21.88[=] | 86.83±21.88[=] | 86.83±21.88[=] | 86.83±21.88[=] | 86.83±21.88[=] | 86.83±21.88[=] | 54.83±30.73[+] | 86.83±21.88[=] | 86.83±21.88 |
| Lymphography | 79.02±10.57[=] | 82.05±9.59[=] | 79.02±10.57[=] | 77.93±10.87[=] | 78.27±10.98[=] | 77.25±11.38[=] | 79.02±10.57[=] | 82.61±9.77[=] | 76.91±11.64[+] | 81.5±10.29 |
| CVR | 94.3±3.15[=] | 94.32±3.12[=] | 93.75±3.57[=] | 93.75±3.57[=] | 93.75±3.57[=] | 93.75±3.57[=] | 94.32±3.26[=] | 93.82±3.28[=] | 93.52±3.18[=] | 93.75±3.57 |
| Sonar | 75.05±8.33[=] | 75.42±9.08[=] | 75.75±8.85[=] | 76.23±9.99[=] | 72.2±9.93[=] | 72.14±9.54[+] | 76.42±8.69[=] | 74.25±8.33[=] | 77.91±8.74[=] | 77±8.98 |
| Thoracic Surgery | 83.83±3.1[=] | 83.38±3.09[=] | 83.38±3.09[=] | 83.85±2.24[=] | 83.38±3.09[=] | 84.64±1.83[=] | 83.55±3.08[=] | 83.06±2.77[=] | 83.85±2.74[=] | 83.38±3.09 |
| Wilt | 94.61±0.06[=] | 94.61±0.06[=] | 94.61±0.06[=] | 94.61±0.06[=] | 94.61±0.06[=] | 94.61±0.06[=] | 94.61±0.06[=] | 94.61±0.06[=] | 94.61±0.06[=] | 94.61±0.06 |
| Zoo | 95.15±5.74[=] | 96.05±5.6[=] | 96.03±5.66[=] | 96.03±5.66[=] | 96.03±5.66[=] | 96.93±5.42[=] | 94.08±6.6[=] | 94.08±6.6[=] | 92.96±7.05[=] | 95.05±5.87 |
| Average | 77.21 | 77.51 | 77.37 | 77.50 | 76.92 | 77.07 | 77.96 | 75.34 | 76.63 | 78.02 |
| Dataset | CMIM | CMIM3 | JMI | JMI3 | JMIM | MIGM | QPFS | Relief | WRFS | FJMIM |
| Acute Inflammations | 98.33±3.35[=] | 99.67±1.64[=] | 98.33±3.35[=] | 98.33±3.35[=] | 98.33±3.35[=] | 99.42±2.44[=] | 99.42±2.44[=] | 100±0[+] | 89.92±9.5[+] | 99.42±2.44 |
| Arrhythmia | 66.09±6.81[=] | 64.08±6.11[=] | 65.01±5.96[=] | 65.7±6.23[=] | 66.14±6.61[=] | 65.23±6.39[=] | 66.82±6.2[=] | 67.22±6.05[=] | 65.23±6.15[=] | 67.11±6.7 |
| Blogger | 68±4.02[=] | 67.7±4.89[=] | 68±4.02[=] | 67.4±5.62[=] | 68±4.02[=] | 68±4.02[=] | 67.7±4.89[=] | 67.7±4.89[=] | 68±4.02[=] | 67.7±4.89 |
| DRD | 68.03±3.92[=] | 67.55±4.07[=] | 67.15±3.85[=] | 67.03±3.95[=] | 68.03±3.92[=] | 67.19±3.8[=] | 65.1±4.33[+] | 61.91±4.51[+] | 67.7±4[=] | 67.96±4.23 |
| Hayes-Roth | 49.94±12.83[=] | 49.94±12.83[=] | 49.94±12.83[=] | 49.94±12.83[=] | 49.94±12.83[=] | 49.94±12.83[=] | 52±11.61[=] | 48.75±13.21[=] | 49.94±12.83[=] | 52±11.61 |
| ILPD | 71.36±0.73[=] | 71.36±0.73[=] | 71.36±0.73[=] | 71.36±0.73[=] | 71.36±0.73[=] | 71.36±0.73[=] | 71.36±0.73[=] | 71.36±0.73[=] | 71.36±0.73[=] | 71.36±0.73 |
| Lenses | 76.17±22.13[=] | 76.17±22.13[=] | 76.17±22.13[=] | 76.17±22.13[=] | 76.17±22.13[=] | 76.17±22.13[=] | 76.17±22.13[=] | 54.5±30.42[+] | 76.17±22.13[=] | 76.17±22.13 |
| Lymphography | 81.57±10.06[=] | 80.56±9.09[=] | 81.42±10.15[=] | 84.55±8.84[=] | 82.02±10.48[=] | 79.06±10.44[+] | 81.57±10.11[=] | 80.3±9[=] | 74.25±11.45[+] | 85.5±8.3 |
| CVR | 94.09±3.33[=] | 94.46±3.37[=] | 94.3±3.34[=] | 94.32±3.32[=] | 94.3±3.34[=] | 94.3±3.34[=] | 94±3.44[=] | 94.43±3.41[=] | 94.37±3.41[=] | 94.3±3.34 |
| Sonar | 79.34±8.37[=] | 80.48±8.05[=] | 80.18±8.33[=] | 78.46±8.44[=] | 80.54±8.43[=] | 76.93±8.71[=] | 79.91±7.73[=] | 82.1±8.07[=] | 77.41±8.41[=] | 80.67±7.88 |
| Thoracic Surgery | 85.11±0[=] | 85.11±0[=] | 85.11±0[=] | 85.11±0[=] | 85.11±0[=] | 85.11±0[=] | 85.11±0[=] | 85.11±0[=] | 85.11±0[=] | 85.11±0 |
| Wilt | 97.62±0.57[+] | 97.97±0.49[=] | 97.97±0.49[=] | 97.97±0.49[=] | 97.62±0.57[+] | 97.97±0.49[=] | 94.61±0.06[+] | 97.97±0.49[=] | 97.62±0.57[+] | 97.98±0.49 |
| Zoo | 95.75±5.12[=] | 93.88±6.72[=] | 89.39±5.72[+] | 89.49±5.64[+] | 89.49±5.64[+] | 94.06±6.9[=] | 95.27±5.85[=] | 93.89±6.96[=] | 90.49±7.96[+] | 95.06±6.2 |
| Average | 79.34 | 79.15 | 78.79 | 78.91 | 79.00 | 78.83 | 79.16 | 77.33 | 77.51 | 80.03 |
| Dataset | CMIM | CMIM3 | JMI | JMI3 | JMIM | MIGM | QPFS | Relief | WRFS | FJMIM |
| Acute Inflammations | 99.5±1.99[=] | 100±0[=] | 99.5±1.99[=] | 99.5±1.99[=] | 99.5±1.99[=] | 100±0[=] | 100±0[=] | 100±0[=] | 99.67±1.64[=] | 100±0 |
| Arrhythmia | 58.27±5.17[+] | 57.63±5.17[+] | 57.79±5.34[+] | 58.32±5.08[+] | 58.94±5.13[=] | 57.67±5.21[+] | 60.73±4.64[=] | 65.13±4.43[-] | 58.39±5.22[+] | 61.11±5.23 |
| Blogger | 72.1±11.31[=] | 72.1±11.31[=] | 72.1±11.31[=] | 78.8±10.18[=] | 72.1±11.31[=] | 72.1±11.31[=] | 72.1±11.31[=] | 70.3±11.41[=] | 76.1±12.05[=] | 78.6±11.37 |
| DRD | 64.3±4.19[=] | 62.62±5.29[=] | 63.69±4.58[=] | 61.96±4.68[=] | 64.3±4.19[=] | 63.58±4.56[=] | 60.47±4.64[+] | 63.51±4.63[=] | 64.06±4.4[=] | 64.7±4.23 |
| Hayes-Roth | 59.19±10.41[=] | 59.19±10.41[=] | 59.19±10.41[=] | 59.19±10.41[=] | 59.19±10.41[=] | 59.19±10.41[=] | 63.12±11.32[=] | 57.56±10.82[=] | 59.19±10.41[=] | 63.12±11.32 |
| ILPD | 67.16±5.41[=] | 68.22±5.06[=] | 67.54±5.44[=] | 68.22±5.06[=] | 67.16±5.41[=] | 67.3±5.42[=] | 68.74±5.01[=] | 67.84±5.56[=] | 66.66±5.25[=] | 69.15±5.62 |
| Lenses | 86.83±21.88[=] | 86.83±21.88[=] | 86.83±21.88[=] | 86.83±21.88[=] | 86.83±21.88[=] | 86.83±21.88[=] | 86.83±21.88[=] | 54.17±30.46[+] | 86.83±21.88[=] | 86.83±21.88 |
| Lymphography | 84.61±7.58[=] | 72.61±9.14[+] | 84.61±7.58[=] | 84.16±9.46[=] | 80.99±9.49[=] | 76.62±8.87[=] | 84.61±7.58[=] | 81.44±9.42[=] | 79.74±9.36[=] | 78.87±8.35 |
| CVR | 93.63±3.43[=] | 94.09±3.33[=] | 95.26±3.01[=] | 95.26±3.01[=] | 95.26±3.01[=] | 95.26±3.01[=] | 94.14±3.37[=] | 93.67±3.5[=] | 94.09±2.95[=] | 95.26±3.01 |
| Sonar | 85.09±7.43[=] | 83.51±7.68[=] | 87.74±7.51[=] | 85.53±8.83[=] | 82.7±9.06[=] | 82.4±7.72[+] | 86.15±7.24[=] | 87.95±6.78[=] | 84.17±7.73[=] | 87.65±7.27 |
| Thoracic Surgery | 80.96±3.45[=] | 81.38±3.53[=] | 81.38±3.53[=] | 83.45±3.05[=] | 81.38±3.53[=] | 82.64±3.28[=] | 83.89±2.95[=] | 82.68±3.08[=] | 84.85±0.97[-] | 81.38±3.53 |
| Wilt | 97.69±0.57[+] | 98.12±0.55[=] | 98.12±0.55[=] | 98.12±0.55[=] | 97.69±0.57[+] | 98.12±0.55[=] | 94.44±0.52[+] | 98.12±0.55[=] | 97.69±0.57[+] | 98.12±0.55 |
| Zoo | 91.9±7.21[=] | 92.06±7.35[=] | 92.05±7.24[=] | 92.05±7.24[=] | 92.05±7.24[=] | 91.16±7.48[=] | 91.42±6.94[=] | 91.42±6.94[=] | 90.29±7.45[+] | 94.08±6.6 |
| Average | 80.09 | 79.10 | 80.45 | 80.88 | 79.85 | 79.45 | 80.51 | 77.98 | 80.13 | 81.45 |
| FS method | Average |
| CMIM | 78.9 |
| CMIM3 | 78.6 |
| JMI | 78.9 |
| JMI3 | 79.1 |
| JMIM | 78.6 |
| MIGM | 78.4 |
| QPFS | 79.2 |
| Relief | 76.9 |
| WRFS | 78.1 |
| FJMIM | 79.8 |
| FS method | NB | SVM | KNN | Average |
| CMIM | 0.702 | 0.641 | 0.704 | 0.682 |
| CMIM3 | 0.742 | 0.641 | 0.705 | 0.696 |
| JMI | 0.705 | 0.640 | 0.713 | 0.686 |
| JMI3 | 0.712 | 0.636 | 0.723 | 0.690 |
| JMIM | 0.701 | 0.642 | 0.707 | 0.683 |
| MIGM | 0.701 | 0.637 | 0.708 | 0.682 |
| QPFS | 0.712 | 0.636 | 0.711 | 0.686 |
| Relief | 0.621 | 0.612 | 0.661 | 0.631 |
| WRFS | 0.698 | 0.632 | 0.708 | 0.679 |
| FJMIM | 0.748 | 0.645 | 0.743 | 0.712 |
| FS method | NB | SVM | KNN | Average |
| CMIM | 0.722 | 0.712 | 0.818 | 0.750 |
| CMIM3 | 0.795 | 0.712 | 0.752 | 0.753 |
| JMI | 0.724 | 0.712 | 0.823 | 0.753 |
| JMI3 | 0.729 | 0.712 | 0.834 | 0.759 |
| JMIM | 0.719 | 0.714 | 0.821 | 0.751 |
| MIGM | 0.722 | 0.710 | 0.818 | 0.750 |
| QPFS | 0.732 | 0.712 | 0.825 | 0.756 |
| Relief | 0.632 | 0.708 | 0.688 | 0.676 |
| WRFS | 0.716 | 0.701 | 0.826 | 0.748 |
| FJMIM | 0.798 | 0.715 | 0.840 | 0.784 |
| FS method | NB | SVM | KNN | Average |
| CMIM | 0.835 | 0.733 | 0.812 | 0.793 |
| CMIM3 | 0.836 | 0.748 | 0.844 | 0.809 |
| JMI | 0.837 | 0.734 | 0.815 | 0.795 |
| JMI3 | 0.831 | 0.729 | 0.805 | 0.788 |
| JMIM | 0.836 | 0.742 | 0.812 | 0.796 |
| MIGM | 0.832 | 0.754 | 0.807 | 0.798 |
| QPFS | 0.838 | 0.710 | 0.807 | 0.785 |
| Relief | 0.804 | 0.707 | 0.801 | 0.771 |
| WRFS | 0.831 | 0.732 | 0.806 | 0.789 |
| FJMIM | 0.839 | 0.748 | 0.852 | 0.813 |
| FS method | NB | SVM | KNN | Average |
| CMIM | 0.848 | 0.708 | 0.798 | 0.785 |
| CMIM3 | 0.862 | 0.715 | 0.862 | 0.813 |
| JMI | 0.850 | 0.708 | 0.797 | 0.785 |
| JMI3 | 0.837 | 0.705 | 0.800 | 0.781 |
| JMIM | 0.852 | 0.713 | 0.797 | 0.787 |
| MIGM | 0.857 | 0.724 | 0.827 | 0.803 |
| QPFS | 0.857 | 0.708 | 0.797 | 0.787 |
| Relief | 0.842 | 0.685 | 0.791 | 0.772 |
| WRFS | 0.850 | 0.697 | 0.795 | 0.781 |
| FJMIM | 0.862 | 0.718 | 0.869 | 0.816 |
| FS method | Stability |
| CMIM | 72.6 |
| CMIM3 | 67.3 |
| JMI | 81.2 |
| JMI3 | 75.2 |
| JMIM | 75.9 |
| MIGM | 66.2 |
| QPFS | 71.7 |
| Relief | 44.3 |
| WRFS | 66.6 |
| FJMIM | 87.8 |
