Research article Special Issues

A negative correlation ensemble transfer learning method for fault diagnosis based on convolutional neural network

  • Received: 23 January 2019 Accepted: 10 April 2019 Published: 17 April 2019
  • With the development of the smart manufacturing, data-driven fault diagnosis has receiving more and more attentions from both academic and engineering fields. As one of the most important data-driven fault diagnosis method, deep learning (DL) has achieved remarkable applications. However, the DL based fault diagnosis methods still have the following two drawbacks: 1) One of the most major branch of deep learning is to construct the deeper structures, however the deep learning models in fault diagnosis is very shadow. 2) As stated by the no-free-lunch theorem, no single model can perform best on every dataset, and the individual deep learning model still suffers from the generalization ability. In this research, a new negative correlation ensemble transfer learning method (NCTE) is proposed. Firstly, the transfer learning based ResNet-50 is proposed to construct a deep learning structure that has 50 layers. Secondly, several fully-connected layers and softmax classifiers are trained cooperatively using negative correlation learning (NCL). Thirdly, the hyper-parameters of the proposed NCTE are determined by cross validation. The proposed NCTE is conducted on the KAT Bearing Dataset, and the prediction accuracy of NCTE is as high as 98.73%. This results show that NCTE has achieved a good results compared with other machine learning and deep learning method.

    Citation: Long Wen, Liang Gao, Yan Dong, Zheng Zhu. A negative correlation ensemble transfer learning method for fault diagnosis based on convolutional neural network[J]. Mathematical Biosciences and Engineering, 2019, 16(5): 3311-3330. doi: 10.3934/mbe.2019165

    Related Papers:

    [1] Marcella Reale . Annual Report 2020. AIMS Allergy and Immunology, 2021, 5(1): 33-37. doi: 10.3934/Allergy.2021003
    [2] Marcella Reale . Annual Report 2021. AIMS Allergy and Immunology, 2022, 6(1): 1-5. doi: 10.3934/Allergy.2022001
    [3] Marcella Reale, Xu Guo . Annual Report 2023. AIMS Allergy and Immunology, 2024, 8(1): 1-7. doi: 10.3934/Allergy.2024001
    [4] Marcella Reale . Annual Report 2022. AIMS Allergy and Immunology, 2023, 7(1): 40-44. doi: 10.3934/Allergy.2023003
    [5] Bono Eleonora, Zucca Federica, Ortolani Valeria Giuseppina Rita, Caron Lea, Eplite Angelo, Carsana Luca, Iemoli Enrico . Non-allergic rhinitis with eosinophilia syndrome treated with mepolizumab: A case report. AIMS Allergy and Immunology, 2023, 7(3): 176-182. doi: 10.3934/Allergy.2023012
    [6] Swathi Mukurala, Jahanavi Bandla, Swetha kappala . Angioedema-post COVID symptoms. AIMS Allergy and Immunology, 2023, 7(4): 273-280. doi: 10.3934/Allergy.2023018
    [7] Howard Mason, Kate Jones . Airborne exposure to laboratory animal allergens: 2005–2022. AIMS Allergy and Immunology, 2024, 8(1): 18-33. doi: 10.3934/Allergy.2024003
    [8] Richard A. Maiella, Kelly M. Staples, Ashokvardhan Veldanda . Migratory dermatographic urticaria following COVID-19 vaccine booster in young adult male. AIMS Allergy and Immunology, 2022, 6(1): 14-18. doi: 10.3934/Allergy.2022003
    [9] Joe Khodeir, Paul Ohanian, Ali Awwad . Mycobacterium chelonae-induced granulomatous nodules following botulinum toxin injections: A case report and literature review. AIMS Allergy and Immunology, 2024, 8(4): 296-302. doi: 10.3934/Allergy.2024018
    [10] Xiujuan Li, Jianmin Lin, Yan Li, Min Zhu, Minchuan Lin, Chenxi Li . Inhalation allergen sensitization patterns in children with allergic rhinitis and asthma. AIMS Allergy and Immunology, 2024, 8(4): 254-264. doi: 10.3934/Allergy.2024015
  • With the development of the smart manufacturing, data-driven fault diagnosis has receiving more and more attentions from both academic and engineering fields. As one of the most important data-driven fault diagnosis method, deep learning (DL) has achieved remarkable applications. However, the DL based fault diagnosis methods still have the following two drawbacks: 1) One of the most major branch of deep learning is to construct the deeper structures, however the deep learning models in fault diagnosis is very shadow. 2) As stated by the no-free-lunch theorem, no single model can perform best on every dataset, and the individual deep learning model still suffers from the generalization ability. In this research, a new negative correlation ensemble transfer learning method (NCTE) is proposed. Firstly, the transfer learning based ResNet-50 is proposed to construct a deep learning structure that has 50 layers. Secondly, several fully-connected layers and softmax classifiers are trained cooperatively using negative correlation learning (NCL). Thirdly, the hyper-parameters of the proposed NCTE are determined by cross validation. The proposed NCTE is conducted on the KAT Bearing Dataset, and the prediction accuracy of NCTE is as high as 98.73%. This results show that NCTE has achieved a good results compared with other machine learning and deep learning method.


    AIMS Allergy and Immunology is an international Open Access journal devoted to publishing peer-reviewed, high quality, original papers in the field of immunology and allergy. At the beginning of the new year and together with the Editorial Office of AIMS Allergy and Immunology, I wish to testify my sincere gratitude to all authors, members of the editorial board and reviewers for their contribution to AIMS Allergy and Immunology in 2024. We have made a meaningful progress in 2024, and we look forward to a more productive year in 2025.

    In 2024, We received 36 manuscripts and 20 were published; these published papers include 11 Research articles, 4 Review articles, 2 Editorial, 2 Case report, and 1 Mini review. The authors of the manuscripts are from more than 10 countries. The data shows a significant increase of international collaborations on the research of allergy and immunology. It now is a significant presence in the academic publishing market. During this 2024 we had 33% rejection ratio and publication time (from submission to online) was 117 days, illustrating the strict and efficient review process.

    We should be congratulated that after we got the first impact factor in 2023, our impact factor increased from 0.7 to 0.9 in 2024. This is inseparable from the efforts of the editorial board members. This is a further sign of the good work done in recent years.

    In recognition of authors' expertise the Best Paper Award was launched by AIMS Allergy and Immunology and the manuscript “Toll-like receptor 9 is involved in the induction of galectin-9 protein by dietary anti-allergic compound fucoidan.” (AIMS Allergy and Immunology, 2023, 7(1): 24–39), and “Dietary and orally-delivered miRNAs: Are they functional and ready to modulate immunity?” (AIMS Allergy and Immunology, 2023, 7(1): 104–131) were the winner.

    AIMS Allergy and Immunology editorial board has 74 members now, and 4 of which joined in 2024. We will continue to renew and accept dedicated researchers to join the Editorial Board in 2025. One of the important strategies of attracting high quality and high impact papers to our journal has been the calls for special issues. In 2024, 3 special issues were established.

    The road is still long and winding but we hope that in 2025, with the support of all the members of the editorial board and reviewers, AIMS Allergy and Immunology can receive and collect more excellent articles to be able to publish. The journal will dedicate to publishing high quality papers by regular issues as well as special issues organized by the members of the editorial board. We believe that all these efforts will increase the impact and citations of the papers published by AIMS Allergy and Immunology.

    Please feel free to let us know your opinions, we will follow your suggestions and make updates to improve AIMS Allergy and Immunology.

    Prof. Marcella Reale, Editor-in-Chief

    AIMS Allergy and Immunology journal

    Dept. of Innovative Technologies in Medicine and Dentistry

    University “G. d'Annunzio” Chieti-Pescara; Via dei Vestini, Chieti, Italy

    In 2024, AIMS Allergy and Immunology journal published 4 issues, a total of 20 articles were published online. The category of published articles is shown in Table 1. These published papers include 11 Research articles, 4 Review articles, 2 Case report, 2 Editorial and 1 Mini review. The journal received a total of 36 submissions (Figure 1). The number of submissions shows an upward trend, and the publishing number remains stable. The publication time (from submission to online) was 117 days, illustrating the efficient review process. In 2024, AIMS Allergy and Immunology places greater emphasis on publishing high-quality articles and hopes to receive more citations.

    Table 1.  The category of published articles.
    Type Number
    Research article 11
    Review 4
    Editorial 2
    Case report 2
    Mini review 1

     | Show Table
    DownLoad: CSV
    Figure 1.  Number of submissions and publications in the past 3 years.

    Figures 2 and 3 provide the counts of submitted and online manuscripts per country. The countrys are derived by affiliation of the corresponding author. In 2024, we received 36 submissions from 17 countries, which shows the diversity of the author distribution. We firmly believe that this widely distributed and powerful group has promoted the development of materials science.

    Figure 2.  Submitted articles by country and region.
    Figure 3.  Online articles by country and region.

    Compared with 2023, the impact factor of AIMS Allergy and Immunology has improved in 2024, which is of great significance for the development of our journal. The following are specific metrics of articles in the journal.

    Table 2 shows the top ten manuscripts in terms of downloads and views in last two years. This will be a motivation for the future promotion of journals and to help them get into the desired databases as early as possible (Data from journal homepage as of December 23, 2024).

    Table 2.  The top 10 articles with highest viewed in last 2 years.
    No. Title Viewed
    1 Migratory dermatographic urticaria following COVID-19 vaccine booster in young adult male 17656
    2 The gastrointestinal effects amongst Ehlers-Danlos syndrome, mast cell activation syndrome and postural orthostatic tachycardia syndrome 6439
    3 Multiple sclerosis and allergic diseases: is there a relationship? 3441
    4 Dietary and orally-delivered miRNAs: are they functional and ready to modulate immunity? 3338
    5 Understanding sex differences in the allergic immune response to food 3216
    6 Epigenetic regulation of the COVID-19 pathogenesis: its impact on the host immune response and disease progression 2705
    7 Mast cells: A dark horse in osteoarthritis treatment 2695
    8 Antibody profiling reveals gender differences in response to SARS-COVID-2 infection 2583
    9 Angioedema-post COVID symptoms 2517
    10 Urinary VPAC1: A potential biomarker in prostate cancer 2490

     | Show Table
    DownLoad: CSV

    Table 3 shows the top five manuscripts in terms of citations in last two years. This will help us increase the visibility and impact of the journal (Data from Web of Science as of December 23, 2024).

    Table 3.  The top 5 articles with highest citations in last 2 years.
    No. Article Citations
    1 Antibody profiling reveals gender differences in response to SARS-COVID-2 infection 5
    2 Migratory dermatographic urticaria following COVID-19 vaccine booster in young adult male 3
    3 Understanding sex differences in the allergic immune response to food 3
    4 Contrast allergies for neurological imaging: When to proceed 3
    5 Impact of etiological factors on citrullination markers and susceptibility of PADI4 allele for CHIKV induced rheumatoid arthritis among South Indian Tamil RA cases 2

     | Show Table
    DownLoad: CSV

    In 2024, AIMS Allergy and Immunology has completed three new special issues. We hope that these three special issues will attract more high-quality submissions in 2025.

    Table 4.  New special issues in 2024.
    No. Special issue Link
    1 Occupational Allergy https://www.aimspress.com/allergy/article/6714/special-articles
    2 Recent Advances in the Diagnosis and Treatment of Kawasaki Disease https://www.aimspress.com/allergy/article/6849/special-articles
    3 The next-generation antibody technologies and biotherapeutics https://www.aimspress.com/allergy/article/6707/special-articles

     | Show Table
    DownLoad: CSV

    AIMS Allergy and Immunology has Editorial Board members representing researchers from 20 countries, representing a diverse range of research experience, expertise and countries. We are constantly assembling the editorial board to be representative to a variety of disciplines across the field of allergy and immunology. AIMS Allergy and Immunology has 74 members now, and 4 of them joined in 2024 (Figure 4). Our EB members are mainly from the United States, Italy, Korea, Australia and Canada. We will continue to invite dedicated experts and researchers in order to renew the Editorial Board in 2025.

    Figure 4.  Country distribution of editorial board members.

    In 2024, our journal developed smoothly. We have received 36 submissions and published 20 papers. The number of submissions increased in 2024 compared with 2023. The processing period (from received to published) and acceptance rate all remains stable. Each submitted article is processed carefully, fairly, promptly, and the accepted papers appear in the journal in the shortest time. In the past year, with the actively support of the guest editor, we have successfully we have successfully established three new special issues. In 2024, with the support of the editorial board members and the editor-in-chief, as well as the contributions of authors and reviewers, the impact factor for AIMS Allergy and Immunology increased from 0.7 to 0.9.

    The goal for us to run this journal is to secure the best scientific authors and papers that ensures Allergy and Immunology to attract more citations and to stay at the forefront of professional publications in allergy and immunology, so that we provide the scientific community with a high-quality journal. In 2025, we expect to publish more articles to enhance the reputation: (1) We will invite more experts in the field of allergy and immunology to publish a review or research article. (2) We will increase influence by soliciting and advertising high quality articles and special issues (topics). (3) We seek to be indexed by more databases. (4) We will continued enlargement of the Editorial Board. (5) We will assign the Best Paper Award for 2024.



    [1] L. Wen, Y. Dong and L. Gao, A new ensemble residual convolutionalneural network for remaining useful life estimation, Math. Biosci. Eng., 16 (2019), 862–880.
    [2] H. D. Shao, H. K. Jiang, X. Zhang, et al., Rolling bearing fault diagnosis using an optimization deepbelief network, Meas. Sci. Tech., 26 (2015), 115002.
    [3] R. Zhao, R. Yan, Z. Chen, et al., Deep learning and itsapplications to machine health monitoring: A survey. arXiv preprint arXiv:1612.07640, 2016.
    [4] M. Cerrada, R. V. Sánchez, C. Li, et al., A review on data-driven fault severityassessment in rolling bearings, Mech. Syst. Signal Proc., 99 (2018), 169–196.
    [5] Y. Bengio, A. Courville and P. Vincent, Representation learning: a review and new perspectives, IEEE T. Pattern Anal. Mach. Intell., 35 (2013), 1798–1828.
    [6] L. Wen, L. Gao and X. Y. Li, A new deep transfer learning based on sparseauto-encoder for fault diagnosis, IEEE T.Syst. Man Cybern. Syst., 49 (2019), 136–144.
    [7] J. L. Wang, J. Zhang and X. X. Wang, A data driven cycle time prediction with featureselection in a semiconductor wafer fabrication system, IEEE T. Semicond. Manuf., 31 (2018), 173–182.
    [8] S. Shao, S. McAleer, R. Yan, et al., Highly-accuratemachine fault diagnosis using deep transfer learning, IEEE T. Ind. Inform., 15 (2019), 2446–2455.
    [9] Z. Y. Wang, C. Lu and B. Zhou, Fault diagnosis for rotary machinery with selectiveensemble neural networks, Mech. Syst.Signal Proc., 113 (2018), 112–130.
    [10] D. H. Wolpert and W. G.Macready, No free lunch theorems for optimization, IEEE T. Evol. Comput., 1 (1997), 67–82.
    [11] H. Y. Sang, Q. K. Pan, J. Q. Li, et al., Effectiveinvasive weed optimization algorithms for distributed assembly permutationflowshop problem with total flowtime criterion, Swarm Evol. Comput., 44 (2019), 64–73.
    [12] X. Y. Li, C. Lu, L. Gao, et al., An Effective Multi-Objective Algorithm forEnergy Efficient Scheduling in a Real-Life Welding Shop, IEEE T. Ind. Inform., 14 (2018), 5400–5409.
    [13] J. Yosinski, J. Clune, Y. Bengio, et al., How transferable arefeatures in deep neural networks? In Advances in neural information processingsystems, (2014), 3320–3328.
    [14] L. Wen, X. Y. Li and L. Gao, A New Two-level Hierarchical Diagnosis Network based onConvolutional Neural Network, IEEE T.Instrum. Meas., (2019).
    [15] H. Y. Sang, Q. K. Pan, P. Y. Duan, et al., Aneffective discrete invasive weed optimization algorithm for lot-streamingflowshop scheduling problems. J. Intell.Manuf., 29 (2018), 1337–1349.
    [16] X.Y. Li, L. Gao, Q. Pan, et al., An effectivehybrid genetic algorithm and variable neighborhood search for integratedprocess planning and scheduling in a packaging machine workshop. IEEE Trans. Syst. Man Cybern. Syst., (2018).
    [17] K. Tidriri, N. Chatti, S. Verron, et al., Bridging data-driven andmodel-based approaches for process fault diagnosis and health monitoring: Areview of researches and future challenges, Annu.Rev. Control, 42 (2016), 63–81.
    [18] Z. Y. Yin and J. Hou, Recent advances on SVMbased fault diagnosis and process monitoring in complicated industrialprocesses, Neurocomputing, 174 (2016), 643–650.
    [19] J. Zheng, L. Gao, H. B. Qiu, et al., Difference mapping methodusing least square support vector regression for variable-fidelityapproximation modelling, Eng. Optimiz., 47 (2015), 719–736.
    [20] T. Han, D. Jiang, Q. Zhao, et al., Comparison of random forest,artificial neural networks and support vector machine for intelligent diagnosisof rotating machinery, Trans. Inst. Meas.Control, (2017), 1–13.
    [21] B. Cai, L. Huang and M. Xie, Bayesian networks in fault diagnosis, IEEE T. Ind. Inform., 13 (2017), 2227–2240.
    [22] L. Wen, L. Gao and X. Y. Li, A new snapshot ensemble convolutional neuralnetwork for fault diagnosis, IEEE Access, 7 (2019), 32037–32047.
    [23] F. Wang, H. K. Jiang, H. D. Shao, et al., An adaptive deepconvolutional neural network for rolling bearing fault diagnosis, Meas. Sci. Technol., 28 (2017), 9.
    [24] J. L. Wang, J. Zhang and X. X. Wang, Bilateral LSTM: Atwo-dimensional long short-term memory model with multiply memory units forshort-term cycle time forecasting in re-entrant manufacturing systems. IEEE T. Ind. Inform., 14 (2018), 748–758.
    [25] J. Pan, Y. Zi, J. Chen, et al., LiftingNet: A novel deep learningnetwork with layerwise feature learning from noisy mechanical data for faultclassification, IEEE T. Ind. Inform., 65 (2018), 4973–4982.
    [26] S. B. Li, G. K. Liu, X. H. Tang, et al., An ensemble deepconvolutional neural network model with improved ds evidence fusion for bearingfault diagnosis, Sensors, 17 (2017), 1729.
    [27] C. Lu, Z. Y. Wang and B. Zhou, Intelligent fault diagnosis of rolling bearing using hierarchicalconvolutional network based health state classification, Adv. Eng. Inform., 32 (2017), 139–151.
    [28] B. Zhang, W. Li, X. Li, et al., Intelligent fault diagnosis undervarying working conditions based on domain adaptive convolutional neural networks, IEEE Access, 6 (2018), 66367–66384.
    [29] J. Donahue, Y. Q. Jia, O, Vinyals, et al., Decaf: A deepconvolutional activation feature for generic visual recognition, Internationalconference on machine learning, (2014),647–655.
    [30] R. Ren, T. Hung and K. C. Tan, A generic deep-learning-based approach for automatedsurface inspection, IEEE T. Cybern., 48 (2018), 929–940.
    [31] J. Wehrmann, G. S. Simoes and R. C. Barros, et al., Adultcontent detection in videos with convolutional and recurrent neural networks, Neurocomputing, 272 (2017), 432–438.
    [32] H. C. Shin, H. R. Roth, M. C. Gao, et al., Deep convolutionalneural networks for computer-aided detection: CNN architectures, datasetcharacteristics and transfer learning, IEEET. Med. Imaging, 35 (2016), 1285–1298.
    [33] E. Rezende, G. Ruppert, T. Carvalho, et al., Malicious softwareclassification using transfer learning of ResNet-50 deep neural network, 201716th IEEE International Conference on Machine Learning and Applications (ICMLA), (2017), 1011–1014.
    [34] O. Janssens, R. Walle, M. Loccufier, et al., Deep learning forinfrared thermal image based machine health monitoring. IEEE-ASME Trans. Mechatron., 23 (2018), 151–159.
    [35] Y. Zhou, W. C. Yi, L. Gao, et al., Adaptive differential evolution with sortingcrossover rate for continuous optimization problems. IEEE T. Cybern., 47 (2017), 2742–2753.
    [36] H. Y. Sang, P. Y. Duan and J. Q. Li, An effective invasive weed optimization algorithm forscheduling semiconductor final testing problem. Swarm Evol. Comput., 38 (2018), 42–53.
    [37] L. K. Hansen and P. Salamon, Neural network ensembles, IEEE T. Pattern Anal. Mach. Intell., 12 (1999), 993–1001.
    [38] H. H. Chen and X. Yao, Regularized negative correlation learning for neuralnetwork ensembles, IEEE T. Neural Netw., 20 (2009), 1962–1979.
    [39] J. C. Fernández, M. Cruz-Ramírez and C. Hervás-Martínez, Sensitivityversus accuracy in ensemble models of artificial neural networks frommulti-objective evolutionary algorithms. NeuralComput. Appl., 30 (2018), 289–305.
    [40] C. Hu, B. D. Youn, P. Wang, et al., Ensemble of data-drivenprognostic algorithms for robust prediction of remaining useful life, Reliab. Eng. Syst. Saf., 1 (2012), 120–35.
    [41] Z. Wu, W. Lin and Y. Ji, An integrated ensemble learning model for imbalanced faultdiagnostics and prognostics. IEEE Access, 6 (2018), 8394–8402.
    [42] U. P. Chong, Signal model-based fault detection and diagnosis forinduction motors using features of vibration signal in two-dimension domain. Strojniski Vestn. J. Mech. Eng., 57 (2011), 655–666.
    [43] L. Wen, X. Y. Li, L. Gao, et al., A new convolutional neuralnetwork based data-driven fault diagnosis method, IEEE Trans. Ind. Electron., 65 (2018), 5990–5998.
    [44] K. M. He, X.Y. Zhang, S. Q. Ren, et al., Deep residuallearning for image recognition, IEEE Conference on Computer Vision and PatternRecognition, (2016),770–778.
    [45] K. M. He, X. Y. Zhang, S. Q. Ren, et al., Delving deep into rectifiers: Surpassing human-level performanceon ImageNet classification. IEEEInternational Conference on Computer Vision, (2015),1026–1034.
    [46] M. Xiao, L. Wen, X. Li, et al., Modelingof the feed-motor transient current in end milling by using varying-coefficientmodel. Math. Probl. Eng., (2015).
    [47] S. Arlot and A. Celisse, A survey ofcross-validation procedures for model selection, Statistics surveys, 4 (2010), 40–79.
    [48] I. H. Witten, E. Frank, M. A. Hall, et al., Data Mining: Practical machinelearning tools and techniques. Morgan Kaufmann, (2016).
    [49] C. Lessmeier, J. K. Kimotho, D. Zimmer, et al., Conditionmonitoring of bearing damage in electromechanical drive systems by using motorcurrent signals of electric motors: A benchmark data set for data-drivenclassification. European Conference of the Prognostics and Health ManagementSociety, 05-08, (2016).
    [50] T. Han, D. Jiang, Q. Zhao, et al., Comparison of random forest, artificial neural networks andsupport vector machine for intelligent diagnosis of rotating machinery. Trans. Inst. Meas. Control, (2017), 1–13.
    [51] Y. H. Chen, G. L. Peng, C. H. Xie, et al., ACDIN: Bridging the gapbetween artificial and real bearing damages for bearing fault diagnosis, Neurocomputing, 294 (2018), 61–71.
    [52] Z. Y. Zhu, G. L. Peng, Y. H. Chen, et al., A convolutional neuralnetwork based on a capsule network with strong generalization for bearing faultdiagnosis, Neurocomputing, 323 (2019), 62–75.
  • Reader Comments
  • © 2019 the Author(s), licensee AIMS Press. This is an open access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/4.0)
通讯作者: 陈斌, bchen63@163.com
  • 1. 

    沈阳化工大学材料科学与工程学院 沈阳 110142

  1. 本站搜索
  2. 百度学术搜索
  3. 万方数据库搜索
  4. CNKI搜索

Metrics

Article views(7700) PDF downloads(1252) Cited by(24)

Other Articles By Authors

/

DownLoad:  Full-Size Img  PowerPoint
Return
Return

Catalog