An interpretable hierarchical semantic convolutional neural network to diagnose melanoma in skin lesions

Hui-Ching Wu; Yu-Chen Tu; Po-Han Chen; Ming-Hseng Tseng; Hui-Ching Wu; Yu-Chen Tu; Po-Han Chen; Ming-Hseng Tseng

doi:10.3934/era.2023094

Electronic Research Archive

2023, Volume 31, Issue 4: 1822-1839. doi: 10.3934/era.2023094

Previous Article Next Article

Research article Special Issues

An interpretable hierarchical semantic convolutional neural network to diagnose melanoma in skin lesions

1.
Department of Medical Sociology and Social Work, Chung Shan Medical University, Taichung 402, Taiwan
2.
Department of Medical Informatics, Chung Shan Medical University, Taichung 402, Taiwan
3.
Information Technology Office, Chung Shan Medical University Hospital, Taichung 402, Taiwan

Received: 28 October 2022 Revised: 31 January 2023 Accepted: 31 January 2023 Published: 08 February 2023

Melanoma is a more dangerous skin cancer than other types of skin cancer because it rapidly spreads to other organs in its early stages. In the increasingly popular task of computer-aided diagnosis using deep learning methods, these models are difficult to interpret and often considered "black boxes". The lack of interpretation of the model prevents the target users from fully understanding it. This study proposes a new interpretable hierarchical semantic convolutional neural network (MEL-HSNet) to diagnose melanoma. The benefits and strength of our approach are a white-box model that not only predicts whether a skin lesion observed in a dermoscopy scan image is melanoma but also provides explanatory information for decision-making. Compared to other convolutional neural networks, the MEL-HSNet model proposed in this study can generate interpretable information on melanoma prediction and obtain significantly better results compared to the other available models.
- melanoma classification,
- skin cancer,
- deep learning,
- convolutional neural networks,
- model interpretability,
- intelligent,
- healthcare
Citation: Hui-Ching Wu, Yu-Chen Tu, Po-Han Chen, Ming-Hseng Tseng. An interpretable hierarchical semantic convolutional neural network to diagnose melanoma in skin lesions[J]. Electronic Research Archive, 2023, 31(4): 1822-1839. doi: 10.3934/era.2023094

Related Papers:

Abstract

Melanoma is a more dangerous skin cancer than other types of skin cancer because it rapidly spreads to other organs in its early stages. In the increasingly popular task of computer-aided diagnosis using deep learning methods, these models are difficult to interpret and often considered "black boxes". The lack of interpretation of the model prevents the target users from fully understanding it. This study proposes a new interpretable hierarchical semantic convolutional neural network (MEL-HSNet) to diagnose melanoma. The benefits and strength of our approach are a white-box model that not only predicts whether a skin lesion observed in a dermoscopy scan image is melanoma but also provides explanatory information for decision-making. Compared to other convolutional neural networks, the MEL-HSNet model proposed in this study can generate interpretable information on melanoma prediction and obtain significantly better results compared to the other available models.

References

[1]	F. Bray, M. Laversanne, E. Weiderpass, I. Soerjomataram, The ever-increasing importance of cancer as a leading cause of premature death worldwide, J. Cancer, 127 (2021), 3029-3030. https://doi.org/10.1002/cncr.33587 doi: 10.1002/cncr.33587
[2]	WHO, Global health estimates 2020: deaths by cause, age, sex, by country and by region, 2000-2019, 2020. Available from: https://www.who.int/data/gho/data/themes/mortality-and-global-health-estimates/ghe-leading-causes-of-death.
[3]	S. Carr, C. Smith, J. Wernberg, Epidemiology and risk factors of melanoma, Surg. Clin. North Am., 100 (2020), 1-12. https://doi.org/10.1016/j.suc.2019.09.005 doi: 10.1016/j.suc.2019.09.005
[4]	D. S. Rigel, J. A. Carucci, Malignant melanoma: prevention, early detection and treatment in the 21st century, CA: Cancer J. Clin., 50 (2000), 215-236. https://doi.org/10.3322/canjclin.50.4.215 doi: 10.3322/canjclin.50.4.215
[5]	I. H. Wolf, J. Smolle, H. P. Soyer, H. Kerl, Sensitivity in the clinical diagnosis of malignant melanoma, J. Melanoma Res., 8 (1998), 425-429. https://doi.org/10.1097/00008390-199810000-00007 doi: 10.1097/00008390-199810000-00007
[6]	M. L. Bafounta, A. Beauchet, P. Aegerter, P. Saiag, Is dermoscopy (epiluminescence microscopy) useful for the diagnosis of melanoma? Results of a meta-analysis using techniques adapted to the evaluation of diagnostic tests, Arch. Dermatol., 137 (2001), 1343-1350. https://doi.org/10.1001/archderm.137.10.1343 doi: 10.1001/archderm.137.10.1343
[7]	H. P. Soyer, G. Argenziano, R. Talamini, S. Chimenti, Is dermoscopy useful for the diagnosis of melanoma, Arch. Dermatol., 137(2001), 1361-1363. https://doi.org/10.1001/archderm.137.10.1361 doi: 10.1001/archderm.137.10.1361
[8]	M. E. Celebi, H. A. Kingravi, B. Uddin, H. Iyatomi, Y. A. Aslandogan, W. V. Stoecker, et al., A methodological approach to the classification of dermoscopy images, Comput. Med. Imaging Graphics, 31(2007), 362-373. https://doi.org/10.1016/j.compmedimag.2007.01.003 doi: 10.1016/j.compmedimag.2007.01.003
[9]	C. Barata, M. E. Celebi, J. S. Marques, A survey of feature extraction in dermoscopy image analysis of skin cancer, IEEE J. Biomed. Health. Inf., 23 (2019), 1096-1109. https://doi.org/10.1109/JBHI.2018.2845939 doi: 10.1109/JBHI.2018.2845939
[10]	T. Y. Wong, N. M. Bressler, Artificial intelligence with deep learning technology looks into diabetic retinopathy screening, JAMA, 316 (2016), 2366-2367. https://doi.org/10.1001/jama.2016.17563 doi: 10.1001/jama.2016.17563
[11]	A. Hosny, C. Parmar, J. Quackenbush, L. H. Schwartz, H. J. W. L. Aerts, Artificial intelligence in radiology, Nat. Rev. Cancer, 18 (2018), 500-510. https://doi.org/10.1038/s41568-018-0016-5 doi: 10.1038/s41568-018-0016-5
[12]	S. Shen, S. X. Han, D. R. Aberle, A. A. Bui, W. Hsu, An interpretable deep hierarchical semantic convolutional neural network for lung nodule malignancy classification, Expert Syst. Appl., 128 (2019), 84-95. https://doi.org/10.1016/j.eswa.2019.01.048 doi: 10.1016/j.eswa.2019.01.048
[13]	D. Popescu, M. El-Khatib, H. El-Khatib, L. Ichim, New trends in melanoma detection using neural networks: a systematic review, Sensors, 22 (2022), 496. https://doi.org/10.3390/s22020496 doi: 10.3390/s22020496
[14]	C. C. Chang, Y. Z. Li, H. C. Wu, M. H. Tseng, Melanoma detection using XGB classifier combined with feature extraction and K-means SMOTE techniques, Diagnostics, 12 (2022), 1747. https://doi.org/10.3390/diagnostics12071747 doi: 10.3390/diagnostics12071747
[15]	E. Nasr-Esfahani, S. Samavi, N. Karimi, S. M. R. Soroushmehr, M. H. Jafari, K. Ward, et al., Melanoma detection by analysis of clinical images using convolutional neural network, in 2016 38th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC), 2016(2016), 1373-1376. https://doi.org/10.1109/EMBC.2016.7590963
[16]	K. Matsunaga, A. Hamada, A. Minagawa, H. Koga, Image classification of melanoma, nevus and seborrheic keratosis by deep neural network ensemble, preprint, arXiv: 1703.03108.
[17]	A. Menegola, J. Tavares, M. Fornaciali, L. T. Li, S. Avila, E. Valle, RECOD titans at ISIC challenge 2017, preprint, arXiv: 1703.04819.
[18]	A. Menegola, J. Tavares, M. Fornaciali, L. T. Li, S. Avila, E.Valle, RECOD Titans participation at the ISBI 2017 challenge - Part 3. Available from: https://github.com/learningtitans/isbi2017-part3.
[19]	A. Mahbod, G. Schaefer, I. Ellinger, R. Ecker, A. Pitiot, C. Wang, Fusing fine-tuned deep features for skin lesion classification, J. Comput. Med. Imaging Graphics, 71 (2019), 19-29. https://doi.org/10.1016/j.compmedimag.2018.10.007 doi: 10.1016/j.compmedimag.2018.10.007
[20]	L. Liu, L. Mou, X. X. Zhu, M. Mandal, Automatic skin lesion classification based on mid-level feature learning, Comput. Med. Imaging Graphics, 84 (2020), 101765. https://doi.org/10.1016/j.compmedimag.2020.101765 doi: 10.1016/j.compmedimag.2020.101765
[21]	I. Iqbal, M. Younus, K. Walayat, M. U. Kakar, J. Ma, Automated multi-class classification of skin lesions through deep convolutional neural network with dermoscopic images, Comput. Med. Imaging Graphics, 88 (2021), 101843. https://doi.org/10.1016/j.compmedimag.2020.101843 doi: 10.1016/j.compmedimag.2020.101843
[22]	I. Gonzalez-Diaz, Dermaknet: incorporating the knowledge of dermatologists to convolutional neural networks for skin lesion diagnosis, IEEE J. Biomed. Health Inf., 23 (2018), 547-559. https://doi.org/10.1109/JBHI.2018.2806962
[23]	K. He, X. Zhang, S. Ren, J. Sun, Deep residual learning for image recognition, in 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), (2016), 770-778. https://doi.org/10.1109/CVPR.2016.90
[24]	S. Banerjee, S. K. Singh, A. Chakraborty, A. Das, R. Bag, Melanoma diagnosis using deep learning and fuzzy logic, Diagnostics, 10 (2020), 577. https://doi.org/10.3390/diagnostics10080577 doi: 10.3390/diagnostics10080577
[25]	I. A. Alfi, Md. M. Rahman, M. Shorfuzzaman, A. Nazir, A non-invasive interpretable diagnosis of melanoma skin cancer using deep learning and ensemble stacking of machine learning models, Diagnostics, 12 (2022). https://doi.org/10.3390/diagnostics12030726
[26]	N. Codella, V. Rotemberg, P. Tschandl, M. E. Celebi, S. Dusza, D. Gutman, et al., Skin lesion analysis toward melanoma detection 2018: a challenge hosted by the international skin imaging collaboration (ISIC), preprint, arXiv: 1902.03368.
[27]	P. Tschandl, C. Rosendahl, H. Kittler, The HAM10000 dataset, a large collection of multi-source dermatoscopic images of common pigmented skin lesions, Sci. Data, 5 (2018), 180161. https://doi.org/10.1038/sdata.2018.161 doi: 10.1038/sdata.2018.161
[28]	C. Shorten, T. M. Khoshgoftaar, A survey on image data augmentation for deep learning, J. Big Data, 6 (2019), 60. https://doi.org/10.1186/s40537-019-0197-0 doi: 10.1186/s40537-019-0197-0
[29]	A. Chattopadhay, A. Sarkar, P. Howlader, V. N. Balasubramanian, Grad-cam++: generalized gradient-based visual explanations for deep convolutional networks, in 2018 IEEE Winter Conference on Applications of Computer Vision (WACV), IEEE, 2018. https://doi.org/10.1109/WACV.2018.00097
[30]	J. A. Almaraz-Damian, V. Ponomaryov, S. Sadovnychiy, H. Castillejos-Fernandez, Melanoma and nevus skin lesion classification using handcraft and deep learning feature fusion via mutual information measures, Entropy, 22 (2020). https://doi.org/10.3390/e22040484
[31]	J. Daghrir, L. Tlig, M. Bouchouicha, M. Sayadi, Melanoma skin cancer detection using deep learning and classical machine learning techniques: a hybrid approach, in 2020 5th International Conference on Advanced Technologies for Signal and Image Processing (ATSIP), IEEE, (2020), 1-5. https://doi.org/10.1109/ATSIP49331.2020.9231544
[32]	R. Caruana, S. Lawrence, L. Giles, Overfitting in neural nets: backpropagation, conjugate gradient and early stopping, in Advances in Neural Information Processing Systems, (2001), 402-408. Available from: https://www.researchgate.net/publication/221620260.

Reader Comments

Your name:*

Email:*
© 2023 the Author(s), licensee AIMS Press. This is an open access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/4.0)