Accurate diagnosis of liver diseases through the application of deep convolutional neural network on biopsy images

Soumyajit Podder; Abhishek Mallick; Sudipta Das; Kartik Sau; Arijit Roy; Soumyajit Podder; Abhishek Mallick; Sudipta Das; Kartik Sau; Arijit Roy

doi:10.3934/biophy.2023026

AIMS Biophysics

2023, Volume 10, Issue 4: 453-481. doi: 10.3934/biophy.2023026

Previous Article Next Article

Research article Special Issues

Accurate diagnosis of liver diseases through the application of deep convolutional neural network on biopsy images

1.
Department of Electronics, West Bengal State University, Barasat, Kolkata, India 700126
2.
Department of Basic Science and Humanities, University of Engineering and Management, Kolkata, India 700160

Received: 10 July 2023 Revised: 09 October 2023 Accepted: 17 October 2023 Published: 08 November 2023

Accurate detection of non-alcoholic fatty liver disease (NAFLD) through biopsies is challenging. Manual detection of the disease is not only prone to human error but is also time-consuming. Using artificial intelligence and deep learning, we have successfully demonstrated the issues of the manual detection of liver diseases with a high degree of precision. This article uses various neural network-based techniques to assess non-alcoholic fatty liver disease. In this investigation, more than five thousand biopsy images were employed alongside the latest versions of the algorithms. To detect prominent characteristics in the liver from a collection of Biopsy pictures, we employed the YOLOv3, Faster R-CNN, YOLOv4, YOLOv5, YOLOv6, YOLOv7, YOLOv8, and SSD models. A highlighting point of this paper is comparing the state-of-the-art Instance Segmentation models, including Mask R-CNN, U-Net, YOLOv5 Instance Segmentation, YOLOv7 Instance Segmentation, and YOLOv8 Instance Segmentation. The extent of severity of NAFLD and non-alcoholic steatohepatitis was examined for liver cell ballooning, steatosis, lobular, and periportal inflammation, and fibrosis. Metrics used to evaluate the algorithms' effectiveness include accuracy, precision, specificity, and recall. Improved metrics are achieved by optimizing the hyperparameters of the associated models. Additionally, the liver is scored in order to analyse the information gleaned from biopsy images. Statistical analyses are performed to establish the statistical relevance in evaluating the score for different zones.
- hepatic steatosis,
- deep learning,
- convolutional neural network,
- biopsy images,
- scoring of liver,
- instance segmentation
Citation: Soumyajit Podder, Abhishek Mallick, Sudipta Das, Kartik Sau, Arijit Roy. Accurate diagnosis of liver diseases through the application of deep convolutional neural network on biopsy images[J]. AIMS Biophysics, 2023, 10(4): 453-481. doi: 10.3934/biophy.2023026

Related Papers:

Abstract

Accurate detection of non-alcoholic fatty liver disease (NAFLD) through biopsies is challenging. Manual detection of the disease is not only prone to human error but is also time-consuming. Using artificial intelligence and deep learning, we have successfully demonstrated the issues of the manual detection of liver diseases with a high degree of precision. This article uses various neural network-based techniques to assess non-alcoholic fatty liver disease. In this investigation, more than five thousand biopsy images were employed alongside the latest versions of the algorithms. To detect prominent characteristics in the liver from a collection of Biopsy pictures, we employed the YOLOv3, Faster R-CNN, YOLOv4, YOLOv5, YOLOv6, YOLOv7, YOLOv8, and SSD models. A highlighting point of this paper is comparing the state-of-the-art Instance Segmentation models, including Mask R-CNN, U-Net, YOLOv5 Instance Segmentation, YOLOv7 Instance Segmentation, and YOLOv8 Instance Segmentation. The extent of severity of NAFLD and non-alcoholic steatohepatitis was examined for liver cell ballooning, steatosis, lobular, and periportal inflammation, and fibrosis. Metrics used to evaluate the algorithms' effectiveness include accuracy, precision, specificity, and recall. Improved metrics are achieved by optimizing the hyperparameters of the associated models. Additionally, the liver is scored in order to analyse the information gleaned from biopsy images. Statistical analyses are performed to establish the statistical relevance in evaluating the score for different zones.

Acknowledgments

The authors acknowledge Esha Das (Department of Electronics, West Bengal State University) for providing assistance in image management. The authors would also like to acknowledge Somnath Bhattacharjee (Department of Electronics, West Bengal State University) for providing valuable sugessions in executing this research project. The author Abhishek Mallick would like to acknowledge DST, Govt. of India for proving research scholarship under DST-Inspire Scheme.

Conflict of interest

The authors declare no conflicts of interest in this paper.

Author contributions

The first three authors contributed in the execution of this research project and among them, the first author took major role in the execution. The idea of this research project was proposed by the last two authors and they supervised the overall implementation of this research work. All the authors contributed equally to the documentation part of this research project.

References

[1]	Berthold MR, Feelders A, Krempl G Advances in Intelligent Data Analysis XVIII (2020). https://doi.org/10.1007/978-3-030-44584-3
[2]	Byra M, Styczynski G, Szmigielski C, et al. (2018) Transfer learning with deep convolutional neural network for liver steatosis assessment in ultrasound images. Int J Comput Ass Rad 13: 1895-1903. https://doi.org/10.1007/s11548-018-1843-2
[3]	Tsiplakidou M, Tsipouras M, Giannakeas N, et al. (2017) Automated detection of liver histopathological findings based on biopsy image processing. Information 8: 36. https://doi.org/10.3390/info8010036
[4]	Siddiqui MS, Vuppalanchi R, Van Natta ML, et al. (2019) Vibration-controlled transient elastography to assess fibrosis and steatosis in patients with nonalcoholic fatty liver disease. Clin Gastroenterol H 17: 156-163. https://doi.org/10.1016/j.cgh.2018.04.043
[5]	Lin J (2014) Virus-related liver cirrhosis: Molecular basis and therapeutic options. World J Gastroentero 20: 6457. https://doi.org/10.3748/wjg.v20.i21.6457
[6]	Mulay S, Deepika G, Jeevakala S, et al. (2019) Liver segmentation from multimodal images using HED-mask R-CNN. Multiscale Multimodal Medical Imaging : 68-75. https://doi.org/10.1007/978-3-030-37969-8_9
[7]	Sethunath D, Morusu S, Tuceryan M, et al. (2018) Automated assessment of steatosis in murine fatty liver. PLos One 13: e0197242. https://doi.org/10.1371/journal.pone.0197242
[8]	Owjimehr M, Danyali H, Helfroush M (2015) An improved method for liver diseases detection by ultrasound image analysis. J Med Signals Sens 5: 21. https://doi.org/10.4103/2228-7477.150387
[9]	Huang Q, Zhang F, Li X (2018) Machine learning in ultrasound computer-aided diagnostic systems: a survey. BioMed Res Int 2018: 5137904. https://doi.org/10.1155/2018/5137904
[10]	Cao W, An X, Cong L, et al. (2019) Application of deep learning in quantitative analysis of 2-dimensional ultrasound imaging of nonalcoholic fatty liver disease. J Ultras Med 39: 51-59. https://doi.org/10.1002/jum.15070
[11]	Acharya UR, Raghavendra U, Fujita H, et al. (2016) Automated characterization of fatty liver disease and cirrhosis using curvelet transform and entropy features extracted from ultrasound images. Comput Biol Med 79: 250-258. https://doi.org/10.1016/j.compbiomed.2016.10.022
[12]	Naik VN, Gamad RS, Bansod PP (2022) Effect of despeckling filters on the segmentation of ultrasound common carotid artery images. Biomed J 45: 686-695. https://doi.org/10.1016/j.bj.2021.07.002
[13]	Liquori GE, Calamita G, Cascella D, et al. (2009) An innovative methodology for the automated morphometric and quantitative estimation of liver steatosis. Histol Histopathol 24: 49-60. https://doi.org/10.14670/HH-24.49
[14]	Qadir HA, Balasingham I, Solhusvik J, et al. (2020) Improving automatic polyp detection using CNN by exploiting temporal dependency in colonoscopy video. IEEE J Biomed Health 24: 180-193. https://doi.org/10.1109/JBHI.2019.2907434
[15]	Shin Y, Qadir HA, Aabakken L, et al. (2018) Automatic colon polyp detection using region based deep CNN and post learning approaches. IEEE Access 6: 40950-40962. https://doi.org/10.1109/ACCESS.2018.2856402
[16]	Zhang X, Chen F, Yu T, et al. (2019) Real-time gastric polyp detection using convolutional neural networks. PLos One 14: e0214133. https://doi.org/10.1371/journal.pone.0214133
[17]	Nogueira-Rodríguez A, Domínguez-Carbajales R, Campos-Tato F, et al. (2021) Real-time polyp detection model using convolutional neural networks. Neural Comput Appl 34: 10375-10396. https://doi.org/10.1371/journal.pone.0214133
[18]	Lundervold AS, Lundervold A (2019) An overview of deep learning in medical imaging focusing on MRI. Z Med Phys 29: 102-127. https://doi.org/10.1016/j.zemedi.2018.11.002
[19]	Zhen S, Cheng M, Tao Y, et al. (2020) Deep learning for accurate diagnosis of liver tumor based on magnetic resonance imaging and clinical data. Front Oncol 10: 680. https://doi.org/10.3389/fonc.2020.00680
[20]	Yang CK, Lee CY, Wang HS, et al. (2022) Glomerular disease classification and lesion identification by machine learning. Biomed J 45: 675-685. https://doi.org/10.1016/j.bj.2021.08.011
[21]	Ben-Cohen A, Diamant I, Klang E, et al. (2016) Fully convolutional network for liver segmentation and lesions detection. Deep Learning and Data Labeling for Medical Applications : 77-85. https://doi.org/10.1007/978-3-319-46976-8_9
[22]	Tang W, Zou D, Yang S, et al. (2018) DSL: Automatic liver segmentation with faster R-CNN and deepLab. Artificial Neural Networks and Machine Learning–ICANN : 137-147. https://doi.org/10.1007/978-3-030-01421-6_14
[23]	Guo X, Wang F, Teodoro G, et al. (2019) Liver steatosis segmentation with deep learning methods. 2019 IEEE 16th International Symposium on Biomedical Imaging (ISBI 2019) . https://doi.org/10.1109/ISBI.2019.8759600
[24]	Li Q, Dhyani M, Grajo JR, et al. (2018) Current status of imaging in nonalcoholic fatty liver disease. World J Hepatol 10: 530-542. https://doi.org/10.4254/wjh.v10.i8.530
[25]	Zhou JH, Cai JJ, She ZG, et al. (2019) Noninvasive evaluation of nonalcoholic fatty liver disease: Current evidence and practice. World J Gastroentero 25: 1307-1326. https://doi.org/10.3748/wjg.v25.i11.1307
[26]	Podder S, Bhattacharjee S, Roy A (2021) An efficient method of detection of COVID-19 using mask R-CNN on chest X-Ray images. AIMS Biophys 8: 281-290. https://doi.org/10.3934/biophy.2021022
[27]	Ünver HM, Ayan E (2019) Skin lesion segmentation in dermoscopic images with combination of YOLO and GrabCut algorithm. Diagnostics 9: 72. https://doi.org/10.3390/diagnostics9030072
[28]	Alam MM, Islam MT (2019) Machine learning approach of automatic identification and counting of blood cells. Healthc Technol Lett 6: 103-108. https://doi.org/10.1049/htl.2018.5098
[29]	Elsalamony HA (2016) Healthy and unhealthy red blood cell detection in human blood smears using neural networks. Micron 83: 32-41. https://doi.org/10.1016/j.micron.2016.01.008
[30]	Xia K, Yin H (2019) Liver detection algorithm based on an improved deep network ccombined wwith edge perception. IEEE Access 7: 175135-175142. https://doi.org/10.1109/ACCESS.2019.2953517
[31]	Liu W, Anguelov D, Erhan D, et al. (2016) SSD: Single shot multibox detector. Computer Vision – ECCV 2016: 21-37. https://doi.org/10.1007/978-3-319-46448-0_2
[32]	Ren S, He K, Girshick R, et al. (2017) Faster r-cnn: Towards real-time object detection with region proposal networks. IEEE T Pattern Anal 39: 1137-1149. https://doi.org/10.1109/TPAMI.2016.2577031
[33]	Redmon J, Divvala S, Girshick R, et al. (2016) You only look once: Unified, real-time object detection. 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) . https://doi.org/10.1109/CVPR.2016.91
[34]	Shotton J, Kohli P (2014) Semantic image segmentation. Computer Vision : 713-716. https://doi.org/10.1007/978-0-387-31439-6_251
[35]	Tsai H-F, Podder S, Chen P-Y (2023) Microsystem advances through integration with artificial intelligence. Micromachines 14: 826. https://doi.org/10.3390/mi14040826
[36]	Ronneberger O, Fischer P, Brox T (2015) U-Net: Convolutional networks for biomedical image segmentation. Lecture Notes in Computer Science : 234-241. https://doi.org/10.1007/978-3-319-24574-4_28
[37]	He K, Gkioxari G, Dollar P, et al. (2017) Mask R-CNN. 2017 IEEE International Conference on Computer Vision (ICCV) . https://doi.org/10.1109/ICCV.2017.322
[38]	Antonello M, Chiesurin S, Ghidoni S (2020) Enhancing semantic segmentation with detection priors and iterated graph cuts for robotics. Eng Appl Artif Intel 90: 103467. https://doi.org/10.1016/j.engappai.2019.103467
[39]	Heinemann F, Birk G, Stierstorfer B (2019) Deep learning enables pathologist-like scoring of NASH models. Sci Rep 9: 18454. https://doi.org/10.1038/s41598-019-54904
[40]	Heinemann F, Gross P, Zeveleva S, et al. (2022) Deep learning-based quantification of NAFLD/NASH progression in human liver biopsies. Sci Rep 12: 19236. https://doi.org/10.1038/s41598-022-23905-3
[41]	Kůrková V, Manolopoulos Y, Hammer B, et al. (2018) Lecture notes in computer science. Artificial Neural Networks and Machine Learning – ICANN 2018 . https://doi.org/10.1007/978-3-030-01421-6
[42]	Amjoud AB, Amrouch M (2023) Object detection using deep learning, CNNs and vision transformers: a review. IEEE Access 11: 35479-35516. https://doi.org/10.1109/ACCESS.2023.3266093
[43]	Han X, Zhong Y, Zhang L (2017) An efficient and robust integrated geospatial object detection framework for high spatial resolution remote sensing imagery. Rem Sens 9: 666. https://doi.org/10.3390/rs9070666
[44]	Alam MM, Islam MT (2019) Machine learning approach of automatic identification and counting of blood cells. Healthc Technol Lett 6: 103-108. https://doi.org/10.1049/htl.2018.5098
[45]	Pang S, Ding T, Qiao S, et al. (2019) A novel YOLOv3-arch model for identifying cholelithiasis and classifying gallstones on CT images. PLos One 14: e0217647. https://doi.org/10.1371/journal.pone.0217647
[46]	Eldho A, Francis T, Hari CV YOLO based Logo detection, 2019 9th International Conference on Advances in Computing and Communication (ICACC) (2019). https://doi.org/10.1109/ICACC48162.2019.8986207
[47]	Saponara S, Elhanashi A, Gagliardi A (2021) Implementing a real-time, AI-based, people detection and social distancing measuring system for Covid-19. J Real-Time Image Pr 18: 1937-1947. https//doi.org/10.1007/s11554-021-01070-6
[48]	Hussain S, Mubeen I, Ullah N, et al. (2022) Modern diagnostic imaging technique applications and risk factors in the medical field: a review. BioMed Res Int 2022: 1-19. https://doi.org/10.1155/2022/5164970
[49]	Amin J, Anjum MA, Sharif M, et al. (2022) Liver tumor localization based on YOLOv3 and 3D-semantic segmentation using deep neural networks. Diagnostics 12: 823. https://doi.org/10.3390/diagnostics12040823
[50]	Kleiner DE, Makhlouf HR (2016) Histology of nonalcoholic fatty liver disease and nonalcoholic steatohepatitis in adults and children. Clin Liver Dis 20: 293-312. https://doi.org/10.1016/j.cld.2015.10.011
[51]	Brunt EM, Kleiner DE, Wilson LA, et al. (2011) Nonalcoholic fatty liver disease (NAFLD) activity score and the histopathologic diagnosis in NAFLD: distinct clinicopathologic meanings. Hepatology 53: 810-820. https://doi.org/10.1002/hep.24127
[52]	Chalasani N, Wilson L, Kleiner DE, et al. (2008) Relationship of steatosis grade and zonal location to histological features of steatohepatitis in adult patients with non-alcoholic fatty liver disease. J Hepatol 48: 829-834. https://doi.org/10.1016/j.jhep.2008.01.016
[53]	Chalasani NP, Sanyal AJ, Kowdley KV, et al. (2009) Pioglitazone versus vitamin E versus placebo for the treatment of non-diabetic patients with non-alcoholic steatohepatitis: PIVENS trial design. Contemp Clin Trials 30: 88-96. https://doi.org/10.1016/j.cct.2008.09.003
[54]	Angulo P, Kleiner DE, Dam-Larsen S, et al. (2015) Liver fibrosis, but no other histologic features, is associated with long-term outcomes of patients with nonalcoholic fatty liver disease. Gastroenterology 149: 389-397. https://doi.org/10.1053/j.gastro.2015.04.043
[55]	Takahashi Y, Dungubat E, Kusano H, et al. (2023) Artificial intelligence and deep learning: New tools for histopathological diagnosis of nonalcoholic fatty liver disease/nonalcoholic steatohepatitis. Comput Struct Biotec 21: 2495-2501. https://doi.org/10.1016/j.csbj.2023.03.048
[56]	Zambrano-Huailla R, Guedes L, Stefano JT, et al. (2020) Diagnostic performance of three non-invasive fibrosis scores (Hepamet, FIB-4, NAFLD fibrosis score) in NAFLD patients from a mixed Latin American population. Ann Hepatol 19: 622-626. https://doi.org/10.1016/j.aohep.2020.08.066
[57]	Brown GT, Kleiner DE (2016) Histopathology of nonalcoholic fatty liver disease and nonalcoholic steatohepatitis. Metabolism 65: 1080-1086. https://doi.org/10.1016/j.metabol.2015.11.008
[58]	Kleiner DE, Brunt EM, Van Natta M, et al. (2005) Design and validation of a histological scoring system for nonalcoholic fatty liver disease. Hepatology 41: 1313-1321. https://doi.org/10.1002/hep.20701
[59]	Chen JR, Chao YP, Tsai YW, et al. (2020) Clinical value of information entropy compared with deep learning for ultrasound grading of hepatic steatosis. Entropy 22: 1006. https://doi.org/10.3390/e22091006
[60]	Rantakokko P, Männistö V, Airaksinen R, et al. (2015) Persistent organic pollutants and non-alcoholic fatty liver disease in morbidly obese patients: a cohort study. Environ Health 14: 79. https://doi.org/10.1186/s12940-015-0066-z
[61]	Arjmand A, Angelis CT, Christou V, et al. (2019) Training of deep convolutional neural networks to identify critical liver alterations in histopathology image samples. Appl Sci 10: 42. https://doi.org/10.3390/app10010042
[62]	Adegun A, Viriri S (2020) Deep learning techniques for skin lesion analysis and melanoma cancer detection: a survey of state-of-the-art. Artif Intell Rev 54: 811-841. https://doi.org/10.1007/s10462-020-09865-y
[63]	Ugail H, Abubakar A, Elmahmudi A, et al. (2022) The use of pre-trained deep learning models for the photographic assessment of donor livers for transplantation. Artif Intell Surg 2: 101-119. https://doi.org/10.20517/ais.2022.06
[64]	Gaber A, Youness HA, Hamdy A, et al. (2022) Automatic classification of fatty liver disease based on supervised learning and genetic algorithm. Appl Sci 12: 521. https://doi.org/10.3390/app12010521
[65]	Wu CC, Yeh WC, Hsu WD, et al. (2019) Prediction of fatty liver disease using machine learning algorithms. Comput Meth Prog Bio 170: 23-29. https://doi.org/10.1016/j.cmpb.2018.12.032
[66]	Yang R, Zhou Y, Liu W, et al. (2022) Study on the grading model of hepatic steatosis based on improved densenet. J Healthc Eng 2022: 1-8. https://doi.org/10.1155/2022/9601470
[67]	Feng B, Ma XH, Wang S, et al. (2021) Application of artificial intelligence in preoperative imaging of hepatocellular carcinoma: Current status and future perspectives. World J Gastroentero 27: 5341-5350. https://doi.org/10.3748/wjg.v27.i32.5341
[68]	Kistler KD, Brunt EM, Clark JM, et al. (2011) Physical activity recommendations, exercise intensity, and histological severity of nonalcoholic fatty liver disease. Am J Gastroenterol 106: 460-468. https://doi.org/10.1038/ajg.2010.488

Reader Comments

Your name:*

Email:*
© 2023 the Author(s), licensee AIMS Press. This is an open access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/4.0)