Diabetic retinopathy is the leading cause of vision loss in working-age adults. Early screening and diagnosis can help to facilitate subsequent treatment and prevent vision loss. Deep learning has been applied in various fields of medical identification. However, current deep learning-based lesion segmentation techniques rely on a large amount of pixel-level labeled ground truth data, which limits their performance and application. In this work, we present a weakly supervised deep learning framework for eye fundus lesion segmentation in patients with diabetic retinopathy.
First, an efficient segmentation algorithm based on grayscale and morphological features is proposed for rapid coarse segmentation of lesions. Then, a deep learning model named Residual-Attention Unet (RAUNet) is proposed for eye fundus lesion segmentation. Finally, a data sample of fundus images with labeled lesions and unlabeled images with coarse segmentation results is jointly used to train RAUNet to broaden the diversity of lesion samples and increase the robustness of the segmentation model.
A dataset containing 582 fundus images with labels verified by doctors, including hemorrhage (HE), microaneurysm (MA), hard exudate (EX) and soft exudate (SE), and 903 images without labels was used to evaluate the model. In ablation test, the proposed RAUNet achieved the highest intersection over union (IOU) on the labeled dataset, and the proposed attention and residual modules both improved the IOU of the UNet benchmark. Using both the images labeled by doctors and the proposed coarse segmentation method, the weakly supervised framework based on RAUNet architecture significantly improved the mean segmentation accuracy by over 7% on the lesions.
This study demonstrates that combining unlabeled medical images with coarse segmentation results can effectively improve the robustness of the lesion segmentation model and proposes a practical framework for improving the performance of medical image segmentation given limited labeled data samples.
Citation: Yu Li, Meilong Zhu, Guangmin Sun, Jiayang Chen, Xiaorong Zhu, Jinkui Yang. Weakly supervised training for eye fundus lesion segmentation in patients with diabetic retinopathy[J]. Mathematical Biosciences and Engineering, 2022, 19(5): 5293-5311. doi: 10.3934/mbe.2022248
Diabetic retinopathy is the leading cause of vision loss in working-age adults. Early screening and diagnosis can help to facilitate subsequent treatment and prevent vision loss. Deep learning has been applied in various fields of medical identification. However, current deep learning-based lesion segmentation techniques rely on a large amount of pixel-level labeled ground truth data, which limits their performance and application. In this work, we present a weakly supervised deep learning framework for eye fundus lesion segmentation in patients with diabetic retinopathy.
First, an efficient segmentation algorithm based on grayscale and morphological features is proposed for rapid coarse segmentation of lesions. Then, a deep learning model named Residual-Attention Unet (RAUNet) is proposed for eye fundus lesion segmentation. Finally, a data sample of fundus images with labeled lesions and unlabeled images with coarse segmentation results is jointly used to train RAUNet to broaden the diversity of lesion samples and increase the robustness of the segmentation model.
A dataset containing 582 fundus images with labels verified by doctors, including hemorrhage (HE), microaneurysm (MA), hard exudate (EX) and soft exudate (SE), and 903 images without labels was used to evaluate the model. In ablation test, the proposed RAUNet achieved the highest intersection over union (IOU) on the labeled dataset, and the proposed attention and residual modules both improved the IOU of the UNet benchmark. Using both the images labeled by doctors and the proposed coarse segmentation method, the weakly supervised framework based on RAUNet architecture significantly improved the mean segmentation accuracy by over 7% on the lesions.
This study demonstrates that combining unlabeled medical images with coarse segmentation results can effectively improve the robustness of the lesion segmentation model and proposes a practical framework for improving the performance of medical image segmentation given limited labeled data samples.
| [1] |
R. K. Meleppat, K. E. Ronning, S. J. Karlen, K. K. Kothandath, M. E. Burns, E. N. Pugh Jr, et al., In situ morphologic and spectral characterization of retinal pigment epithelium organelles in mice using multicolor confocal fluorescence imaging, Invest. Ophthalmol. Visual Sci., 16 (2020). https://doi.org/10.1167/iovs.61.13.1 doi: 10.1167/iovs.61.13.1
|
| [2] |
R. K. Meleppat, K. E. Ronning, S. J. Karlen, M. E. Burns, E. N. Pugh Jr, R. J. Zawadzki, In vivo multimodal retinal imaging of disease-related pigmentary changes in retinal pigment epithelium, Sci. Rep., 11 (2021), 16252. https://doi.org/10.1038/s41598-021-95320-z doi: 10.1038/s41598-021-95320-z
|
| [3] |
S. Fu, Analysis of 56 cases of type 2 diabetes mellitus with ocular lesions as the first manifestation, Clin. Focus, 22 (2007), 256-257. https://doi.org/10.3969/j.issn.1004-583X.2007.04.013 doi: 10.3969/j.issn.1004-583X.2007.04.013
|
| [4] |
V. Gulshan, L. Peng, M. Coram, M. C. Stumpe, D. Wu, A. Narayanaswamy, et al., Development and validation of a deep learning algorithm for detection of diabetic retinopathy in retinal fundus photographs, JAMA, 316 (2016), 2402-2410. https://doi.org/10.1001/jama.2016.17216 doi: 10.1001/jama.2016.17216
|
| [5] |
S. Sengupta, A. Singh, H. A. Leopold, T. Gulati, V. Lakshminarayanan, Ophthalmic diagnosis using deep learning with fundus images - A critical review, Artif. Intell. Med., 102 (2020), 101758. https://doi.org/10.1016/j.artmed.2019.101758 doi: 10.1016/j.artmed.2019.101758
|
| [6] |
J. Son, J. Y. Shin, H. D. Kim, K. Jung, K. Park, S. J. Park, Development and validation of deep learning models for screening multiple abnormal findings in retinal fundus images, Ophthalmology, 127 (2020), 85-94. https://doi.org/10.1016/j.ophtha.2019.05.029 doi: 10.1016/j.ophtha.2019.05.029
|
| [7] |
L. M. Devi, K. Wahengbam, A. D. Singh, Dehazing buried tissues in retinal fundus images using a multiple radiance pre-processing with deep learning based multiple feature-fusion, Opt. Laser Technol., 138 (2021), 106908. https://doi.org/10.1016/j.optlastec.2020.106908 doi: 10.1016/j.optlastec.2020.106908
|
| [8] |
A. V. Varadarajan, P. Bavishi, P. Ruamviboonsuk, P. Chotcomwongse, S. Venugopalan, A. Narayanaswamy, et al., Predicting optical coherence tomography-derived diabetic macular edema grades from fundus photographs using deep learning, Nat. Commun., 11 (2020), 130. https://doi.org/10.1038/s41467-019-13922-8 doi: 10.1038/s41467-019-13922-8
|
| [9] |
H. N. Veena, A. Muruganandham, T. S. Kumaran, A review on the optic disc and optic cup segmentation and classification approaches over retinal fundus images for detection of glaucoma, SN Appl. Sci., 2 (2020), 1476. https://doi.org/10.1007/s42452-020-03221-z doi: 10.1007/s42452-020-03221-z
|
| [10] | Q. Wu, A. Cheddad, Segmentation-based deep learning fundus image analysis, in 2019 Ninth International Conference on Image Processing Theory, Tools and Applications (IPTA), (2019), 1-5. https://doi.org/10.1109/IPTA.2019.8936078 |
| [11] |
S. Guo, T. Li, H. Kang, N. Li, Y. Zhang, K. Wang, L-Seg: An end-to-end unified framework for multi-lesion segmentation of fundus images, Neurocomputing, 349 (2019), 52-63. https://doi.org/10.1016/j.neucom.2019.04.019 doi: 10.1016/j.neucom.2019.04.019
|
| [12] |
C. Playout, R. Duval, F. Cheriet, A novel weakly supervised multitask architecture for retinal lesions segmentation on fundus images, IEEE Trans. Med. Imaging, 38 (2019), 2434-2444. https://doi.org/10.1109/tmi.2019.2906319 doi: 10.1109/tmi.2019.2906319
|
| [13] |
R. Wang, B. Chen, D. Meng, L. Wang, Weakly supervised lesion detection from fundus images, IEEE Trans. Med. Imaging, 38 (2019), 1501-1512. https://doi.org/10.1109/TMI.2018.2885376 doi: 10.1109/TMI.2018.2885376
|
| [14] |
K. Shankar, A. R. W. Saitet, D. Gupta, S. K. Lakshmanaprabu, A. Khanna, H. M. Pandey, Automated detection and classification of fundus diabetic retinopathy images using synergic deep learning model, Pattern Recognit. Lett., 133 (2020), 210-216. https://doi.org/10.1016/j.patrec.2020.02.026 doi: 10.1016/j.patrec.2020.02.026
|
| [15] | M. Hire, S. Shinde, Ant colony optimization based exudates segmentation in retinal fundus images and classification, in 2018 Fourth International Conference on Computing Communication Control and Automation (ICCUBEA), (2018), 1-6. https://doi.org/10.1109/ICCUBEA.2018.8697727 |
| [16] |
E. Imani, H. Pourreza, A novel method for retinal exudate segmentation using signal separation algorithm, Comput. Methods Programs Biomed., 133 (2016), 195-205. https://doi.org/10.1016/j.cmpb.2016.05.016 doi: 10.1016/j.cmpb.2016.05.016
|
| [17] | V. Sathananthavathi, G. Indumathi, R. Rajalakshmi, Abnormalities detection in retinal fundus images, in 2017 International Conference on Inventive Communication and Computational Technologies (ICICCT), (2017), 89-93. https://doi.org/10.1109/ICICCT.2017.7975165 |
| [18] | O. Oktay, J. Schlemper, L. L. Folgoc1, M. Lee, M. Heinrich, K. Misawa, et al., Attention U-Net: Learning where to look for the pancreas, preprint, arXiv: 1804.03999v3. |
| [19] | N. Ilyasova, A. Shirokanev, N. Demin, R. Paringer, Graph-based segmentation for diabetic macular edema selection in OCT images, in 2019 Fifth International Conference on Frontiers of Signal Processing (ICFSP), (2019), 77-81. https://doi.org/10.1109/ICFSP48124.2019.8938047 |
| [20] | M. Tavakoli, S. Jazani, M. Nazar, Automated detection of microaneurysms in color fundus images using deep learning with different preprocessing approaches, in Medical Imaging 2020: Imaging Informatics for Healthcare, Research, and Applications, 11318 (2020), 113180E. https://doi.org/10.1117/12.2548526 |
| [21] | Z. Yu, C. Feng, M. Y. Liu, S. Ramalingam, CASENet: deep category-aware semantic edge detection, in 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), (2017), 1761-1770. https://doi.org/10.1109/CVPR.2017.191 |
| [22] |
J. Mo, L. Zhang, Y. Feng, Exudate-based diabetic macular edema recognition in retinal images using cascaded deep residual networks, Neurocomputing, 290 (2018), 161-171. https://doi.org/10.1016/j.neucom.2018.02.035 doi: 10.1016/j.neucom.2018.02.035
|
| [23] |
N. Kasabov, N. M. Scott, E. Tu, S. Marks, N. Sengupta, E. Capecci, et al., Evolving spatio-temporal data machines based on the NeuCube neuromorphic framework: Design methodology and selected applications, Neural Networks, 78 (2016), 1-14. https://doi.org/10.1016/j.neunet.2015.09.011 doi: 10.1016/j.neunet.2015.09.011
|
| [24] |
L. K. Abood, Contrast enhancement of infrared images using Adaptive Histogram Equalization (AHE) with Contrast Limited Adaptive Histogram Equalization (CLAHE), Iraqi J. Phys., 16 (2018). https://doi.org/10.30723/ijp.v16i37.84 doi: 10.30723/ijp.v16i37.84
|
| [25] |
O. Ramos-Soto, E. Rodríguez-Esparza, S. E. Balderas-Mata, D. Oliva, A. E. Hassanien, R. K. Meleppat, et al., An efficient retinal blood vessel segmentation in eye fundus images by using optimized top-hat and homomorphic filtering, Comput. Methods Programs Biomed., 201 (2021), 105949. https://doi.org/10.1016/j.cmpb.2021.105949 doi: 10.1016/j.cmpb.2021.105949
|
| [26] | X. Fan, J. Gong, Y. Yan, Red lesion detection in fundus images based on convolution neural network, in 2019 Chinese Control And Decision Conference (CCDC), (2019), 5661-5666. https://doi.org/10.1109/CCDC.2019.8833280 |
| [27] | P. Maragos, 3.3 - Morphological filtering for image enhancement and feature detection, in Handbook of Image and Video Processing (Second Edition), (2005), 135-156. https://doi.org/10.1016/B978-012119792-6/50072-3 |
| [28] |
L. Cheng, J. Xiong, L. He, Non-gaussian statistical timing analysis using second-order polynomial fitting, IEEE Trans. Comput.-Aided Des. Integr. Circuits Syst., 28 (2009), 130-140. https://doi.org/10.1109/TCAD.2008.2009143 doi: 10.1109/TCAD.2008.2009143
|
| [29] | A. Valizadeh, Z. J. Wang, Minimum mean square error detector for multimessage spread spectrum embedding, in 2009 Sixteenth IEEE International Conference on Image Processing (ICIP), (2009), 121-124. https://doi.org/10.1109/ICIP.2009.5414115 |
| [30] | O. Ronneberger, U-Net convolutional networks for biomedical image segmentation, in Bildverarbeitung für die Medizin 2017, (2017), 3. https://doi.org/10.1007/978-3-662-54345-0_3 |
| [31] |
L. Han, Y. Chen, J. Li, B. Zhong, Y. Lei, M. Sun, Liver segmentation with 2.5D perpendicular UNets, Comput. Electr. Eng., 91 (2021), 107118. https://doi.org/10.1016/j.compeleceng.2021.107118 doi: 10.1016/j.compeleceng.2021.107118
|
| [32] | Y. Zhang, H. Lai, W. Yang, Cascade UNet and CH-UNet for thyroid nodule segmentation and benign and malignant classification, in MICCAI 2020: Segmentation, Classification, and Registration of Multi-modality Medical Imaging Data, (2021), 129-134. https://doi.org/10.1007/978-3-030-71827-5_17 |
| [33] |
Z. H. Zhou, A brief introduction to weakly supervised learning, Natl. Sci. Rev., 5 (2018), 44-53. https://doi.org/10.1093/nsr/nwx106 doi: 10.1093/nsr/nwx106
|
| [34] |
T. Li, Y. Gao, K. Wang, S. Guo, H. Liu, H. Kang, Diagnostic assessment of deep learning algorithms for diabetic retinopathy screening, Inf. Sci., 501 (2019), 511-522. https://doi.org/10.1016/j.ins.2019.06.011 doi: 10.1016/j.ins.2019.06.011
|
| [35] |
E. Decencière, X. Zhang, G. Cazuguel, B. Lay, B. Cochener, C. Trone, et al., Feedback on a publicly distributed image database: the messidor database, Image Anal. Stereol., 33 (2014), 231-234. https://doi.org/10.5566/ias.1155 doi: 10.5566/ias.1155
|
| [36] | P. Porwal, S. Pachade, M. Kokare, G. Deshmukh, F. Mériaudeau, IDRiD: Diabetic retinopathy - segmentation and grading challenge, Med. Image Anal., 59 (2020), 101561. |
| [37] | L. C. Chen, Y. Zhu, G. Papandreou, F. Schroff, H. Adam, Encoder-decoder with atrous separable convolution for semantic image segmentation, in Computer Vision - ECCV 2018, 833-851. https://doi.org/10.1007/978-3-030-01234-2_49 |
Figures(9) / Tables(4)
Yu Li, Meilong Zhu, Guangmin Sun, Jiayang Chen, Xiaorong Zhu, Jinkui Yang. Weakly supervised training for eye fundus lesion segmentation in patients with diabetic retinopathy[J]. Mathematical Biosciences and Engineering, 2022, 19(5): 5293-5311. doi: 10.3934/mbe.2022248
DownLoad: