Medical image fusion is a crucial technology for biomedical diagnoses. However, current fusion methods struggle to balance algorithm design, visual effects, and computational efficiency. To address these challenges, we introduce a novel medical image fusion method based on the multi-scale shearing rolling weighted guided image filter (MSRWGIF). Inspired by the rolling guided filter, we construct the rolling weighted guided image filter (RWGIF) based on the weighted guided image filter. This filter offers progressive smoothing filtering of the image, generating smooth and detailed images. Then, we construct a novel image decomposition tool, MSRWGIF, by replacing non-subsampled shearlet transform's non-sampling pyramid filter with RWGIF to extract richer detailed information. In the first step of our method, we decompose the original images under MSRWGIF to obtain low-frequency subbands (LFS) and high-frequency subbands (HFS). Since LFS contain a large amount of energy-based information, we propose an improved local energy maximum (ILGM) fusion strategy. Meanwhile, HFS employ a fast and efficient parametric adaptive pulse coupled-neural network (AP-PCNN) model to combine more detailed information. Finally, the inverse MSRWGIF is utilized to generate the final fused image from fused LFS and HFS. To test the proposed method, we select multiple medical image sets for experimental simulation and confirm its advantages by combining seven high-quality representative metrics. The simplicity and efficiency of the method are compared with 11 classical fusion methods, illustrating significant improvements in the subjective and objective performance, especially for color medical image fusion.
Citation: Fang Zhu, Wei Liu. A novel medical image fusion method based on multi-scale shearing rolling weighted guided image filter[J]. Mathematical Biosciences and Engineering, 2023, 20(8): 15374-15406. doi: 10.3934/mbe.2023687
Medical image fusion is a crucial technology for biomedical diagnoses. However, current fusion methods struggle to balance algorithm design, visual effects, and computational efficiency. To address these challenges, we introduce a novel medical image fusion method based on the multi-scale shearing rolling weighted guided image filter (MSRWGIF). Inspired by the rolling guided filter, we construct the rolling weighted guided image filter (RWGIF) based on the weighted guided image filter. This filter offers progressive smoothing filtering of the image, generating smooth and detailed images. Then, we construct a novel image decomposition tool, MSRWGIF, by replacing non-subsampled shearlet transform's non-sampling pyramid filter with RWGIF to extract richer detailed information. In the first step of our method, we decompose the original images under MSRWGIF to obtain low-frequency subbands (LFS) and high-frequency subbands (HFS). Since LFS contain a large amount of energy-based information, we propose an improved local energy maximum (ILGM) fusion strategy. Meanwhile, HFS employ a fast and efficient parametric adaptive pulse coupled-neural network (AP-PCNN) model to combine more detailed information. Finally, the inverse MSRWGIF is utilized to generate the final fused image from fused LFS and HFS. To test the proposed method, we select multiple medical image sets for experimental simulation and confirm its advantages by combining seven high-quality representative metrics. The simplicity and efficiency of the method are compared with 11 classical fusion methods, illustrating significant improvements in the subjective and objective performance, especially for color medical image fusion.
| [1] |
M. M. Emam, E. H. Houssein, R. M. Ghoniem, A modified reptile search algorithm for global optimization and image segmentation: Case study brain MRI images, Comput. Biol. Med., 152 (2023), 106404. https://doi.org/10.1016/j.compbiomed.2022.106404 doi: 10.1016/j.compbiomed.2022.106404
|
| [2] |
E. H. Houssein, D. A. Abdelkareem, M. M. Emam, M. A. Hameed, M. Younan, An efficient image segementation method for skin cancer imaging using improved golden jackal optimization algorithm, Comput. Biol. Med., 149 (2022), 106075. https://doi.org/10.1016/j.compbiomed.2022.106075 doi: 10.1016/j.compbiomed.2022.106075
|
| [3] |
W. Zhu, L. Liu, F. Kuang, L. Li, S. Xu, Y. Liang, An efficient multi-threshold image segmentation for skin cancer using boosting whale optimizer, Comput. Biol. Med., 151 (2022), 106227. https://doi.org/10.1016/j.compbiomed.2022.106227 doi: 10.1016/j.compbiomed.2022.106227
|
| [4] |
L. Nie, L. Zhang, L. Meng, X. Song, X. Chang, X. Li, Modeling disease progression via multisource multitask learners: A case study with Alzheimer's disease, IEEE Trans. Neural Networks Learn. Syst., 28 (2017), 1508–1519. https://doi.org/10.1109/TNNLS.2016.2520964 doi: 10.1109/TNNLS.2016.2520964
|
| [5] | J. Tang, Q. Sun, Z. Wang, Y. Cao, Perfect-reconstruction 4-tap size-limited filter banks for image fusion application, in 2007 International Conference on Mechatronics and Automation, (2007), 255–260. https://doi.org/10.1109/ICMA.2007.4303550 |
| [6] |
J. Tang, A contrast based image fusion technique in the DCT domain, Digital Signal Process., 14 (2004), 218–226. https://doi.org/10.1016/j.dsp.2003.06.001 doi: 10.1016/j.dsp.2003.06.001
|
| [7] |
E. Candès, L. Demanet, D. Donoho, L. Ying, Fast discrete curvelet transforms, Multiscale Model. Simul., 5 (2006), 861–899. https://doi.org/10.1137/05064182X doi: 10.1137/05064182X
|
| [8] |
B. Yu, B. Jia, L. Ding, Z. Cai, Q. Wu, R. Law, et al., Hybrid dual-tree complex wavelet transform and support vector machine for digital multi-focus image fusion, Neurocomputing, 182 (2016), 1–9. https://doi.org/10.1016/j.neucom.2015.10.084 doi: 10.1016/j.neucom.2015.10.084
|
| [9] |
Z. Zhu, M. Zheng, G. Qi, D. Wang, Y. Xiang, A phase congruency and local Laplacian energy based multi-modality medical image fusion method in NSCT domain, IEEE Access, 7 (2019), 20811–20824. https://doi.org/10.1109/ACCESS.2019.2898111 doi: 10.1109/ACCESS.2019.2898111
|
| [10] |
M. Yin, X. Liu, Y. Liu, X. Chen, Medical image fusion with parameter-adaptive pulse coupled neural network in nonsubsampled shearlet transform domain, IEEE Trans. Instrum. Meas., 68 (2019), 49–64. https://doi.org/10.1109/TIM.2018.2838778 doi: 10.1109/TIM.2018.2838778
|
| [11] |
H. Ullah, B. Ullah, L. Wu, F. Y. O. Abdalla, G. Ren, Y. Zhao, Multi-modality medical images fusion based on local-features fuzzy sets and novel sum-modified-Laplacian in non-subsampled shearlet transform domain, Biomed. Signal Process. Control, 57 (2020), 101724. https://doi.org/10.1016/j.bspc.2019.101724 doi: 10.1016/j.bspc.2019.101724
|
| [12] |
Z. Zhou, B. Wang, S. Li, M. Dong, Perceptual fusion of infrared and visible images through a hybrid multi-scale decomposition with Gaussian and bilateral filters, Inf. Fusion, 30 (2016), 15–26. https://doi.org/10.1016/j.inffus.2015.11.003 doi: 10.1016/j.inffus.2015.11.003
|
| [13] |
X. Qiu, M. Li, L. Zhang, X. Yuan, Guided filter-based multi-focus image fusion through focus region detection, Signal Process. Image Commun., 72 (2019), 35–46. https://doi.org/10.1016/j.image.2018.12.004 doi: 10.1016/j.image.2018.12.004
|
| [14] |
L. Caraffa, J. P. Tarel, P. Charbonnier, The guided bilateral filter: when the joint/cross bilateral filter becomes robust, IEEE Trans. Image Process., 24 (2015), 1119–1208. https://doi.org/10.1109/TIP.2015.2389617 doi: 10.1109/TIP.2015.2389617
|
| [15] |
L. Jian, X. Yang, Z. Zhou, K. Zhou, K. Liu, Multi-scale image fusion through rolling guidance filter, Future Gener. Comput. Syst., 83 (2018), 310–325. https://doi.org/10.1016/j.future.2018.01.039 doi: 10.1016/j.future.2018.01.039
|
| [16] |
J. Du, W. Li, B. Xiao, Fusion of anatomical and function images using parallel saliency features, Inf. Sci., 430–431 (2018), 567–576. https://doi.org/10.1016/j.ins.2017.12.008 doi: 10.1016/j.ins.2017.12.008
|
| [17] | R. J. Jevnisek, S. Avidan, Co-occurrence filter, in 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), (2017), 3816–3824. https://doi.org/10.1109/CVPR.2017.406 |
| [18] |
Z. Li, J. Zheng, Z. Zhu, W. Yao, S. Wu, Weighted guided image filtering, IEEE Trans. Image Process., 24 (2015), 120–129. https://doi.org/10.1109/TIP.2014.2371234 doi: 10.1109/TIP.2014.2371234
|
| [19] |
H. Yin, Y. Gong, G. Qiu, Side window guided filtering, Signal Process., 165 (2019), 315–330. https://doi.org/10.1016/j.sigpro.2019.07.026 doi: 10.1016/j.sigpro.2019.07.026
|
| [20] |
M. Diwakar, P. Singh, A. Shankar, Multi-modal medical image fusion framework using co-occurrence filter and local extrema in NSST domain, Biomed. Signal Process. Control, 68 (2021), 102788. https://doi.org/10.1016/j.bspc.2021.102788 doi: 10.1016/j.bspc.2021.102788
|
| [21] |
W. Liu, Z. Wang, A novel multi-focus image fusion method using multiscale shearing non-local guided averaging filter, Signal Process., 166 (2020), 107252. https://doi.org/10.1016/j.sigpro.2019.107252 doi: 10.1016/j.sigpro.2019.107252
|
| [22] |
B. Meher, S. Agrawal, R. Panda, A. Abraham, A survey on region based image fusion methods, Inf. Fusion, 48 (2019), 119–132. https://doi.org/10.1016/j.inffus.2018.07.010 doi: 10.1016/j.inffus.2018.07.010
|
| [23] |
X. Li, F. Zhou, H. Tan, W. Zhang, C. Zhao, Multimodal medical image fusion based on joint bilateral filter and local gradient energy, Inf. Sci., 569 (2021), 302–325. https://doi.org/10.1016/j.ins.2021.04.052 doi: 10.1016/j.ins.2021.04.052
|
| [24] |
C. Xing, Z. Wang, Q. Quyang, C. Dong, C. Duan, Image fusion method based on spatially masked convolutional sparse representation, Image Vision Comput., 90 (2019), 103806. https://doi.org/10.1016/j.imavis.2019.08.010 doi: 10.1016/j.imavis.2019.08.010
|
| [25] |
S. Maqsood, U. Javed, Multi-modal medical image fusion based on two-scale image decomposition and sparse representation, Biomed. Signal Process. Control, 57 (2020), 101810. https://doi.org/10.1016/j.bspc.2019.101810 doi: 10.1016/j.bspc.2019.101810
|
| [26] |
S. Goyal, V. Singh, A. Rani, N. Yadav, FPRSGF denoised non-subsampled shearlet transform-based image fusion using sparse representation, Signal Image Video Process., 14 (2020), 719–726. https://doi.org/10.1007/s11760-019-01597-z doi: 10.1007/s11760-019-01597-z
|
| [27] |
F. Zhou, X. Li, M. Zhou, Y. Chen, H. Tan, A new dictionary construction based multimodal medical image fusion framework, Entropy, 21 (2019), 267. https://doi.org/10.3390/e21030267 doi: 10.3390/e21030267
|
| [28] |
Y. Liu, X. Chen, R. K. Ward, Z. J. Wang, Medical image fusion via convolutional sparsity based morphological component analysis, IEEE Signal Process. Lett., 26 (2019), 485–489. https://doi.org/10.1109/lsp.2019.2895749 doi: 10.1109/LSP.2019.2895749
|
| [29] |
Y. Zhang, Y. Liu, P. Sun, H. Yan, X. Zhao, L. Zhang, IFCNN: A general image fusion framework based on convolutional neural network, Inf. Fusion, 54 (2020), 99–118. https://doi.org/10.1016/j.inffus.2019.07.011 doi: 10.1016/j.inffus.2019.07.011
|
| [30] |
H. Li, Y. Wang, Z. Yang, R. Wang, X. Li, D. Tao, Discriminative dictionary learning-based multiple component decomposition for detail-preserving noisy image fusion, IEEE Trans. Instrum. Meas., 69 (2020), 1082–1102. https://doi.org/10.1109/TIM.2019.2912239 doi: 10.1109/TIM.2019.2912239
|
| [31] |
H. Li, M. Yang, Z. Yu, Joint image fusion and super-resolution for enhanced visualization via semi-coupled discriminative dictionary learning and advantage embedding, Neurocomputing, 422 (2021), 62–84. https://doi.org/10.1016/j.neucom.2020.09.024 doi: 10.1016/j.neucom.2020.09.024
|
| [32] |
Q. Hu, S. Hu, F. Zhang, Multi-modality medical image fusion based on separable dictionary learning and Gabor filtering, Signal Process. Image Commun., 83 (2020), 115758. https://doi.org/10.1016/j.image.2019.115758 doi: 10.1016/j.image.2019.115758
|
| [33] |
J. Ma, H. Xu, J. Jiang, X. Mei, X. Zhang, DDcGAN: A dual-discriminator conditional generative adversarial network for multi-resolution image fusion, IEEE Trans. Image Process., 29 (2020), 4980–4995. https://doi.org/10.1109/TIP.2020.2977573 doi: 10.1109/TIP.2020.2977573
|
| [34] |
H. Zhang, H. Xu, X. Tian, J. Jiang, J. Ma, Image fusion meets deep learning: A survey and perspective, Inf. Fusion, 76 (2021), 323–336. https://doi.org/10.1016/j.inffus.2021.06.008 doi: 10.1016/j.inffus.2021.06.008
|
| [35] |
K. Zhan, J. Shi, H. Wang, Y. Xie, Q. Li, Computational mechanisms of pulse-coupled neural networks: A comprehensive review, Arch. Computat. Methods Eng., 24 (2017), 573–588. https://doi.org/10.1007/s11831-016-9182-3 doi: 10.1007/s11831-016-9182-3
|
| [36] |
Y. Chen, S. Park, Y. Ma, R. Ala, A new automatic parameter setting method of a simplified PCNN for image segmentation, IEEE Trans. Neural Networks, 22 (2011). https://doi.org/10.1109/TNN.2011.2128880 doi: 10.1109/TNN.2011.2128880
|
| [37] |
G. Qu, D. Zhang, P. Yan, Information measure for performance of image fusion, Electron. Lett., 38 (2002), 313–315. https://doi.org/10.1049/EL:20020212 doi: 10.1049/el:20020212
|
| [38] |
C. S. Xydeas, V. Petrovic, Objective image fusion performance measure, Electron. Lett., 36 (2000), 308–309. https://doi.org/10.1049/el:20000267 doi: 10.1049/el:20000267
|
| [39] |
Y. Han, Y. Cai, Y. Cao, X. Xu, A new image fusion performance metric based on visual information fidelity, Inf. Fusion, 14 (2013), 127–135. https://doi.org/10.1016/j.inffus.2011.08.002 doi: 10.1016/j.inffus.2011.08.002
|
| [40] |
Y. Chen, R. S. Blum, A new automated quality assessment algorithm for image fusion, Image Vision Comput., 27 (2009), 1421–1432. https://doi.org/10.1016/j.imavis.2007.12.002 doi: 10.1016/j.imavis.2007.12.002
|
| [41] |
M. B. A. Haghighat, A. Aghagolzadeh, H. Seyedarabi, A non-reference image fusion metric based on mutual information of image features, Comput. Electr. Eng., 37 (2011), 744–756. https://doi.org/10.1016/j.compeleceng.2011.07.012 doi: 10.1016/j.compeleceng.2011.07.012
|
| [42] | L. Zhang, H. Li, SR-SIM: A fast and high performance IQA index based on spectral residual, in 2012 19th IEEE International Conference on Image Processing, 19 (2012), 6467149. https://doi.org/10.1109/ICIP.2012.6467149 |
| [43] |
Z. Liu, E. Blasch, Z. Xue, J. Zhao, R. Laganiere, W. Wu, Objective assessment of multiresolution image fusion algorithms for context enhancement in night vision: a comparative study, IEEE Trans. Pattern Anal. Mach. Intell., 34 (2012), 94–109. https://doi.org/10.1109/TPAMI.2011.109 doi: 10.1109/TPAMI.2011.109
|
| [44] |
Z. Zhu, Y. Chai, H. Yin, Y. Li, Z. Liu, A novel dictionary learning approach for multi-modality medical image fusion, Neurocomputing, 214 (2016), 471–482. https://doi.org/10.1016/j.neucom.2016.06.036 doi: 10.1016/j.neucom.2016.06.036
|
| [45] |
F. Zhou, X. Li, M. Zhou, Y. Chen, H. Tan, A new dictionary construction based multimodal medical image fusion framework, Entropy, 21 (2019), 1–20. https://doi.org/10.3390/e21030267 doi: 10.3390/e21030267
|
| [46] |
M. Kim, D. K. Han, H. Ko, Joint patch clustering-based dictionary learning for multimodal image fusion, Inf. Fusion, 27 (2016), 198–214. https://doi.org/10.1016/j.inffus.2015.03.003 doi: 10.1016/j.inffus.2015.03.003
|
| [47] |
C. He, Q. Liu, H. Li, H. Wang, Multimodal medical image fusion based on IHS and PCA, Procedia Eng., 7 (2010), 280–285. https://doi.org/10.1016/j.proeng.2010.11.045 doi: 10.1016/j.proeng.2010.11.045
|
| [48] |
Z. Xu, Medical image fusion using multi-level local extrema, Inf. Fusion, 19 (2014), 38–48. https://doi.org/10.1016/j.inffus.2013.01.001 doi: 10.1016/j.inffus.2013.01.001
|
| [49] |
J. Du, W. Li, B. Xiao, Anatomical-Functional image fusion by information of interest in local Laplacian filtering domain, IEEE Trans. Image Process., 26 (2017), 5855–5866. https://doi.org/10.1109/TIP.2017.2745202 doi: 10.1109/TIP.2017.2745202
|
| [50] | J. Tang, Q. Sun, K. Agyepong, An image enhancement algorithm based on a new contrast measure in the wavelet domain for screening mammograms, in 2007 IEEE International Conference on Image Processing, 5 (2007), 16–19. https://doi.org/10.1109/ICIP.2007.4379757 |
Figures(16) / Tables(5)
Fang Zhu, Wei Liu. A novel medical image fusion method based on multi-scale shearing rolling weighted guided image filter[J]. Mathematical Biosciences and Engineering, 2023, 20(8): 15374-15406. doi: 10.3934/mbe.2023687
DownLoad: