Chinese herbal medicine (CHM) classification is an important and emerging topic in intelligent medicine. However, due to the limited local receptive field, existing Convolutional Neural Network-based methods are susceptible to background interference, which fails to capture discriminative visual cues and hampers the accurate recognition of visually similar herbal categories. To address these limitations, we propose a dual attention enhancement network for Chinese herbal medicine classification. Specifically, we introduce an object localization module to suppress background interference by accurately localizing the target region, thus guiding the network to focus on discriminative regions. Subsequently, we introduce a fused attention module to integrate horizontal and vertical directional information at different feature scales to capture long-range spatial dependencies and enhance the global contextual perception. Moreover, we propose a dual attention module composed of self-attention and cross-attention mechanisms to achieve explicit fusion and the semantic alignment of multi-scale features. Finally, we build a semantic feature enhancement module to further strengthen the inter-layer complementarity through adaptive semantic fusion, thereby improving the discriminative ability and robustness of feature representations. Extensive experimental results on two CHM datasets demonstrate that the proposed method outperforms existing state-of-the-art methods.
Citation: Min Fu, Hanyu Hong. Dual attention enhancement network for Chinese herbal medicine classification[J]. Electronic Research Archive, 2026, 34(6): 3655-3677. doi: 10.3934/era.2026165
Chinese herbal medicine (CHM) classification is an important and emerging topic in intelligent medicine. However, due to the limited local receptive field, existing Convolutional Neural Network-based methods are susceptible to background interference, which fails to capture discriminative visual cues and hampers the accurate recognition of visually similar herbal categories. To address these limitations, we propose a dual attention enhancement network for Chinese herbal medicine classification. Specifically, we introduce an object localization module to suppress background interference by accurately localizing the target region, thus guiding the network to focus on discriminative regions. Subsequently, we introduce a fused attention module to integrate horizontal and vertical directional information at different feature scales to capture long-range spatial dependencies and enhance the global contextual perception. Moreover, we propose a dual attention module composed of self-attention and cross-attention mechanisms to achieve explicit fusion and the semantic alignment of multi-scale features. Finally, we build a semantic feature enhancement module to further strengthen the inter-layer complementarity through adaptive semantic fusion, thereby improving the discriminative ability and robustness of feature representations. Extensive experimental results on two CHM datasets demonstrate that the proposed method outperforms existing state-of-the-art methods.
| [1] |
W. Peng, R. Lauche, C. Ferguson, J. Frawley, J. Adams, D. Sibbritt, Efficacy of Chinese herbal medicine for stroke modifiable risk factors: A systematic review, Chin. Med., 12 (2017), 25–54. https://doi.org/10.1186/s13020-017-0146-9 doi: 10.1186/s13020-017-0146-9
|
| [2] |
S. Hu, D. Luo, Q. Zhu, J. Pan, B. Chen, M. Furian, et al., An updated meta-analysis of Chinese herbal medicine for the prevention of COVID-19 based on western-eastern medicine, Front. Pharmacol., 14 (2023), 1–25. https://doi.org/10.3389/fphar.2023.1257345 doi: 10.3389/fphar.2023.1257345
|
| [3] |
Y. Yang, G. Xiao, P. Cheng, J. Zeng, Y. Liu, Protective application of Chinese herbal compounds and formulae in intestinal inflammation in humans and animals, Molecules, 28 (2023), 6811. https://doi.org/10.3390/molecules28196811 doi: 10.3390/molecules28196811
|
| [4] |
Y. Xi, X. Lu, L. Zhu, X. Sun, Y. Jiang, W. He, et al., Clinical trial for conventional medicine integrated with traditional Chinese medicine (TCM) in the treatment of patients with chronic kidney disease, Medicine, 99 (2020), 20234–20240. https://doi.org/10.1097/md.0000000000020234 doi: 10.1097/md.0000000000020234
|
| [5] |
Y. Gao, Z. Li, Y. Wang, H. Zhang, K. Huang, Y. Fu, et al., Analysis of clinical evidence on traditional Chinese medicine for the treatment of diabetic nephropathy: A comprehensive review with evidence mapping, Front. Endocrinol., 15 (2024), 1324782. https://doi.org/10.3389/fendo.2024.1324782 doi: 10.3389/fendo.2024.1324782
|
| [6] |
Q. Hou, W. Yang, G. Liu, Chinese herbal medicine recognition network based on knowledge distillation and cross-attention, Sci. Rep., 15 (2025), 1687–1702. https://doi.org/10.1038/s41598-025-85697-6 doi: 10.1038/s41598-025-85697-6
|
| [7] |
X. Sun, H. Qian, Chinese herbal medicine image recognition and retrieval by convolutional neural network, PLoS One, 11 (2016), 0156327. https://doi.org/10.1371/journal.pone.0156327 doi: 10.1371/journal.pone.0156327
|
| [8] |
S. S. Chang, H. J. Huang, C. Y. C. Chen, Two birds with one stone? Possible dual-targeting h1n1 inhibitors from traditional Chinese medicine, PLoS Comput. Biol., 7 (2011), 1002315. https://doi.org/10.1371/journal.pcbi.1002315 doi: 10.1371/journal.pcbi.1002315
|
| [9] | N. L. Zhang, S. Yuan, T. Chen, Y. Wang, Hierarchical latent class models and statistical foundation for traditional Chinese medicine, in Conference on Artificial Intelligence in Medicine in Europe, (2007), 139–143. https://doi.org/10.1007/978-3-540-73599-1_15 |
| [10] |
J. M. Yang, P. T. Yu, B. C. Kuo, A nonparametric feature extraction and its application to nearest neighbor classification for hyperspectral image data, IEEE Trans. Geosci. Remote Sens., 48 (2010), 1279–1293. https://doi.org/10.1109/tgrs.2009.2031812 doi: 10.1109/tgrs.2009.2031812
|
| [11] |
P. Rebentrost, M. Mohseni, S. Lloyd, Quantum support vector machine for big data classification, Phys. Rev. Lett., 113 (2014), 130503. https://doi.org/10.1103/physrevlett.113.130503 doi: 10.1103/physrevlett.113.130503
|
| [12] |
D. M. Farid, L. Zhang, C. M. Rahman, M. A. Hossain, R. Strachan, Hybrid decision tree and naïve bayes classifiers for multi-class classification tasks, Expert Syst. Appl., 41 (2014), 1937–1946. https://doi.org/10.1016/j.eswa.2013.08.089 doi: 10.1016/j.eswa.2013.08.089
|
| [13] | Y. Chen, S. Wang, Y. Ge, A survey on the applications of image classification based on convolution neural network, in 2022 IEEE Asia-Pacific Conference on Image Processing, Electronics and Computers (IPEC), (2022), 381–384. https://doi.org/10.1109/ipec54454.2022.9777354 |
| [14] |
C. Mallah, J. Cope, J. Orwell, Plant leaf classification using probabilistic integration of shape, texture and margin features, Signal Process. Pattern Recognit. Appl., 5 (2013), 45–54. https://doi.org/10.2316/p.2013.798-098 doi: 10.2316/p.2013.798-098
|
| [15] | D. Luo, J. Wang, Y. Chen, G. Hamid, Classification of Chinese herbal medicines based on svm, in 2014 International Conference on Information Science, Electronics and Electrical Engineering, (2014), 453–456. https://doi.org/10.1109/infoseee.2014.6948152 |
| [16] |
Y. LeCun, Y. Bengio, G. Hinton, Deep learning, Nature, 521 (2015), 436–444. https://doi.org/10.1038/nature14539 doi: 10.1038/nature14539
|
| [17] |
A. Krizhevsky, I. Sutskever, G. E. Hinton, Imagenet classification with deep convolutional neural networks, Commun. ACM, 60 (2017), 84–90. https://doi.org/10.1145/3065386 doi: 10.1145/3065386
|
| [18] |
L. Mu, Z. Gao, Y. Cui, K. Li, H. Liu, L. Fu, Kiwifruit detection of far-view and occluded fruit based on improved alexnet (in Chinese), Trans. Chin. Soc. Agric. Mach., 50 (2019), 24–34. https://doi.org/10.6041/j.issn.1000-1298.2019.10.003 doi: 10.6041/j.issn.1000-1298.2019.10.003
|
| [19] |
Y. Chen, C. Gong, Y. Liu, X. Fang, Fish identification method based on ftvgg16 convolutional neural network (in Chinese), Trans. Chin. Soc. Agric. Mach., 50 (2019), 223–231. https://doi.org/10.6041/j.issn.1000-1298.2019.05.026 doi: 10.6041/j.issn.1000-1298.2019.05.026
|
| [20] | S. Liu, W. Chen, X. Dong, Automatic classification of Chinese herbal based on deep learning method, in 14th International Conference on Natural Computation, Fuzzy Systems and Knowledge Discovery (ICNC-FSKD), (2018), 235–238. https://doi.org/10.1109/fskd.2018.8687165 |
| [21] |
M. Han, J. Zhang, Y. Zeng, F. Hao, Y. Ren, A novel method of Chinese herbal medicine classification based on mutual learning, Mathematics, 10 (2022), 1557–1566. https://doi.org/10.3390/math10091557 doi: 10.3390/math10091557
|
| [22] |
Y. Kang, M. Wu, Y. Gao, L. Fu, C. Li, Z. Song, Deep learning-based classification of traditional Chinese medicine: A novel approach, Quant. Imaging Med. Surg., 15 (2025), 7483–7496. https://doi.org/10.21037/qims-24-1354 doi: 10.21037/qims-24-1354
|
| [23] | C. Cai, S. Liu, L. Wang, B. Yang, M. Zhi, R. Wang, et al., Classification of Chinese herbal medicine using combination of broad learning system and convolutional neural network, in 2019 IEEE International Conference on Systems, Man and Cybernetics (SMC), (2019), 3907–3912. https://doi.org/10.1109/smc.2019.8914437 |
| [24] |
Q. Liu, X. P. Liu, L. J. Zhang, L. M. Zhao, Image texture feature extraction & recognition of Chinese herbal medicine based on gray level co-occurrence matrix, Adv. Mater. Res., 605 (2013), 2240–2244. https://doi.org/10.4028/www.scientific.net/amr.605-607.2240 doi: 10.4028/www.scientific.net/amr.605-607.2240
|
| [25] | Z. Tang, H. Yang, C. Chen, Weakly supervised posture mining for fine grained classification, in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, (2023), 23735–23744. https://doi.org/10.1109/cvpr52729.2023.02273 |
| [26] | Z. Wang, T. Li, Y. Qin, B. Sheng, Contrastive decoupling: Dynamic regularization for enhanced fine-grained image classification, IEEE Trans. Circuits Syst. Video Technol., (2026), 1–15. https://doi.org/10.1109/tcsvt.2026.3661760 |
| [27] |
A. Bera, Z. Wharton, Y. Liu, N. Bessis, A. Behera, Sr-gnn: Spatial relation aware graph neural network for fine-grained image categorization, IEEE Trans. Image Process., 31 (2022), 6017–6031. https://doi.org/10.1109/tip.2022.3205215 doi: 10.1109/tip.2022.3205215
|
| [28] |
H. Yang, H. He, Y. Li, F. Cao, Revisiting fine-grained classification: A dual branch method for noise-resilient and global-local discriminative learning, Expert Syst. Appl., 295 (2026), 128802. https://doi.org/10.1016/j.eswa.2025.128802 doi: 10.1016/j.eswa.2025.128802
|
| [29] | J. He, J. Chen, S. Liu, A. Kortylewski, C. Yang, Y. Bai, et al., Transfg: A transformer architecture for fine-grained recognition, in Proceedings of the AAAI Conference on Artificial Intelligence, 36 (2022), 852–860. https://doi.org/10.1609/aaai.v36i1.19967 |
| [30] |
Q. Wang, J. Wang, H. Deng, X. Wu, Y. Wang, G. Hao, Aa-trans: Core attention aggregating transformer with information entropy selector for fine-grained visual classification, Pattern Recognit., 140 (2023), 109547. https://doi.org/10.1016/j.patcog.2023.109547 doi: 10.1016/j.patcog.2023.109547
|
| [31] |
J. Wang, Q. Xu, B. Jiang, B. Luo, J. Tang, Multi-granularity part sampling attention for fine-grained visual classification, IEEE Trans. Image Process., 33 (2024), 4529–4542. https://doi.org/10.1109/tip.2024.3441813 doi: 10.1109/tip.2024.3441813
|
| [32] | X. Jiang, H. Tang, J. Gao, X. Du, S. He, Z. Li, Delving into multimodal prompting for fine-grained visual classification, in Proceedings of the 38th AAAI Conference on Artificial Intelligence, 38 (2024), 2570–2578. https://doi.org/10.1609/aaai.v38i3.28034 |
| [33] |
Y. Huang, Z. Hechen, M. Zhou, Z. Li, S. Kwong, An attention-locating algorithm for eliminating background effects in fine-grained visual classification, IEEE Trans. Circuits Syst. Video Technol., 35 (2025), 5993–6006. https://doi.org/10.1109/tcsvt.2025.3535818 doi: 10.1109/tcsvt.2025.3535818
|
| [34] |
S. Yang, J. Wen, B. Fang, Fg-moe: Heterogeneous mixture of experts model for fine-grained visual classification, Pattern Recognit., 175 (2026), 113050. https://doi.org/10.1016/j.patcog.2026.113050 doi: 10.1016/j.patcog.2026.113050
|
| [35] | A. Dosovitskiy, L. Beyer, A. Kolesnikov, D. Weissenborn, X. Zhai, T. Unterthiner, et al., An image is worth 16x16 words: Transformers for image recognition at scale, preprint, arXiv: 2010.11929. |
| [36] | T. Li, F. Sun, R. Sun, L. Wang, M. Li, H. Yang, Chinese herbal medicine classification using convolutional neural network with multiscale images and data augmentation, in 2018 International Conference on Security, Pattern Analysis, and Cybernetics (SPAC), (2018), 109–113. https://doi.org/10.1109/spac46244.2018.8965566 |
| [37] | J. Hu, Y. Wang, Z. Che, Q. Li, H. Jiang, L. Liu, Image recognition of Chinese herbal pieces based on multi-task learning model, in Proceedings of the 2020 IEEE International Conference on Bioinformatics and Biomedicine (BIBM), (2020), 1555–1559. https://doi.org/10.1109/bibm49941.2020.9313412 |
| [38] | C. Xing, Y. Huo, X. Huang, C. Lu, Y. Liang, A. Wang, Research on image recognition technology of traditional Chinese medicine based on deep transfer learning, in 2020 International Conference on Artificial Intelligence and Electromechanical Automation (AIEA), (2020), 140–146. https://doi.org/10.1109/aiea51086.2020.00037 |
| [39] |
Y. Chen, L. Zou, Intelligent screening of pieces of Chinese medicine based on bmfnet-wgan (in Chinese), Chin. J. Exp. Tradit. Med. Formulae, 27 (2021), 107–114. https://doi.org/10.13422/j.cnki.syfjx.20210819 doi: 10.13422/j.cnki.syfjx.20210819
|
| [40] | S. Liu, W. Chen, Z. Li, X. Dong, Chinese herbal classification based on image segmentation and deep learning methods, in Advances in Natural Computation, Fuzzy Systems and Knowledge Discovery, Springer, 89 (2022), 267–275. https://doi.org/10.1007/978-3-030-89698-0_28 |
| [41] |
J. Miao, Y. Huang, Z. Wang, Z. Wu, J. Lv, Image recognition of traditional Chinese medicine based on deep learning, Front. Bioeng. Biotechnol., 11 (2023), 1199803. https://doi.org/10.3389/fbioe.2023.1199803 doi: 10.3389/fbioe.2023.1199803
|
| [42] |
M. Huang, Y. Xu, Image classification of Chinese medicinal flowers based on convolutional neural network, Math. Biosci. Eng., 20 (2023), 14978–14994. https://doi.org/10.3934/mbe.2023671 doi: 10.3934/mbe.2023671
|
| [43] | Q. Hou, D. Zhou, J. Feng, Coordinate attention for efficient mobile network design, in 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), (2021), 13713–13722. https://doi.org/10.1109/cvpr46437.2021.01350 |
| [44] | M. Ghahremani, M. Khateri, B. Jian, B. Wiestler, E. Adeli, C. Wachinger, H-vit: A hierarchical vision transformer for deformable image registration, in 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), (2024), 11513–11523. https://doi.org/10.1109/cvpr52733.2024.01094 |
| [45] | A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, et al., Attention is all you need, Adv. Neural Inf. Process. Syst., 30 (2017), 5998–6008. |
| [46] | Figshare, 2025. https://doi.org/10.6084/m9.figshare.29432726 |
| [47] |
D. Tian, C. Zhou, Y. Wang, R. Zhang, Y. Yao, Nb-tcm-chm: Image dataset of the Chinese herbal medicine fruits and its application in classification through deep learning, Data Brief, 54 (2024), 110405. https://doi.org/10.1016/j.dib.2024.110405 doi: 10.1016/j.dib.2024.110405
|
| [48] |
S. Mehta, M. Rastegari, Mobilevit: Light-weight, general-purpose, and mobile friendly vision transformer, J. Phys. Conf. Ser., 2562 (2023), 012012. https://doi.org/10.1088/1742-6596/2562/1/012012 doi: 10.1088/1742-6596/2562/1/012012
|
| [49] | P. Y. Chou, C. H. Lin, W. C. Kao, A novel plug-in module for fine-grained visual classification, preprint, arXiv: 2202.03822. |
| [50] | R. Du, D. Chang, A. K. Bhunia, J. Xie, Z. Ma, Y. Z. Song, et al., Fine grained visual classification via progressive multi-granularity training of jigsaw patches, in Lecture Notes in Computer Science, Springer, 12370 (2020), 153–168. https://doi.org/10.1007/978-3-030-58565-5_10 |
| [51] | X. Ding, Y. Zhang, Y. Ge, S. Zhao, L. Song, X. Yue, et al., Unireplknet: A universal perception large-kernel convnet for audio video point cloud time series and image recognition, in 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), (2024), 5513–5524. https://doi.org/10.1109/cvpr52733.2024.00527 |
| [52] |
D. Liu, L. Zhao, Y. Wang, J. Kato, Learn from each other to classify better: Cross-layer mutual attention learning for fine-grained visual classification, Pattern Recognit., 140 (2023), 109550. https://doi.org/10.1016/j.patcog.2023.109550 doi: 10.1016/j.patcog.2023.109550
|
| [53] |
D. Yu, Z. Fang, Y. Jiang, Alleviating category confusion in fine-grained visual classification, Visual Comput., 41 (2025), 7417–7432. https://doi.org/10.1007/s00371-025-03814-y doi: 10.1007/s00371-025-03814-y
|
| [54] |
S. Wu, J. Hu, C. Sun, F. Zhong, Q. Zhang, G. Wang, A cross-granularity feature fusion method for fine-grained image recognition, Appl. Intell., 55 (2025), 1–19. https://doi.org/10.1007/s10489-024-05891-3 doi: 10.1007/s10489-024-05891-3
|
| [55] | K. Simonyan, A. Zisserman, Very deep convolutional networks for large-scale image recognition, preprint, arXiv: 1409.1556. |
| [56] | M. Sandler, A. Howard, M. Zhu, A. Zhmoginov, L. C. Chen, Mobilenetv2: Inverted residuals and linear bottlenecks, in 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, (2018), 4510–4520. https://doi.org/10.1109/cvpr.2018.00474 |
| [57] | N. Ma, X. Zhang, H. T. Zheng, J. Sun, Shufflenet v2: Practical guidelines for efficient cnn architecture design, in Lecture Notes in Computer Science, Springer, (2018), 122–138. https://doi.org/10.1007/978-3-030-01264-9_8 |
| [58] | K. Han, Y. Wang, Q. Tian, J. Guo, C. Xu, C. Xu, Ghostnet: More features from cheap operations, in 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), (2020), 1577–1586. https://doi.org/10.1109/cvpr42600.2020.00165 |
| [59] | R. R. Selvaraju, M. Cogswell, A. Das, R. Vedantam, D. Parikh, D. Batra, Grad-cam: Visual explanations from deep networks via gradient-based localization, in 2017 IEEE International Conference on Computer Vision (ICCV), (2017), 618–626. https://doi.org/10.1109/iccv.2017.74 |
| [60] | L. van der Maaten, G. Hinton, Visualizing data using t-SNE, J. Mach. Learn. Res., 9 (2008), 2579–2605. |
Figures(8) / Tables(8)
Min Fu, Hanyu Hong. Dual attention enhancement network for Chinese herbal medicine classification[J]. Electronic Research Archive, 2026, 34(6): 3655-3677. doi: 10.3934/era.2026165
DownLoad: