Flower image classification based on an improved lightweight neural network with multi-scale feature fusion and attention mechanism

Zhigao Zeng; Cheng Huang; Wenqiu Zhu; Zhiqiang Wen; Xinpan Yuan; Zhigao Zeng; Cheng Huang; Wenqiu Zhu; Zhiqiang Wen; Xinpan Yuan

doi:10.3934/mbe.2023619

Mathematical Biosciences and Engineering

2023, Volume 20, Issue 8: 13900-13920. doi: 10.3934/mbe.2023619

Previous Article Next Article

Research article

Flower image classification based on an improved lightweight neural network with multi-scale feature fusion and attention mechanism

1.
School of Computer Science, Hunan University of Technology, Zhuzhou, Hunan 412007, China
2.
Hunan Key Laboratory of Intelligent Information Perception and Processing Technology, Zhuzhou, Hunan 412007, China

Academic Editor: Vladimir Mityushev

Received: 23 May 2023 Revised: 04 June 2023 Accepted: 06 June 2023 Published: 19 June 2023

In order to solve the problem that deep learning-based flower image classification methods lose more feature information in the early feature extraction process, and the model takes up more storage space, a new lightweight neural network model based on multi-scale feature fusion and attention mechanism is proposed in this paper. First, the AlexNet model is chosen as the basic framework. Second, a multi-scale feature fusion module (MFFM) is used to replace the shallow single-scale convolution. MFFM, which contains three depthwise separable convolution branches with different sizes, can fuse features with different scales and reduce the feature loss caused by single-scale convolution. Third, two layers of improved Inception module are first added to enhance the extraction of deep features, and a layer of hybrid attention module is added to strengthen the focus of the model on key information at a later stage. Finally, the flower image classification is completed using a combination of global average pooling and fully connected layers. The experimental results demonstrate that our lightweight model has fewer parameters, takes up less storage space and has higher classification accuracy than the baseline model, which helps to achieve more accurate flower image recognition on mobile devices.
- flower image classification,
- multi-scale feature fusion,
- depthwise separable convolution,
- attention mechanism
Citation: Zhigao Zeng, Cheng Huang, Wenqiu Zhu, Zhiqiang Wen, Xinpan Yuan. Flower image classification based on an improved lightweight neural network with multi-scale feature fusion and attention mechanism[J]. Mathematical Biosciences and Engineering, 2023, 20(8): 13900-13920. doi: 10.3934/mbe.2023619

Related Papers:

Abstract

In order to solve the problem that deep learning-based flower image classification methods lose more feature information in the early feature extraction process, and the model takes up more storage space, a new lightweight neural network model based on multi-scale feature fusion and attention mechanism is proposed in this paper. First, the AlexNet model is chosen as the basic framework. Second, a multi-scale feature fusion module (MFFM) is used to replace the shallow single-scale convolution. MFFM, which contains three depthwise separable convolution branches with different sizes, can fuse features with different scales and reduce the feature loss caused by single-scale convolution. Third, two layers of improved Inception module are first added to enhance the extraction of deep features, and a layer of hybrid attention module is added to strengthen the focus of the model on key information at a later stage. Finally, the flower image classification is completed using a combination of global average pooling and fully connected layers. The experimental results demonstrate that our lightweight model has fewer parameters, takes up less storage space and has higher classification accuracy than the baseline model, which helps to achieve more accurate flower image recognition on mobile devices.

References

[1]	H. Hiary, H. Saadeh, M. Saadeh, M. Yaqub, Flower classification using deep convolutional neural networks, IET Comput. Vision, 12 (2018), 855–862. https://doi.org/10.1049/iet-cvi.2017.0155 doi: 10.1049/iet-cvi.2017.0155
[2]	M. E. Nilsback, A. Zisserman, Automated flower classification over a large number of classes, in 2008 Sixth Indian Conference on Computer Vision, Graphics & Image Processing, (2008), 722–729. https://doi.org/10.1109/ICVGIP.2008.47
[3]	B. Fernando, E. Fromont, D. Muselet, M. Sebban, Discriminative feature fusion for image classification, in 2012 IEEE Conference on Computer Vision and Pattern Recognition, (2012), 3434–3441. https://doi.org/10.1109/CVPR.2012.6248084
[4]	A. Angelova, S. Zhu, Efficient object detection and segmentation for fine-grained recognition, in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), (2013), 811–818.
[5]	H. M. Zawbaa, M. Abbass, S. H. Basha, M. Hazman, A. E. Hassenian, An automatic flower classification approach using machine learning algorithms, in 2014 International Conference on Advances in Computing, Communications and Informatics (ICACCI), (2014), 895–901. https://doi.org/10.1109/ICACCI.2014.6968612
[6]	S. Inthiyaz, B. Madhav, P. Kishore, Flower segmentation with level sets evolution controlled by colour, texture and shape features, Cogent Eng., 4 (2017). https://doi.org/10.1080/23311916.2017.1323572 doi: 10.1080/23311916.2017.1323572
[7]	G. E. Hinton, R. R. Salakhutdinov, Reducing the dimensionality of data with neural networks, Science, 313 (2006), 504–507. https://doi.org/10.1126/science.1127647 doi: 10.1126/science.1127647
[8]	A. Krizhevsky, I. Sutskever, G. E. Hinton, ImageNet classification with deep convolutional neural networks, Commun. ACM, 60 (2017), 84–90. https://doi.org/10.1145/3065386 doi: 10.1145/3065386
[9]	K. Simonyan, A. Zisserman, Very deep convolutional networks for large-scale image recognition, arXiv preprint, (2014), arXiv: 1409.1556. https://doi.org/10.48550/arXiv.1409.1556
[10]	C. Szegedy, W. Liu, Y. Q. Jia, P. Sermanet, S. Reed, D. Anguelov, et al., Going deeper with convolutions, in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), (2015), 1–9.
[11]	F. Marzialetti, S. Giulio, M. Malavasi, M. G. Sperandii, A. T. R. Acosta, M. L. Carranza, Capturing coastal dune natural vegetation types using a phenology-based mapping approach: The potential of sentinel-2, Remote Sens., 11 (2019), 1506. https://doi.org/10.3390/rs11121506 doi: 10.3390/rs11121506
[12]	M. Ragab, A. E. Abouelregal, H. F. AlShaibi, R. A. Mansouri, Heat transfer in biological spherical tissues during hyperthermia of magnetoma, Biology, 10 (2021), 1259. https://doi.org/10.3390/biology10121259 doi: 10.3390/biology10121259
[13]	M. Versaci, G. Angiulli, P. Crucitti, D. D. Carlo, F. Laganà, D. Pellicanò, et al., A fuzzy similarity-based approach to classify numerically simulated and experimentally detected carbon fiber-reinforced polymer plate defects, Sensors, 22 (2022), 4232. https://doi.org/10.3390/s22114232 doi: 10.3390/s22114232
[14]	Y. Y. Liu, F. Tang, D. W. Zhou, Y. P. Meng, W. M. Dong, Flower classification via convolutional neural network, in 2016 IEEE International Conference on Functional-Structural Plant Growth Modeling, Simulation, Visualization and Applications (FSPMA), (2016), 110–116. https://doi.org/10.1109/FSPMA.2016.7818296
[15]	S. Cao, B. Song, Visual attentional-driven deep learning method for flower recognition, Math. Biosci. Eng., 18 (2021), 1981–1991.
[16]	X. L. Xia, C. Xu, B. Nan, Inception-v3 for flower classification, in 2017 2nd International Conference on Image, Vision and Computing (ICIVC), (2017), 783–787. https://doi.org/10.1109/ICIVC.2017.7984661
[17]	J. H. Qin, W. Y. Pan, X. X. Xiang, Y. Tan, G. M. Hou, A biological image classification method based on improved CNN, Ecol. Inf., 58 (2020), 101093. https://doi.org/10.1016/j.ecoinf.2020.101093 doi: 10.1016/j.ecoinf.2020.101093
[18]	M. Simon, E. Rodner, Neural activation constellations: Unsupervised part model discovery with convolutional networks, in Proceedings of the IEEE International Conference on Computer Vision (ICCV), (2015), 1143–1151.
[19]	M. Cıbuk, U. Budak, Y. Guo, M. C. Ince, A. Sengur, Efficient deep features selections and classification for flower species recognition, Measurement, 137 (2019), 7–13. https://doi.org/10.1016/j.measurement.2019.01.041 doi: 10.1016/j.measurement.2019.01.041
[20]	K. Bae, J. Park, J. Lee, Y. Lee, C. Lim, Flower classification with modified multimodal convolutional neural networks, Expert Syst. Appl., 159 (2020), 113455. https://doi.org/10.1016/j.eswa.2020.113455 doi: 10.1016/j.eswa.2020.113455
[21]	C. Pang, W. H. Wang, R. S. Lan, Z. Shi, X. N. Luo, Bilinear pyramid network for flower species categorization, Multimedia Tools Appl., 80 (2021), 215–225. https://doi.org/10.1007/s11042-020-09679-8 doi: 10.1007/s11042-020-09679-8
[22]	C. Liu, L. Huang, Z. Q. Wei, W. F. Zhang, Subtler mixed attention network on fine-grained image classification, Appl. Intell., 51 (2021), 7903–7916. https://doi.org/10.1007/s10489-021-02280-y doi: 10.1007/s10489-021-02280-y
[23]	X. Guan, G. Q. Wang, X. Xu, Y. Bin, Learning hierarchal channel attention for fine-grained visual classification, in Proceedings of the 29th ACM International Conference on Multimedia, (2021), 5011–5019. https://doi.org/10.1145/3474085.3475184
[24]	M. Sandler, A. Howard, M. L. Zhu, A. Zhmoginov, L. C. Chen, Mobilenetv2: Inverted residuals and linear bottlenecks, in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), (2018), 4510–4520.
[25]	S. Ioffe, C. Szegedy, Batch normalization: Accelerating deep network training by reducing internal covariate shift, in Proceedings of the 32nd International Conference on Machine Learning, (2015), 448–456.
[26]	C. Szegedy, V. Vanhoucke, S. Ioffe, J. Shlens, Z. Wojna, Rethinking the inception architecture for computer vision, in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), (2016), 2818–2826.
[27]	K. M. He, X. Y. Zhang, S. Q. Ren, J. Sun, Deep residual learning for image recognition, in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), (2016), 770–778.
[28]	C. Szegedy, S. Ioffe, V. Vanhoucke, A. A. Alemi, Inception-v4, inception-resnet and the impact of residual connections on learning, in Proceedings of the AAAI Conference on Artificial Intelligence, (2017), 4278–4284. https://doi.org/10.1609/aaai.v31i1.11231
[29]	S. Chaudhari, V. Mithal, G. Polatkan, R. Ramanath, An attentive survey of attention models, ACM Trans. Intell. Syst. Technol., 12 (2021), 1–32. https://doi.org/10.1145/3465055 doi: 10.1145/3465055
[30]	J. Hu, L. Shen, G. Sun, Squeeze-and-excitation networks, in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), (2018), 7132–7141.
[31]	S. Woo, J. Park, J. Y. Lee, I. S. Kweon, Cbam: Convolutional block attention module, in Proceedings of the European Conference on Computer Vision (ECCV), (2018), 3–19.
[32]	I. Goodfellow, D. W. Farley, M. Mirza, A. Courville, Y. Bengio, Maxout networks, in Proceedings of the 30th International Conference on Machine Learning, (2013), 1319–1327.
[33]	Y. LeCun, L. Bottou, Y. Bengio, P. Haffner, Gradient-based learning applied to document recognition, Proc. IEEE, 86 (1998), 2278–2324. https://doi.org/10.1109/5.726791 doi: 10.1109/5.726791
[34]	M. E. Nilsback, A. Zisserman, A visual vocabulary for flower classification, in 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'06), (2006), 1447–1454. https://doi.org/10.1109/CVPR.2006.42

Reader Comments

Your name:*

Email:*
© 2023 the Author(s), licensee AIMS Press. This is an open access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/4.0)