Sign language (SL) recognition for individuals with hearing disabilities involves leveraging machine learning (ML) and computer vision (CV) approaches for interpreting and understanding SL gestures. By employing cameras and deep learning (DL) approaches, namely convolutional neural networks (CNN) and recurrent neural networks (RNN), these models analyze facial expressions, hand movements, and body gestures connected with SL. The major challenges in SL recognition comprise the diversity of signs, differences in signing styles, and the need to recognize the context in which signs are utilized. Therefore, this manuscript develops an SL detection by Improved Coyote Optimization Algorithm with DL (SLR-ICOADL) technique for hearing disabled persons. The goal of the SLR-ICOADL technique is to accomplish an accurate detection model that enables communication for persons using SL as a primary case of expression. At the initial stage, the SLR-ICOADL technique applies a bilateral filtering (BF) approach for noise elimination. Following this, the SLR-ICOADL technique uses the Inception-ResNetv2 for feature extraction. Meanwhile, the ICOA is utilized to select the optimal hyperparameter values of the DL model. At last, the extreme learning machine (ELM) classification model can be utilized for the recognition of various kinds of signs. To exhibit the better performance of the SLR-ICOADL approach, a detailed set of experiments are performed. The experimental outcome emphasizes that the SLR-ICOADL technique gains promising performance in the SL detection process.
Citation: Mashael M Asiri, Abdelwahed Motwakel, Suhanda Drar. Robust sign language detection for hearing disabled persons by Improved Coyote Optimization Algorithm with deep learning[J]. AIMS Mathematics, 2024, 9(6): 15911-15927. doi: 10.3934/math.2024769
Sign language (SL) recognition for individuals with hearing disabilities involves leveraging machine learning (ML) and computer vision (CV) approaches for interpreting and understanding SL gestures. By employing cameras and deep learning (DL) approaches, namely convolutional neural networks (CNN) and recurrent neural networks (RNN), these models analyze facial expressions, hand movements, and body gestures connected with SL. The major challenges in SL recognition comprise the diversity of signs, differences in signing styles, and the need to recognize the context in which signs are utilized. Therefore, this manuscript develops an SL detection by Improved Coyote Optimization Algorithm with DL (SLR-ICOADL) technique for hearing disabled persons. The goal of the SLR-ICOADL technique is to accomplish an accurate detection model that enables communication for persons using SL as a primary case of expression. At the initial stage, the SLR-ICOADL technique applies a bilateral filtering (BF) approach for noise elimination. Following this, the SLR-ICOADL technique uses the Inception-ResNetv2 for feature extraction. Meanwhile, the ICOA is utilized to select the optimal hyperparameter values of the DL model. At last, the extreme learning machine (ELM) classification model can be utilized for the recognition of various kinds of signs. To exhibit the better performance of the SLR-ICOADL approach, a detailed set of experiments are performed. The experimental outcome emphasizes that the SLR-ICOADL technique gains promising performance in the SL detection process.
| [1] | N. Basnin, L. Nahar, M. S. Hossain, An integrated CNN-lSTM model for bangla lexical sign language recognition, in Proc. of Int. Conf. on Trends in Computational and Cognitive Engineering, Advances in Intelligent Systems and Computing Book Series, Springer, Singapore, 1309 (2020), 695–707. https://doi.org/10.1007/978-981-33-4673-4_57 |
| [2] | C. U. Bharathi, G. Ragavi, K. Karthika, Signtalk: Sign language to text and speech conversion, in 2021 Int. Conf. on Advancements in Electrical, Electronics, Communication, Computing and Automation (ICAECA), Coimbatore, India, 2021, 1–4. https://doi.org/10.1109/ICAECA52838.2021.9675751 |
| [3] | F. Beser, M. A. Kizrak, B. Bolat, T. Yildirim, Recognition of sign language using capsule networks, in 2018 26th Signal Processing and Communications Applications Conf. (SIU), Izmir, Turkey, 2018, 1–4. https://doi.org/10.1109/ICAECA52838.2021.9675751 |
| [4] | M. A. Ahmed, B. B. Zaidan, A. A. Zaidan, M. M. Salih, M. M. b. Lakulu, A review on systems-based sensory gloves for sign language recognition state of the art between 2007 and 2017, Sensors, 18 (2018), 2208. https://doi.org/10.3390/s18072208 |
| [5] |
Y. Dai, Z. Luo, Review of unsupervised person re-identification, J. New Media, 3 (2021), 129–136. https://doi.org/10.32604/jnm.2021.023981 doi: 10.32604/jnm.2021.023981
|
| [6] |
K. Snoddon, Sign language planning and policy in ontario teacher education, Lang. Policy, 20 (2021), 577–598. https://doi.org/10.1007/s10993-020-09569-7 doi: 10.1007/s10993-020-09569-7
|
| [7] |
H. M. Mohammdi, D. M. Elbourhamy, An intelligent system to help deaf students learn arabic sign language, Interact. Learn. Envir., 2021, 1–16. https://doi.org/10.1080/10494820.2021.1920431 doi: 10.1080/10494820.2021.1920431
|
| [8] | M. P. Kumar, M. Thilagaraj, S. Sakthivel, C. Maduraiveeran, M. P. Rajasekaran, S. Rama, Sign language translator using LabVIEW enabled with internet of things, Smart Intelligent Computing and Applications, Smart Innovation, Systems and Technologies Book Series, Springer, Singapore, 104 (2019), 603–612. https://doi.org/10.1007/978-981-13-1921-1_59 |
| [9] |
X. R. Zhang, J. Zhou, W. Sun, S. K. Jha, A lightweight CNN based on transfer learning for COVID-19 diagnosis, Comput. Mater. Con., 72 (2022), 1123–1137. https://doi.org/10.32604/cmc.2022.024589 doi: 10.32604/cmc.2022.024589
|
| [10] |
A. Kumar, R. Kumar, A novel approach for iSL alphabet recognition using extreme learning machine, Int. J. Inf. Technol., 13 (2021), 349–357. https://doi.org/10.1007/s41870-020-00525-6 doi: 10.1007/s41870-020-00525-6
|
| [11] |
B. B. Al-onazi, M. K. Nour, H. Alshahran, M. A. Elfaki, M. M. Alnfiai, R. Marzouk, et al., Arabic sign language gesture classification using Deer Hunting Optimization with machine learning model, Comput. Mater. Con., 75 (2023). https://doi.org/10.1007/s41870-020-00525-6 doi: 10.1007/s41870-020-00525-6
|
| [12] | M. Potnis, D. Raul, M. Inamdar, Recognition of Indian Sign Language using Machine Learning Algorithms, In 2021 8th International Conference on Signal Processing and Integrated Networks (SPIN), IEEE, 2021,579–584. https://doi.org/10.1007/s41870-020-00525-6 |
| [13] | S. Aliyev, A. Abd Almisreb, S. Turaev, Azerbaijani sign language recognition using machine learning approach, In Journal of Physics: Conference Series, IOP Publishing, 2251 (2022), 012007. https://doi.org/10.1088/1742-6596/2251/1/012007 |
| [14] |
M. M. Asiri, A. Motwakel, S. Drar, Enhanced Bald Eagle Search Optimizer with Transfer Learning-based Sign Language Recognition for Hearing-impaired Persons, J. Disabil. Res., 2 (2023), 86–93. https://doi.org/10.1088/1742-6596/2251/1/012007 doi: 10.1088/1742-6596/2251/1/012007
|
| [15] | S. Anthoniraj, V. Ganashree, B. J. R. Umdor, G. D. Sai, B. R. Navya, Sign Language Interpreter Using Machine Learning, In 2021 International Conference on Advancements in Electrical, Electronics, Communication, Computing and Automation (ICAECA), (pp. 1–6). IEEE, 2021. https://doi.org/10.1109/ICAECA52838.2021.9675693 |
| [16] |
M. Zakariah, Y. A. Alotaibi, D. Koundal, Y. Guo, M. Mamun Elahi, Sign language recognition for Arabic alphabets using transfer learning technique, Comput. Intell. Neurosci., 2022 (2022), Article ID 4567989. https://doi.org/10.1155/2022/4567989 doi: 10.1155/2022/4567989
|
| [17] |
N. K. Bose Duraimutharasan, K. Sangeetha, Machine Learning and Vision Based Techniques for Detecting and Recognizing Indian Sign Language, Revue d'Intelligence Artificielle, 37 (2023). https://doi.org/10.1109/ICAECA52838.2021.9675693 doi: 10.1109/ICAECA52838.2021.9675693
|
| [18] | M. Marais, D. Brown, J. Connan, A. Boby, Improving signer-independence using pose estimation and transfer learning for sign language recognition, In International Advanced Computing Conference, (pp. 415–428), Cham: Springer Nature Switzerland, 2022. https://doi.org/10.1109/ICAECA52838.2021.9675693 |
| [19] | B. Desai, U. Kushwaha, S. Jha, M. NMIMS, Image filtering-techniques algorithms and applications, Applied GIS, 7 (2020), 101. |
| [20] | M. Neshat, M. Ahmedb, H. Askarid, M. Thilakaratnee, S. Mirjalilia, Hybrid Inception Architecture with Residual Connection: Fine-tuned Inception-ResNet Deep Learning Model for Lung Inflammation Diagnosis from Chest Radiographs, 2023. arXiv preprint arXiv: 2310.02591. |
| [21] |
A. Sari, A. Majdi, M. J. C. Opulencia, A. Timoshin, D. T. N. Huy, N. D. Trung, et al., New optimized configuration for a hybrid PV/diesel/battery system based on coyote optimization algorithm: A case study for Hotan county, Energy Rep., 8 (2022), 15480–15492. https://doi.org/10.1016/j.egyr.2022.11.059 doi: 10.1016/j.egyr.2022.11.059
|
| [22] |
H. Dehghanisanij, S. Emami, V. Rezaverdinejad, A. Amini, Potential of the hazelnut tree search–ELM hybrid approach in estimating yield and water productivity, Appl. Water Sci., 13 (2023), 61. https://doi.org/10.1016/j.egyr.2022.11.059 doi: 10.1016/j.egyr.2022.11.059
|
| [23] | https://www.kaggle.com/datasets/grassknoted/asl-alphabet |
| [24] |
F. Alrowais, S. S. Alotaibi, S. Dhahbi, R. Marzouk, A. Mohamed, A. M. Hilal, Sign Language Recognition and Classification Model to Enhance Quality of Disabled People, Networks, 9 (2022), 10. https://doi.org/10.32604/cmc.2022.029438 doi: 10.32604/cmc.2022.029438
|
Figures(8) / Tables(3)
Mashael M Asiri, Abdelwahed Motwakel, Suhanda Drar. Robust sign language detection for hearing disabled persons by Improved Coyote Optimization Algorithm with deep learning[J]. AIMS Mathematics, 2024, 9(6): 15911-15927. doi: 10.3934/math.2024769
DownLoad: