TEP2MP: A text-emotion prediction model oriented to multi-participant text-conversation scenario with hybrid attention enhancement

Huan Rong; Tinghuai Ma; Xinyu Cao; Xin Yu; Gongchi Chen; Huan Rong; Tinghuai Ma; Xinyu Cao; Xin Yu; Gongchi Chen

doi:10.3934/mbe.2022122

Mathematical Biosciences and Engineering

2022, Volume 19, Issue 3: 2671-2699. doi: 10.3934/mbe.2022122

Previous Article Next Article

Research article Special Issues

TEP2MP: A text-emotion prediction model oriented to multi-participant text-conversation scenario with hybrid attention enhancement

1.
School of Artificial Intelligence (School of Future Technology), Nanjing University of Information Science and Technology, Nanjing 210044, China
2.
School of Computer & Software, Nanjing University of Information Science and Technology, Nanjing 210044, China
3.
School of Artificial Intelligence, Nanjing University of Information Science and Technology, Nanjing 210044, China

Academic Editor: Kuen-Suan Chen

Received: 19 October 2021 Revised: 01 January 2021 Accepted: 04 January 2021 Published: 10 January 2022

With the rapid development of online social networks, text-communication has become an indispensable part of daily life. Mining the emotion hidden behind the conversation-text is of prime significance and application value when it comes to the government public-opinion supervision, enterprise decision-making, etc. Therefore, in this paper, we propose a text emotion prediction model in a multi-participant text-conversation scenario, which aims to effectively predict the emotion of the text to be posted by target speaker in the future. Specifically, first, an affective space mapping is constructed, which represents the original conversation-text as an n-dimensional affective vector so as to obtain the text representation on different emotion categories. Second, a similar scene search mechanism is adopted to seek several sub-sequences which contain similar tendency on emotion shift to that of the current conversation scene. Finally, the text emotion prediction model is constructed in a two-layer encoder-decoder structure with the emotion fusion and hybrid attention mechanism introduced at the encoder and decoder side respectively. According to the experimental results, our proposed model can achieve an overall best performance on emotion prediction due to the auxiliary features extracted from similar scenes and the adoption of emotion fusion as well as the hybrid attention mechanism. At the same time, the prediction efficiency can still be controlled at an acceptable level.
- text sentiment analysis,
- time series prediction,
- deep learning
Citation: Huan Rong, Tinghuai Ma, Xinyu Cao, Xin Yu, Gongchi Chen. TEP2MP: A text-emotion prediction model oriented to multi-participant text-conversation scenario with hybrid attention enhancement[J]. Mathematical Biosciences and Engineering, 2022, 19(3): 2671-2699. doi: 10.3934/mbe.2022122

Related Papers:

Abstract

With the rapid development of online social networks, text-communication has become an indispensable part of daily life. Mining the emotion hidden behind the conversation-text is of prime significance and application value when it comes to the government public-opinion supervision, enterprise decision-making, etc. Therefore, in this paper, we propose a text emotion prediction model in a multi-participant text-conversation scenario, which aims to effectively predict the emotion of the text to be posted by target speaker in the future. Specifically, first, an affective space mapping is constructed, which represents the original conversation-text as an n-dimensional affective vector so as to obtain the text representation on different emotion categories. Second, a similar scene search mechanism is adopted to seek several sub-sequences which contain similar tendency on emotion shift to that of the current conversation scene. Finally, the text emotion prediction model is constructed in a two-layer encoder-decoder structure with the emotion fusion and hybrid attention mechanism introduced at the encoder and decoder side respectively. According to the experimental results, our proposed model can achieve an overall best performance on emotion prediction due to the auxiliary features extracted from similar scenes and the adoption of emotion fusion as well as the hybrid attention mechanism. At the same time, the prediction efficiency can still be controlled at an acceptable level.

References

[1]	D. Bertero, F. B. Siddique, C. S. Wu, Y. Wan, R. H. Y. Chan, P. Fung, Real-time speech emotion and sentiment recognition for interactive dialogue systems, in Proceedings of the 2016 conference on empirical methods in natural language processing, (2016), 1042–1047. https://doi.org/10.18653/v1/D16-1110
[2]	Y. Zhang, P. Tiwari, D. Song, X. Mao, P. Wang, X. Li, et al., Learning interaction dynamics with an interactive LSTM for conversational sentiment analysis, Neural Networks, 133 (2021), 40–56. https://doi.org/10.1016/j.neunet.2020.10.001 doi: 10.1016/j.neunet.2020.10.001
[3]	F. Hemmatian, M. K. Sohrabi, A survey on classification techniques for opinion mining and sentiment analysis, Artif. Intell. Rev., 52 (2019), 1495–1545. https://doi.org/10.1007/s10462-017-9599-6 doi: 10.1007/s10462-017-9599-6
[4]	A. Yadav, D. K. Vishwakarma, Sentiment analysis using deep learning architectures: a review, Artif. Intell. Rev., 53 (2020), 4335–4385. https://doi.org/10.1007/s10462-019-09794-5 doi: 10.1007/s10462-019-09794-5
[5]	Y. Song, S. Shi, J. Li, H. Zhang, Directional skip-gram: Explicitly distinguishing left and right context for word embeddings, in Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics, Human Language Technologies, 2 (2018), 175–180. https://doi.org/10.18653/v1/N18-2028
[6]	C. Y. Liou, W. C. Cheng, J. W. Liou, D. R. Liou, Autoencoder for words, Neurocomputing, 139 (2014), 84–96. https://doi.org/10.1016/j.neucom.2013.09.055 doi: 10.1016/j.neucom.2013.09.055
[7]	J. Deng, F. Ren, A survey of textual emotion recognition and its challenges, IEEE Trans. Affective Comput., 2021. https://doi.org/10.1109/TAFFC.2021.3053275 doi: 10.1109/TAFFC.2021.3053275
[8]	I. Chaturvedi, E. Cambria, R. E. Welsch, F. Herrera, Distinguishing between facts and opinions for sentiment analysis: Survey and challenges, Inf. Fusion, 44 (2018), 65–77. https://doi.org/10.1016/j.inffus.2017.12.006 doi: 10.1016/j.inffus.2017.12.006
[9]	E. Cambria, Y. Li, F. Z. Xing, S. Poria, K. Kwok, SenticNet 6: Ensemble application of symbolic and subsymbolic AI for sentiment analysis, in Proceedings of the 29th ACM international conference on information & knowledge management, (2020), 105–114. https://doi.org/10.1145/3340531.3412003
[10]	M. E. Basiri, S. Nemati, M. Abdar, E. Cambria, U. R. Acharya, ABCDM: An attention-based bidirectional CNN-RNN deep model for sentiment analysis, Future Gener. Comput. Syst., 115 (2021), 279–294. https://doi.org/10.1016/j.future.2020.08.005 doi: 10.1016/j.future.2020.08.005
[11]	C. Gong, J. Yu, R. Xia, Unified feature and instance based domain adaptation for end-to-end aspect-based sentiment analysis, in Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), (2020), 7035–7045. https://doi.org/10.18653/v1/2020.emnlp-main.572
[12]	H. Peng, L. Xu, L. Bing, F. Huang, W. Lu, L. Si, Knowing what, how and why: A near complete solution for aspect-based sentiment analysis, in Proceedings of the AAAI Conference on Artificial Intelligence, 34 (2020), 8600–8607. https://doi.org/10.1609/aaai.v34i05.6383
[13]	C. Yang, H. Zhang, B. Jiang, K. Li, Aspect-based sentiment analysis with alternating coattention networks, Inf. Process. Manage., 56 (2019), 463–478. https://doi.org/10.1016/j.ipm.2018.12.004 doi: 10.1016/j.ipm.2018.12.004
[14]	H. Cai, Y. Tu, X. Zhou, J. Yu, R. Xia, Aspect-category based sentiment analysis with hierarchical graph convolutional network, in Proceedings of the 28th International Conference on Computational Linguistics, (2020), 833–843. https://doi.org/10.18653/v1/2020.coling-main.72
[15]	M. H. Phan, P. O. Ogunbona, Modelling context and syntactical features for aspect-based sentiment analysis, in Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, (2020), 3211–3220. https://doi.org/10.18653/v1/2020.acl-main.293
[16]	Y. Ma, K. L. Nguyen, F. Z. Xing, E. Cambria, A survey on empathetic dialogue systems, Inf. Fusion, 64 (2020), 50–70. https://doi.org/10.1016/j.inffus.2020.06.011 doi: 10.1016/j.inffus.2020.06.011
[17]	S. Poria, D. Hazarika, N. Majumder, R. Mihalcea, Beneath the tip of the iceberg: Current challenges and new directions in sentiment analysis research, IEEE Trans. Affective Comput., 2020. https://doi.org/10.1109/TAFFC.2020.3038167 doi: 10.1109/TAFFC.2020.3038167
[18]	W. Li, W. Shao, S. Ji, E. Cambria, BiERU: Bidirectional emotional recurrent unit for conversational sentiment analysis, Neurocomputing, 467 (2022), 73–82. https://doi.org/10.1016/j.neucom.2021.09.057 doi: 10.1016/j.neucom.2021.09.057
[19]	Z. Lian, B. Liu, J. Tao, CTNet: Conversational transformer network for emotion recognition, IEEE/ACM Trans. Audio, Speech, Lang. Process., 29 (2021), 985–1000. https://doi.org/10.1109/TASLP.2021.3049898 doi: 10.1109/TASLP.2021.3049898
[20]	Y. Zhang, P. Tiwari, D. Song, X. Mao, P. Wang, X. Li, et al., Learning interaction dynamics with an interactive LSTM for conversational sentiment analysis, Neural Networks, 133 (2021), 40–56. https://doi.org/10.1016/j.neunet.2020.10.001 doi: 10.1016/j.neunet.2020.10.001
[21]	J. Wang, J. Wang, C. Sun, S. Li, X. Liu, L. Si et al., Sentiment classification in customer service dialogue with topic-aware multi-task learning, in Proceedings of the AAAI Conference on Artificial Intelligence, 34 (2020), 9177–9184. https://doi.org/10.1609/aaai.v34i05.6454
[22]	W. J. Huang, Y. T. Li, Y. Huang, Prediction of chaotic time series using hybrid neural network and attention mechanism, Acta Physica Sinica, 70 (2021), 010501. https://doi.org/10.7498/aps.70.20200899 doi: 10.7498/aps.70.20200899
[23]	Y. Liu, C. Gong, L. Yang, Y. Chen, DSTP-RNN: A dual-stage two-phase attention-based recurrent neural network for long-term and multivariate time series prediction, Expert Syst. Appl., 143 (2020), 113082. https://doi.org/10.1016/j.eswa.2019.113082 doi: 10.1016/j.eswa.2019.113082
[24]	D. Salinas, V. Flunkert, J. Gasthaus, T. Januschowski, DeepAR: Probabilistic forecasting with autoregressive recurrent networks, Int. J. Forecast., 36 (2020), 1181–1191. https://doi.org/10.1016/j.ijforecast.2019.07.001 doi: 10.1016/j.ijforecast.2019.07.001
[25]	J. Koutnik, K. Greff, F. Gomez, J. Schmidhuber, A clockwork rnn, in International Conference on Machine Learning, PMLR, 32 (2014), 1863–1871.
[26]	X. Liu, J. Zheng, Research on time series forecasting based on integrating clockwork recurrent neural network, Comput. Digital Eng., 48 (2020), 1590–1594.
[27]	O. Sagi, L. Rokach, Ensemble learning: A survey, Wiley Interdiscip. Rev., Data Mining Knowl. Discovery, 8 (2018), e1249. https://doi.org/10.1002/widm.1249 doi: 10.1002/widm.1249
[28]	Z. Liu, J. Liu, A robust time series prediction method based on empirical mode decomposition and high-order fuzzy cognitive maps, Knowl.-Based Syst., 203 (2020), 106105. https://doi.org/10.1016/j.knosys.2020.106105 doi: 10.1016/j.knosys.2020.106105
[29]	P. Liu, J. Liu, K. Wu, CNN-FCM: System modeling promotes stability of deep learning in time series prediction, Knowl.-Based Syst., 203 (2020), 106081. https://doi.org/10.1016/j.knosys.2020.106081 doi: 10.1016/j.knosys.2020.106081
[30]	H. Zhou, M. Huang, T. Zhang, X. Zhu, B. Liu, Emotional chatting machine: Emotional conversation generation with internal and external memory, in Thirty-Second AAAI Conference on Artificial Intelligence, 32 (2018), 730–738.
[31]	Y. G. Cinar, H. Mirisaee, P. Goswami, E. Gaussier, A. Aït-Bachir, Period-aware content attention RNNs for time series forecasting with missing values, Neurocomputing, 312 (2018), 177–186. https://doi.org/10.1016/j.neucom.2018.05.090 doi: 10.1016/j.neucom.2018.05.090
[32]	J. Chen, K. Li, H. Rong, K. Bilal, K. Li, S. Y. Philip, A periodicity-based parallel time series prediction algorithm in cloud computing environments, Inf. Sci., 496 (2019), 506–537. https://doi.org/10.1016/j.ins.2018.06.045 doi: 10.1016/j.ins.2018.06.045
[33]	L. K. Hansen, P. Salamon, Neural network ensembles, IEEE Trans. Pattern Anal. Machine Intell., 12 (1990), 993–1001. https://doi.org/10.1109/34.58871 doi: 10.1109/34.58871
[34]	T. Chai, R. R. Draxler, Root mean square error (RMSE) or mean absolute error (MAE), Geosci. Model Dev. Discuss., 7 (2014), 1525–1534. https://doi.org/10.5194/gmdd-7-1525-2014 doi: 10.5194/gmdd-7-1525-2014
[35]	Q. Song, Y. J. Zheng, Y. Xue, W. G. Sheng, M. R. Zhao, An evolutionary deep neural network for predicting morbidity of gastrointestinal infections by food contamination, Neurocomputing, 226 (2017), 16–22. https://doi.org/10.1016/j.neucom.2016.11.018 doi: 10.1016/j.neucom.2016.11.018
[36]	F. Krebs, B. Lubascher, T. Moers, P. Schaap, G. Spanakis, Social emotion mining techniques for Facebook posts reaction prediction, in Proceedings of the 10th International Conference on Agents and Artificial Intelligence (ICAART), 2 (2018), 211–220. https://doi.org/10.5220/0006656002110220
[37]	Y. Qin, D. Song, H. Cheng, W. Cheng, G. Jiang, G. Cottrell, A dual-stage attention-based recurrent neural network for time series prediction, in Proceedings of the 26th International Joint Conference on Artificial Intelligence, (2017), 2627–2633. https://doi.org/10.24963/ijcai.2017/366
[38]	H. Goel, I. Melnyk, A. Banerjee, R2N2, residual recurrent neural networks for multivariate time series forecasting, preprint, arXiv: 1709.03159.
[39]	A. Borovykh, S. Bohte, C. W. Oosterlee, Conditional time series forecasting with convolutional neural networks, preprint, arXiv, 1703.04691.
[40]	G. E. Hinton, R. S. Zemel, Autoencoders, minimum description length, and Helmholtz free energy, Adv. Neural Inf. Process. Syst., 6 (1994), 3–10.

Reader Comments

Your name:*

Email:*
© 2022 the Author(s), licensee AIMS Press. This is an open access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/4.0)