Plot-aware transformer for recommender systems

Suhua Wang; Zhen Huang; Bingjie Zhang; Xiantao Heng; Yeyi Jiang; Xiaoxin Sun; Suhua Wang; Zhen Huang; Bingjie Zhang; Xiantao Heng; Yeyi Jiang; Xiaoxin Sun

doi:10.3934/era.2023160

Electronic Research Archive

2023, Volume 31, Issue 6: 3169-3186. doi: 10.3934/era.2023160

Previous Article Next Article

Research article Special Issues

Plot-aware transformer for recommender systems

1.
Computer Department, Changchun Humanities and Sciences College, Changchun 130117, China
2.
Department of Computer Science, University of Science and Technology of China, Hefei 230026, China
3.
School of Information Science and Technology, Northeast Normal University, Changchun 130117, China

Academic Editor: William Guo

Received: 27 November 2022 Revised: 14 March 2023 Accepted: 14 March 2023 Published: 23 March 2023

Plot text is very valuable supporting information in movie recommendations. It has several characteristics: 1) It is rich in content. Each movie often has a document of more than 200 words to describe it, which can give the movie a rich semantic meaning. 2) Objectivity. Plot texts are different from review information. A movie may have thousands of reviews with mixed and conflicting opinions. However, a film has only one plot text, which is fair in tone and does not take a position. Despite its appealing properties and potential for accurate movie portrayal, the lack of a building block for effectively mining plot semantics has led to the marginalization of plot text in the design of movie recommendation algorithms. Therefore, in this paper, we explore the application of the Transformer, currently the best natural language processing module, to learning movie plot texts to help achieve more accurate rating prediction. We propose the "Plot-Aware Transformer" model (PAT) to model the process of "user-movie" rating interaction. We test the PAT model on several movie datasets and demonstrated that the model is competitive. In all tasks, PAT achieves state-of-the-art performance compared to baseline experiments.
- Plot-Aware Transformer,
- embedding representation,
- recommendation algorithms,
- semantics mining
Citation: Suhua Wang, Zhen Huang, Bingjie Zhang, Xiantao Heng, Yeyi Jiang, Xiaoxin Sun. Plot-aware transformer for recommender systems[J]. Electronic Research Archive, 2023, 31(6): 3169-3186. doi: 10.3934/era.2023160

Related Papers:

Abstract

Plot text is very valuable supporting information in movie recommendations. It has several characteristics: 1) It is rich in content. Each movie often has a document of more than 200 words to describe it, which can give the movie a rich semantic meaning. 2) Objectivity. Plot texts are different from review information. A movie may have thousands of reviews with mixed and conflicting opinions. However, a film has only one plot text, which is fair in tone and does not take a position. Despite its appealing properties and potential for accurate movie portrayal, the lack of a building block for effectively mining plot semantics has led to the marginalization of plot text in the design of movie recommendation algorithms. Therefore, in this paper, we explore the application of the Transformer, currently the best natural language processing module, to learning movie plot texts to help achieve more accurate rating prediction. We propose the "Plot-Aware Transformer" model (PAT) to model the process of "user-movie" rating interaction. We test the PAT model on several movie datasets and demonstrated that the model is competitive. In all tasks, PAT achieves state-of-the-art performance compared to baseline experiments.

References

[1]	A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, et al., Attention is all you need, Adv. Neural Inf. Process. Sys. 30, 2017 (2017).
[2]	X. Wang, K. Zhou, J. R. Wen, W. X. Zhao, Towards unified conversational recommender systems via knowledge-enhanced prompt learning, in Proceedings of the 28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, (2022), 1929–1937. https://doi.org/10.1145/3534678.3539382
[3]	A. Montazeralghaem, J. Allan, Learning relevant questions for conversational product search using deep reinforcement learning, in Proceedings of the Fifteenth ACM International Conference on Web Search and Data Mining, (2022), 746–754. https://doi.org/10.1145/3488560.3498526
[4]	Y. Ma, Y. He, A. Zhang, X. Wang, T. S. Chua, CrossCBR: Cross-view contrastive learning for bundle recommendation, preprint, arXiv: 220600242.
[5]	K. Wu, W. Bian, Z. Chan, L. Ren, S. Xiang, S. Han, et al., Adversarial gradient driven exploration for deep click-through rate prediction, in Proceedings of the 28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, (2022), 2050–2058. https://doi.org/10.1145/3534678.3539461
[6]	Q. Dai, H. Li, P. Wu, Z. Dong, X. Zhou, R. Zhang, et al., A generalized doubly robust learning framework for debiasing post-click conversion rate prediction, in Proceedings of the 28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, (2022), 252–262. https://doi.org/10.1145/3534678.3539270
[7]	T. Wei, J. He, Comprehensive fair meta-learned recommender system, in Proceedings of the 28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, (2022), 1989–1999. https://doi.org/10.1145/3534678.3539269
[8]	E. Gholami, M. Motamedi, A. Aravindakshan, Using session partial actions, preprint, arXiv: 220913015.
[9]	Y. Liu, J. N. Yen, B. Yuan, R. D. Shi, P. Yan, C. J. Lin, Practical counterfactual policy learning for Top-K recommendations, in Proceedings of the 28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, (2022), 1141–1151. https://doi.org/10.1145/3534678.3539295
[10]	J. Chen, W. Fan, G. Zhu, X. Zhao, C. Yuan, Q. Li, et al., Knowledge-enhanced black-box attacks for recommendations, in Proceedings of the 28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, (2022), 108–117. https://doi.org/10.1145/3534678.3539359
[11]	S. Ishikawa, Y. J. Chung, Y. Hirate, Dynamic collaborative filtering Thompson Sampling for cross-domain advertisements recommendation, preprint, arXiv: 220811926.
[12]	S. Oh, A. Bhardwaj, J. Han, S. Kim, R. A. Rossi, S. Kumar, Implicit session contexts for next-item recommendations, in Proceedings of the 31st ACM International Conference on Information & Knowledge Management, (2022), 4364–4368. https://doi.org/10.1145/3511808.3557613
[13]	D. R. Turnbull, S. McQuillan, V. Crabtree, S. Zhang, Exploring popularity bias in music recommendation models and commercial steaming services, preprint, arXiv: 220809517.
[14]	M. Naghiaei, H. A. Rahmani, M. Aliannejadi, N. Sonboli, Towards confidence-aware calibrated recommendation, in Proceedings of the 31st ACM International Conference on Information & Knowledge Management, (2022), 4344–4348. https://doi.org/10.1145/3511808.3557713
[15]	S. Luo, Y. Xiao, Y. Liu, C. Li, L. Song, Towards communication efficient and fair federated personalized sequential recommendation, preprint, arXiv: 220810692.
[16]	Y. Zhang, Z. Chan, S. Xu, W. Bian, S. Han, H. Deng, KEEP: An industrial pre-training framework for online recommendation via knowledge extraction and plugging, in Proceedings of the 31st ACM International Conference on Information & Knowledge Management, 2022, 3684–3693. https://doi.org/10.1145/3511808.3557106
[17]	H. Chen, J. Fu, L. Zhang, S. Wang, K. Lin, L. Shi, Deformable convolutional matrix factorization for document context-aware recommendation in social networks, IEEE Access, 7 (2019), 66347–66357. https://doi.org/10.1109/ACCESS.2019.2917257 doi: 10.1109/ACCESS.2019.2917257
[18]	L. Guo, Y. Han, H. Jiang, X. Yang, X. Wang, X. Liu, Learning to make document context-aware recommendation with joint convolutional matrix factorization, Complexity, 2020 (2020), 1–15. https://doi.org/10.1155/2020/1401236 doi: 10.1155/2020/1401236
[19]	M. Gan, Y. Ma, K. Xiao, CDMF: A deep learning model based on convolutional and dense-layer matrix factorization for context-aware recommendation, 2019 (2019).
[20]	G. Xu, L. He, M. Hu, Document context-aware social recommendation method, in 2019 International Conference on Computing, Networking and Communications (ICNC), (2019), 787–791. https://doi.org/10.1109/ICCNC.2019.8685666
[21]	H. Chen, Z. Li, Z. Wang, Z. Ni, J. Li, G. Xu, et al., Edge data based trailer inception probabilistic matrix factorization for context-aware movie recommendation, World Wide Web, 25 (2021), 1–20. https://doi.org/10.1007/s11280-021-00974-4 doi: 10.1007/s11280-021-00974-4
[22]	Z. Wang, H. Chen, Z. Li, K. Lin, N. Jiang, F. Xia, VRConvMF: Visual recurrent convolutional matrix factorization for movie recommendation, IEEE Trans. Emerg. Topics Comput. Intell., 6 (2021), 519–529. https://doi.org/10.1109/TETCI.2021.3102619 doi: 10.1109/TETCI.2021.3102619
[23]	M. Nguyen, J. Yu, Q. Bai, S. Yongchareon, Y. Han, Attentional matrix factorization with document-context awareness and implicit API relationship for service recommendation, in Proceedings of the Australasian Computer Science Week Multiconference, (2020), 1–10. https://doi.org/10.1145/3373017.3373034
[24]	H. Liu, C. Ling, L. Yang, P. Zhao, Supervised convolutional matrix factorization for document recommendation, Int. J. Comput. Intell. Appl., 17 (2018), 1850018. https://doi.org/10.1142/S1469026818500189 doi: 10.1142/S1469026818500189
[25]	X. Zheng, D. Dong, An adversarial deep hybrid model for text-aware recommendation with convolutional neural networks, Appl. Sci., 10 (2019), 156. https://doi.org/10.3390/app10010156 doi: 10.3390/app10010156
[26]	P. Liu, J. Du, Z. Xue, A. Li, Bi-convolution matrix factorization algorithm based on improved ConvMF, in Intelligent Networked Things: 5th China Conference, CINT 2022, Urumqi, China, August 7–8, 2022, Revised Selected Papers, Springer, (2023), 122–134. https://doi.org/10.1007/978-981-19-8915-5_11
[27]	G. Cai, N. Chen, Constrained probabilistic matrix factorization with neural network for recommendation system, in Intelligent Information Processing IX. ⅡP 2018. IFIP Advances in Information and Communication Technology, Springer, (2018), 236–246. https://doi.org/10.1007/978-3-030-00828-4_24
[28]	Q. Wang, S. Li, G. Chen, Word-driven and context-aware review modeling for recommendation, in Proceedings of the 27th ACM International Conference on Information and Knowledge Management, (2018), 1859–1862. https://doi.org/10.1145/3269206.3269258
[29]	H. Wu, Z. Zhang, K. Yue, B. Zhang, R. Zhu, Content embedding regularized matrix factorization for recommender systems, in 2017 IEEE International Congress on Big Data (BigData Congress), (2017), 209–215. https://doi.org/10.1109/BigDataCongress.2017.36
[30]	Y. Xia, D. Ding, Z. Chang, F. Li, Joint deep networks based multi-source feature learning for QoS prediction, IEEE Trans. Serv. Comput., 15 (2021), 2314–2327. https://doi.org/10.1109/TSC.2021.3050178 doi: 10.1109/TSC.2021.3050178
[31]	J. Zhao, Z. Liu, H. Chen, J. Zhang, Q. Wen, Hybrid recommendation algorithms based on ConvMF deep learning model, in 2019 International Conference on Wireless Communication, Network and Multimedia Engineering (WCNME 2019), Atlantis Press, (2019), 151–154. https://doi.org/10.2991/wcnme-19.2019.36
[32]	J. Pennington, R. Socher, C. D. Manning, Glove: Global vectors for word representation, in Proceedings of the 2014 conference on empirical methods in natural language processing (EMNLP), (2014), 1532–1543.
[33]	T. Mikolov, K. Chen, G. Corrado, J. Dean, Efficient estimation of word representations in vector space, preprint, arXiv: 1301.3781.
[34]	D. Kim, C. Park, J. Oh, S. Lee, H. Yu, Convolutional matrix factorization for document context-aware recommendation, in Proceedings of the 10th ACM Conference on Recommender Systems, (2016), 233–240. https://doi.org/10.1145/2959100.2959165
[35]	X. He, L. Liao, H. Zhang, L. Nie, X. Hu, T. S. Chua, Neural collaborative filtering, in Proceedings of the 26th International Conference on World Wide Web, (2017), 173–182. https://doi.org/10.1145/3038912.3052569
[36]	S. Sedhain, A. K. Menon, S. Sanner, L. Xie, Autorec: Autoencoders meet collaborative filtering, in Proceedings of the 24th International Conference on World Wide Web, (2015), 111–112. https://doi.org/10.1145/2740908.2742726
[37]	F. Strub, R. Gaudel, J. Mary, Hybrid recommender system based on autoencoders, in Proceedings of the 1st workshop on deep learning for recommender systems, (2016), 11–16. https://doi.org/10.1145/2988450.2988456
[38]	X. Sun, H. Zhang, M. Wang, M. Yu, M. Yin, B. Zhang, Deep plot-aware generalized matrix factorization for collaborative filtering, Neural Process. Lett., 52 (2020), 1983–1995. https://doi.org/10.1007/s11063-020-10333-5 doi: 10.1007/s11063-020-10333-5

Reader Comments

Your name:*

Email:*
© 2023 the Author(s), licensee AIMS Press. This is an open access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/4.0)