FRS: A simple knowledge graph embedding model for entity prediction

Lifang Wang; Xinyu Lu; Zejun Jiang; Zhikai Zhang; Ronghan Li; Meng Zhao; Daqing Chen; Lifang Wang; Xinyu Lu; Zejun Jiang; Zhikai Zhang; Ronghan Li; Meng Zhao; Daqing Chen

doi:10.3934/mbe.2019391

Mathematical Biosciences and Engineering

2019, Volume 16, Issue 6: 7789-7807. doi: 10.3934/mbe.2019391

Previous Article Next Article

Research article Special Issues

FRS: A simple knowledge graph embedding model for entity prediction

1.
School of Computer Science and Engineering, Northwestern Polytechnical University, Xi’an, 710072, PR China
2.
Division of Computer Science and Informatics, School of Engineering, London South Bank University, London SE1 0AA, UK

Received: 22 May 2019 Accepted: 30 July 2019 Published: 26 August 2019

Entity prediction is the task of predicting a missing entity that has a specific relation-ship with another given entity. Researchers usually use knowledge graphs embedding(KGE) methods to embed triples into continuous vectors for computation and perform the tasks of entity prediction. However, KGE models tend to use simple operations to refactor entities and relationships, resulting in insufficient interaction of components of knowledge graphs (KGs), thus limiting the performance of the entity prediction model. In this paper, we propose a new entity prediction model called FRS(Feature Refactoring Scoring) to alleviate the problem of insufficient interaction and solve information incom-pleteness problems in the KGs. Different from the traditional KGE methods of directly using simple operations, the FRS model innovatively provides the procedure of feature processing in the entity prediction tasks, realizing the alignment of entities and relationships in the same feature space and improving the performance of entity prediction model. Although FRS is a simple three-layer network, we find that our own model outperforms state-of-the-art KGC methods in FB15K and WN18. Through extensive experiments on FRS, we discover several insights. For example, the effect of embedding size and negative candidate sampling probability on experimental results is in reverse.
- knowledge graphs,
- entity prediction,
- knowledge graphs embedding,
- FRS
Citation: Lifang Wang, Xinyu Lu, Zejun Jiang, Zhikai Zhang, Ronghan Li, Meng Zhao, Daqing Chen. FRS: A simple knowledge graph embedding model for entity prediction[J]. Mathematical Biosciences and Engineering, 2019, 16(6): 7789-7807. doi: 10.3934/mbe.2019391

Related Papers:

Abstract

Entity prediction is the task of predicting a missing entity that has a specific relation-ship with another given entity. Researchers usually use knowledge graphs embedding(KGE) methods to embed triples into continuous vectors for computation and perform the tasks of entity prediction. However, KGE models tend to use simple operations to refactor entities and relationships, resulting in insufficient interaction of components of knowledge graphs (KGs), thus limiting the performance of the entity prediction model. In this paper, we propose a new entity prediction model called FRS(Feature Refactoring Scoring) to alleviate the problem of insufficient interaction and solve information incom-pleteness problems in the KGs. Different from the traditional KGE methods of directly using simple operations, the FRS model innovatively provides the procedure of feature processing in the entity prediction tasks, realizing the alignment of entities and relationships in the same feature space and improving the performance of entity prediction model. Although FRS is a simple three-layer network, we find that our own model outperforms state-of-the-art KGC methods in FB15K and WN18. Through extensive experiments on FRS, we discover several insights. For example, the effect of embedding size and negative candidate sampling probability on experimental results is in reverse.

References

[1]	Knowledge graph, Available from: https://en.wikipedia.org/wiki/Knowledge_Graph.
[2]	M. H. Gad-Elrab, D. Stepanova, J. Urbani, et al., Exfakt: A framework for explaining facts over knowledge graphs and text, in Proceedings of the Twelfth ACM International Conference on Web Search and Data Mining, ACM, (2019), 87–95.
[3]	C. W. Lee, W. Fang, C. K. Yeh, et al., Multi-label zero-shot learning with structured knowledge graphs, in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, (2018), 1576–1585.
[4]	E. B. Hadley, D. K. Dickinson, K. Hirsh-Pasek, et al., Building semantic networks: The impact of a vocabulary intervention on preschoolers depth of word knowledge, Read. Res. Quart., 54 (2019), 41–61.
[5]	K. Bollacker, C. Evans, P. Paritosh, et al., Freebase: a collaboratively created graph database for structuring human knowledge, in Proceedings of the 2008 ACM SIGMOD international confer- ence on Management of data, ACM, (2008), 1247–1250.
[6]	Wikidata, Available from: https://www.wikidata.org/wiki/Wikidata:Main_Page.
[7]	C. Bizer, J. Lehmann, G. Kobilarov, et al., Dbpedia-a crystallization point for the web of data, Web Semantics: Science, Services and Agents on The World Wide Web, 7 (2009), 154–165.
[8]	T. Young, D. Hazarika, S. Poria, et al., Recent trends in deep learning based natural language processing, IEEE Comput. Intell. M., 13 (2018), 55–75.
[9]	B. Shi and T. Weninger, Open-world knowledge graph completion, in Thirty-Second AAAI Con-ference on Artificial Intelligence, 2018.
[10]	C. Meilicke, M. Fink, Y. Wang, et al., Fine-grained evaluation of rule-and embedding-based systems for knowledge graph completion, in International Semantic Web Conference, Springer, (2018), 3–20.
[11]	D. Q. Nguyen, An overview of embedding models of entities and relationships for knowledge base completion, arXiv preprint arXiv:1703.08098.
[12]	H. Wang, F. Zhang, M. Hou, et al., Shine: Signed heterogeneous information network embedding for sentiment link prediction, in Proceedings of the Eleventh ACM International Conference on Web Search and Data Mining, ACM, (2018), 592–600.
[13]	R. Xie, Z. Liu, J. Jia, et al., Representation learning of knowledge graphs with entity descriptions, in Thirtieth AAAI Conference on Artificial Intelligence, 2016.
[14]	Q. Wang, Z. Mao, B. Wang, et al., Knowledge graph embedding: A survey of approaches and applications, IEEE T. Knowl. Data En., 29 (2017), 2724–2743.
[15]	T. Zhen, Z. Xiang, F. Yang, et al., Knowledge graph representation via similarity-based embed- ding, Sci. Programming, 2018 (2018), 1–12.
[16]	B. Shi and T. Weninger, Proje: Embedding projection for knowledge graph completion, in Thirty-First AAAI Conference on Artificial Intelligence, 2017.
[17]	A. Bordes, X. Glorot, J. Weston, et al., A semantic matching energy function for learning with multi-relational data, Mach. Learn., 94 (2014), 233–259.
[18]	A. Bordes, J. Weston, R. Collobert, et al., Learning structured embeddings of knowledge bases, in Twenty-Fifth AAAI Conference on Artificial Intelligence, 2011.
[19]	T. Mikolov, I. Sutskever, K. Chen, et al., Distributed representations of words and phrases and their compositionality, in Advances in neural information processing systems, (2013), 3111–3119.
[20]	A. Bordes, N. Usunier, A. Garcia-Duran, et al., Translating embeddings for modeling multi-relational data, in Advances in neural information processing systems, (2013), 2787–2795.
[21]	Z. Wang, J. Zhang, J. Feng, et al., Knowledge graph embedding by translating on hyperplanes, in Twenty-Eighth AAAI conference on artificial intelligence, 2014.
[22]	M. Nickel, V. Tresp and H. P. Kriegel, A three-way model for collective learning on multi-relational data., in ICML, 11 (2011), 809–816.
[23]	R. Socher, D. Chen, C. D. Manning, et al., Reasoning with neural tensor networks for knowledge base completion, in Advances in neural information processing systems, (2013), 926–934.
[24]	S. Guan, X. Jin, Y. Wang, et al., Shared embedding based neural networks for knowledge graph completion, in Proceedings of the 27th ACM International Conference on Information and Knowl-edge Management, ACM, (2018), 247–256.
[25]	Y. Lin, X. Han, R. Xie, et al., Knowledge representation learning: A quantitative review.
[26]	K. Xu, C. Li, Y. Tian, et al., Representation learning on graphs with jumping knowledge networks, arXiv preprint arXiv:1806.03536.
[27]	D. H. Pham and A. C. Le, Learning multiple layers of knowledge representation for aspect based sentiment analysis, Data Knowl. Eng., 114 (2018), 26–39.
[28]	K. He, X. Zhang, S. Ren, et al., Identity mappings in deep residual networks, in European confer-ence on computer vision, Springer, (2016), 630–645.
[29]	M. Nickel, L. Rosasco and T. Poggio, Holographic embeddings of knowledge graphs, in Thirtieth Aaai conference on artificial intelligence, 2016.
[30]	Y. Lin, Z. Liu, M. Sun, et al., Learning entity and relation embeddings for knowledge graph completion, in Twenty-ninth AAAI conference on artificial intelligence, 2015.
[31]	Z. Wang and J. Z. Li, Text-enhanced representation learning for knowledge graph., in IJCAI, (2016), 1293–1299.
[32]	S. He, K. Liu, G. Ji, et al., Learning to represent knowledge graphs with gaussian embedding, in Proceedings of the 24th ACM International on Conference on Information and Knowledge Management, ACM, (2015), 623–632.
[33]	G. Ji, S. He, L. Xu, et al., Knowledge graph embedding via dynamic mapping matrix, in Pro-ceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing (Volume 1: Long Papers), 1 (2015), 687–696.
[34]	H. G. Yoon, H. J. Song, S. B. Park, et al., A translation-based knowledge graph embedding pre-serving logical property of relations, in Proceedings of the 2016 Conference of the North Amer-ican Chapter of the Association for Computational Linguistics: Human Language Technologies, (2016), 907–916.
[35]	H. Xiao, M. Huang, L. Meng, et al., Ssp: semantic space projection for knowledge graph embed-ding with text descriptions, in Thirty-First AAAI Conference on Artificial Intelligence, 2017.
[36]	G. Ji, K. Liu, S. He, et al., Knowledge graph completion with adaptive sparse transfer matrix, in Thirtieth AAAI Conference on Artificial Intelligence, 2016.
[37]	H. Xiao, M. Huang and X. Zhu, Transg: A generative model for knowledge graph embedding, in Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Vol-ume 1: Long Papers), 1 (2016), 2316–2325.
[38]	L. Chang, M. Zhu, T. Gu, et al., Knowledge graph embedding by dynamic translation, IEEE Access, 5 (2017), 20898–20907.
[39]	Y. Lin, Z. Liu, H. Luan, et al., Modeling relation paths for representation learning of knowledge bases, arXiv preprint arXiv:1506.00379.
[40]	T. Trouillon, J. Welbl, S. Riedel, et al., Complex embeddings for simple link prediction, in Inter-national Conference on Machine Learning, (2016), 2071–2080.

Reader Comments

Your name:*

Email:*
© 2019 the Author(s), licensee AIMS Press. This is an open access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/4.0)