|
|
|
| Knowledge representation learning method integrating textual description and hierarchical type |
Song LI1( ),Shi-tai SHU1,Xiao-hong HAO1,Zhong-xiao HAO1,2 |
1. School of Computer Science and Technology, Harbin University of Science and Technology, Harbin 150080, China
2. School of Computer Science and Technology, Harbin Institute of Technology, Harbin 150001, China |
|
|
|
Abstract Existing knowledge representation methods only consider triplet itself or one kind of additional information, and do not make use of external information to semantic supplement knowledge representation. The convolutional neural network was used to extract feature information from text. The convolutional neural network based on attention mechanism was used to distinguish the feature reliability of different relationships, enhance the representation of entity relationship structure vector in the existing knowledge graph and obtain rich semantic information. A weighted hierarchical encoder which combined all the hierarchical type projection matrix of the entity with the relationship-specific type constraints, was used to construct the projection matrix of the hierarchical type, The link prediction and the triplet classification were performed on WN18, WN18RR, FB15K, FB15K-237 and YAGO3-10 datasets to analyze and verify the validity of the proposed model. The experiment showed that in the entity prediction experiment, the proposed model reduced the MeanRank(Filter) by 11.8% compared to the TransD model, and increased Hits@10 by 3.5%. In the triple classification experiment, the classification accuracy of the proposed model was 8.4% higher than the DKRL model and 8.5% higher than the TKRL model, which fully proved that the ability of knowledge representation could be improved by using external multi-source information.
|
|
Received: 08 May 2022
Published: 09 May 2023
|
|
|
| Fund: 国家自然科学基金资助项目(61872105, 62072136)；黑龙江省自然科学基金资助项目(LH2020F047)；黑龙江省留学归国人员科学基金资助项目(LC2018030)；河南省科技攻关项目(232102210068)；国家重点研发计划资助项目(2020YFB1710200) |
融合文本描述和层次类型的知识表示学习方法
现有的知识表示方法只考虑三元组本身或一种额外信息，没有充分利用外部信息对知识表示进行语义补充， 为此提出一种融合文本描述信息和层次类型信息的知识表示学习方法.使用卷积神经网络(CNN)从文本中提取特征信息；使用基于注意力机制的卷积神经网络区分不同关系的特征可信度，以增强实体关系结构向量在现有知识图谱中的表示，获得丰富的语义信息；使用加权层次编码器来构造层次类型投影矩阵，将实体的所有层次类型投影矩阵与特定关系类型约束结合起来.在WN18、WN18RR、FB15K、FB15K-237和YAGO3-10数据集上,进行链接预测和三元组分类等任务，以分析和验证所提模型的有效性. 实验结果表明: 在实体预测实验中，所提模型与TransD模型相比，MeanRank(Filter)降低了11.8%，Hits@10提升了3.5%；在三元组分类实验中，所提模型的分类精度比DKRL模型提高了8.4%，比TKRL模型提升了8.5%，充分证明利用外部多源信息能够提高知识表示能力.
关键词：
知识图谱,
知识表示,
多源信息融合,
表示学习,
链接预测
|
|
| [1] |
CHEN X J, JIA S B, XIANG Y A review: knowledge reasoning over knowledge graph[J]. Expert Systems with Applications, 2020, 141: 1- 21
|
|
|
| [2] |
舒世泰, 李松, 郝晓红, 等 知识图谱嵌入技术研究进展[J]. 计算机科学与探索, 2021, 15 (11): 2048- 2062
SHU Shi-tai, Li Song, HAO Xiao-hong, et al Knowledge graph embedding technology: a review[J]. Journal of Frontiers of Computer Science and Technology, 2021, 15 (11): 2048- 2062
|
|
|
| [3] |
刘知远, 孙茂松, 林衍凯, 等 知识表示学习研究进展[J]. 计算机研究与发展, 2016, 53 (2): 247- 261
LIU Zhi-yuan, SUN Mao-song, LIN Yan-kai, et al Research progress of knowledge representation learning[J]. Journal of Computer Research and Development, 2016, 53 (2): 247- 261
|
|
|
| [4] |
BORDES A, USUNIER N, GARCÍA-DURÁN A, et al. Translating embeddings for modeling multi-relational data [C]// Proceedings of the 27th International Conference on Neural Information Processing Systems. Lake Tahoe: MITP, 2013: 2787-2795.
|
|
|
| [5] |
WANG Z, ZHANG J W, FENG J L, et al. Knowledge graph embedding by translating on hyperplanes [C]// Proceedings of the 28th AAAI Conference on Artificial Intelligence. Québec: AAAI, 2014: 1112-1119.
|
|
|
| [6] |
LIN Y K, LIU Z Y, SUN M S, et al. Learning entity and relation embeddings for knowledge graph completion [C]// Proceedings of the 29th AAAI Conference on Artificial Intelligence. Austin: AAAI, 2015: 2181-2187.
|
|
|
| [7] |
JI G L, HE S Z, XU L H, et al. Knowledge graph embedding via dynamic mapping matrix [C]// Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and 7th International Joint Conference on Natural Language Processing. Beijing: ACL, 2015: 687-696.
|
|
|
| [8] |
TransA: an adaptive approach for knowledge graph embedding [EB/OL].[2022-05-02]. http://arxiv.org/abs/1509.05490.
|
|
|
| [9] |
XIAO H, HUANG M L, ZHU X Y. TransG: a generative model for knowledge graph embedding [C]// Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics. Berlin: ACL, 2016: 2316-2325.
|
|
|
| [10] |
WANG Z, ZHANG J W, FENG J L, et al. Knowledge graph and text jointly embedding[C]// Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing. Doha: ACL, 2014: 1591-1601.
|
|
|
| [11] |
XIE R B, LIU Z Y, JIA J, et al. Representation learning of knowledge graphs with entity descriptions [C]// Proceedings of the 30th National Conference on Artificial Intelligence. Phoenix: AAAI, 2015: 2659-2665.
|
|
|
| [12] |
XIE R B, LIU Z Y, SUN M S. Representation learning of knowledge graphs with hierarchical types [C]// Proceedings of the 25th International Joint Conference on Artificial Intelligence. New York: AAAI, 2016: 2965-2971.
|
|
|
| [13] |
JI S X, PAN S R, ERIK C, et al A survey on knowledge graphs: representation, acquisition and applications[J]. IEEE Transactions on Neural Networks and Learning Systems, 2021, 33 (2): 494- 514
|
|
|
| [14] |
ZENG D J, LIU K, CHEN Y B, et al. Distant supervision for relation extraction via piecewise convolutional neural networks [C]// Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing. Lisbon: ACL, 2014: 1753-1762.
|
|
|
| [15] |
JIANG X T, WANG Q, LI P, et al. Relation extraction with multi-instance multi-label convolutional neural networks [C]// Proceedings of COLING 2016, the 26th International Conference on Computational Linguistics. Osaka: COC, 2016: 1471-1480.
|
|
|
| [16] |
HAN X, YU P F, LIU Z Y, et al. Hierarchical relationextraction with coarse-to-fine grained attention [C]// Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing. Brussels: ACL, 2018: 2236-2245.
|
|
|
| [17] |
ZHANG N Y, DENG S M, SUN Z L, et al. Long-tail relation extraction via knowledge graph embeddings and graph convolution networks [C]// Proceedings of the 2019 Conference of the NAACL: Human Language Technologies. Minneapolis: ACL, 2019: 3016-3025.
|
|
|
| [18] |
TANG X, CHEN L, CUI J, et al Knowledge representation learning with entity descriptions, hierarchical types, and textual relations[J]. Information Processing and Management, 2019, 56: 809- 822
doi: 10.1016/j.ipm.2019.01.005
|
|
|
| [19] |
FAN M, ZHOU Q, CHANG E, et al. Transition-based knowledge graph embedding with relational mapping properties [C]// Proceedings of the 28th Pacific Asia Conference on Language, Information and Computation. Phuket: ACL, 2014: 328-337.
|
|
|
| [20] |
FENG J, HUANG M L, WANG M D, et al. Knowledge graph embedding by flexible translation [C]// Proceedings of the 15th International Conference on Principles of Knowledge Representation and Reasoning. Cape Town: AAAI, 2016: 557-560.
|
|
|
| [21] |
NICKEL M, TRESP V, KRIEGEL H. A three-way model for collective learning on multi-relational data [C]// Proceedings of the 28th International Conference on Machine Learning. Washington: ACM, 2011: 809-816.
|
|
|
| [22] |
YANG B S, YIH W, HE X D, et al. Embedding entities and relations for learning and inference in knowledge bases [C]// Proceedings of the 3rd International Conference on Learning Representations. San Diego: [s.n.], 2015: 1-12.
|
|
|
| [23] |
TROUILLON T, WELBL J, RIEDEL S, et al. Complex embeddings for simple link prediction [C]// Proceedings of the 33rd International Conference on Machine Learning. New York: IMLS, 2016: 2071-2080.
|
|
|
| [24] |
ZHANG Z, ZHUANG F Z, QU M, et al. Knowledge graph embedding with hierarchical relation structure [C]// Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing. Brussels: ACL, 2018: 3198-3207.
|
|
|
| [25] |
FENG J, HUANG M L, YANG Y, et al. GAKE: Graph aware knowledge embedding [C]// Proceedings of COLING 2016, the 26th International Conference on Computational Linguistics. Osaka: COC, 2016: 641-651.
|
|
|
| [26] |
XIE R B, LIU Z Y, LUAN H B, et al. Image embodied knowledge representation learning [C]// Proceedings of the 26th International Joint Conference on Artificial Intelligence. Melbourne: AAAI, 2017: 3140-3146.
|
|
|
| [27] |
TOUTANOVA K, LIN X V, YIH W T, et al. Compositional learning of embeddings for relation paths in knowledge base and text [C]// Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics. Berlin: ACL, 2016: 1434-1444.
|
|
|
| [28] |
夏光兵, 李瑞轩, 辜希武, 等 融合多源信息的知识表示学习[J]. 计算机科学与探索, 2022, 16 (3): 591- 597
XIA Guang-bing, LI Rui-xuan, GU Xi-wu, et al Knowledge representation learning based on multi-source information combination[J]. Journal of Frontiers of Computer Science and Technology, 2022, 16 (3): 591- 597
|
|
|
| [29] |
杜文倩, 李弼程, 王瑞 融合实体描述及类型的知识图谱表示学习方法[J]. 中文信息学报, 2020, 34 (7): 50- 59
DU Wen-qing, LI Bi-cheng, WANG Rui Representation learning of knowledge graph integrating entity description and entity type[J]. Journal of Chinese Information Processing, 2020, 34 (7): 50- 59
|
|
|
| [30] |
WANG P, ZHOU J JECI++: a modified joint knowledge graph embedding model for concepts and instances[J]. Big Data Research, 2021, 24: 1- 10
|
|
|
| [31] |
ZHAO F, XU T, JIN L J Q, et al Convolutional network embedding of text-enhanced representation for knowledge graph completion[J]. IEEE Internet of Things Journal, 2021, 8 (23): 16758- 16769
doi: 10.1109/JIOT.2020.3039750
|
|
|
| [32] |
MAHDISOLTANI F, BIEGA J, SUCHANEK F. Yago3: A knowledge base from multilingual wikipedias [C]// Proceedings of the 7th Biennial Conference on Innovative Data Systems Research. Asilomar: [s. n.], 2015: 1-11.
|
|
|
|
Viewed |
|
|
|
Full text
|
|
|
|
|
Abstract
|
|
|
|
|
Cited |
|
|
|
|
| |
Shared |
|
|
|
|
| |
Discussed |
|
|
|
|
