Dynamic knowledge graph completion of temporal aware combination

doi:10.3785/j.issn.1008-973X.2024.08.020

Journal of ZheJiang University (Engineering Science)

2024, Vol. 58

Issue (8): 1738-1747 DOI: 10.3785/j.issn.1008-973X.2024.08.020

Dynamic knowledge graph completion of temporal aware combination

Zhongliang LI1,2(

),Qi CHEN1,2,Lin SHI1,2,*(

),Chao YANG1,2,Xianming ZOU1,2

1. College of Artificial Intelligence, North China University of Science and Technology, Tangshan 063210, China
2. Hebei Key Laboratory of Industrial Intelligent Perception, Tangshan 063210, China

Download:

HTML

PDF(970KB) HTML
Export: BibTeX | EndNote (RIS)

Abstract

A time-aware combination (TAC) method for temporal knowledge graph completion was proposed aiming at the problem that the existing temporal knowledge graph embedding methods only consider the relationship of temporal information or encode independent temporal vectors and the completion performance of these methods is not high enough. The effectiveness of temporal information on knowledge graph completion methods was analyzed by modeling dimensional features. Different learning methods have different effects on the representation learning ability after considering the embedding of temporal information through the embedding method of combining the embedded and independent temporal information. Long short-term memory (LSTM) network was utilized to encode temporal information, learn more accurate temporal dimension features and help to improve the performance of temporal graph. Experiments on ICEWS14, ICEWS05-15 and GDELT datasets verified the effectiveness of the time-aware combination method. The related research performance metrics were compared. Results show that the proposed method performs better in link prediction.

Key words： temporal knowledge graph attention mechanism long short-term memory (LSTM) temporal embedding

Received: 10 July 2023 Published: 23 July 2024

CLC:

TP 391

Corresponding Authors: Lin SHI E-mail: bbzqlzl@163.com;shilin@ncst.edu.cn

	Service
	E-mail this article
	Add to my bookshelf
	Add to citation manager
	E-mail Alert
	RSS
	Articles by authors
	Zhongliang LI
	Qi CHEN
	Lin SHI
	Chao YANG
	Xianming ZOU

Cite this article:

Zhongliang LI,Qi CHEN,Lin SHI,Chao YANG,Xianming ZOU. Dynamic knowledge graph completion of temporal aware combination. Journal of ZheJiang University (Engineering Science), 2024, 58(8): 1738-1747.

URL:

https://www.zjujournals.com/eng/10.3785/j.issn.1008-973X.2024.08.020 OR https://www.zjujournals.com/eng/Y2024/V58/I8/1738

时间感知组合的动态知识图谱补全

针对现有时序知识图谱嵌入方法仅考虑时序信息的关系或仅编码独立的时序向量，知识图谱补全性能不高的问题，提出时间感知组合（TAC）的时序知识图谱补全方法. 通过建模维度特征，分析时序信息对知识图谱补全方法的有效程度. 通过时序信息内嵌和独立相结合的嵌入方式，考虑时序信息嵌入后，不同学习方式对表示学习能力产生不同的影响. 提出的方法利用长短时记忆（LSTM）网络编码时序信息，学习到更准确的时间维度特征，有助于提升时序图谱的性能. 在ICEWS14、ICEWS05-15和GDELT数据集上进行实验，验证了时间感知组合方法的有效性. 对比相关的研究性能指标可知，本文方法在链接预测上表现较优.

关键词： 时序知识图谱, 注意力机制, 长短时记忆(LSTM), 时序嵌入

补全方法	适用知识库	建模维度	时间特征	时间维度融合	相似性评价函数
TransE	三元组	s,r,o	×	×	$ f\left( {s,r,o} \right) = \left\\| {{{\boldsymbol{e}}_s}+{{\boldsymbol{e}}_r} - {{\boldsymbol{e}}_o}} \right\\| $
DistMult	三元组	s,r,o	×	×	$ f\left( {s,r,o} \right) = \left\langle {{{\boldsymbol{e}}_s},{{\boldsymbol{e}}_r},{{\boldsymbol{e}}_o}} \right\rangle $
ComplEx	三元组	s,r,o	×	×	$ f\left( {s,r,o} \right) = {{\mathrm{Re}}} \left( {\langle {{\boldsymbol{e}}_s},{{\boldsymbol{w}}_r},{{\boldsymbol{e}}_o}\rangle } \right) $
ConvE	三元组	s,r,o	×	×	$ f\left( {s,r,o} \right) = f({\mathrm{vec}}(f([{{\boldsymbol{\bar e}}_s};{{\boldsymbol{\bar r}}_r}] * {\boldsymbol{\varOmega}})){{\boldsymbol{W}}}){{\boldsymbol{e}}_o} $
ConvKB	三元组	s,r,o	×	×	$ f\left( {s,r,o} \right) = {\text{concat}}(f([{{\boldsymbol{e}}_s},{{\boldsymbol{e}}_r},{{\boldsymbol{e}}_o}] * {\boldsymbol{\varOmega}} )) \cdot {\boldsymbol{w}} $
HyTE	四元组	s,r,o	√	√	$ f\left( {s,r,o,t} \right) = \left\\| {{P_t}({{\boldsymbol{e}}_s})+{P_t}({{\boldsymbol{e}}_r}) - {P_t}({{\boldsymbol{e}}_o})} \right\\| $
TA-TransE	四元组	s,r,o	√	√	$ f\left( {s,r,o,t} \right) = \left\\| {{{\boldsymbol{e}}_s}+{{\boldsymbol{e}}_{{r_{{\mathrm{seq}}}}}} - {{\boldsymbol{e}}_o}} \right\\| $
TA-DistMult	四元组	s,r,o	√	√	$ f\left( {s,r,o,t} \right) = ({{\boldsymbol{e}}_s} \circ {{\boldsymbol{e}}_o}){{\boldsymbol{e}}_{{r_{{\mathrm{seq}}}}}}^{\text{T}} $
ST-ConvKB	四元组	s,r,o	√	√	$ f\left( {s,r,o,t} \right) = {\text{concat}}(f([{{\boldsymbol{e}}_{{s_t}}},{{\boldsymbol{e}}_r},{{\boldsymbol{e}}_{{o_t}}}] * {\boldsymbol{\varOmega}} )) \cdot {\boldsymbol{w}} $
TTransE	四元组	s,r,o,t	√	×	$ f\left( {s,r,o,t} \right) = \left\\| {{{\boldsymbol{e}}_s}+{{\boldsymbol{e}}_r}+{{\boldsymbol{e}}_t} - {{\boldsymbol{e}}_o}} \right\\| $
TComplEx	四元组	s,r,o,t	√	×	$ f\left( {s,r,o,t} \right) = {{\mathrm{Re}}} (\langle {{\boldsymbol{e}}_s},{{\boldsymbol{w}}_r},{{\boldsymbol{e}}_o},{{\boldsymbol{w}}_t}\rangle ) $
RE-GCN^[27]	四元组	s,r,o	√	√	$ \vec p(o\|s,r,{{\boldsymbol{H}}_t},{R_t}) = \sigma ({{\boldsymbol{H}}_t}{\text{ConvTransE}}({{\boldsymbol{e}}_{{s_t}}},{{\boldsymbol{e}}_{{r_t}}})) $
ATiSE	四元组	s,r,o	√	√	$ f\left( {s,r,o,t} \right) = {D_{{\mathrm{KL}}}}({{\boldsymbol{P}}_{r,t}},{{\boldsymbol{P}}_{e,t}}) $
TeRo	四元组	s,r,o	√	√	$ f\left( {s,r,o,t} \right) = \|\|{{\boldsymbol{e}}_{{s_t}}}+{{\boldsymbol{e}}_r} - \overline {{{\boldsymbol{e}}_{ot}}} \|\| $

Tab.1 Comparison of approaches for knowledge graph completion

Tab.2 Time series index marker

Fig.1 Embedding method of temporal aware composition

Tab.3 Information statistics of dataset

Tab.4 Link prediction results of different embedding methods in ICEWS14 and ICEWS05-15 %

Tab.5 Comparison of experimental results of different embedding methods on ICEWS14 and ICEWS11-14 datasets %

Tab.6 Comparison of experimental results of different temporal embedding methods on ICEWS14 dataset %

Fig.2 Ablation contrast of embedded dimensions


[1]	LI Z, GUAN S, JIN X, et al. Complex evolutional pattern learning for temporal knowledge graph reasoning [C]// Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers) . Dublin: ACL, 2022: 290-296.

[2]	CHEN Z, ZHAO X, LIAO J, et al Temporal knowledge graph question answering via subgraph reasoning[J]. Knowledge-Based Systems, 2022, 251: 109134 doi: 10.1016/j.knosys.2022.109134

[3]	LONG J, CHEN Z, HE W, et al An integrated framework of deep learning and knowledge graph for prediction of stock price trend: an application in Chinese stock exchange market[J]. Applied Soft Computing, 2020, 91: 106205 doi: 10.1016/j.asoc.2020.106205

[4]	GARCIA-DURAN A, DUMANČIĆ S, NIEPERT M. Learning sequence encoders for temporal knowledge graph completion [C]// Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing . Brussel: ACL, 2018: 4816-4821.

[5]	LEBLAY J, CHEKOL M W. Deriving validity time in knowledge graph [C]// Companion Proceedings of the Web Conference . Lyon: ACM, 2018: 1771-1776.

[6]	LEETARU K, SCHRODT P A. Gdelt: global data on events, location, and tone, 1979–2012 [C]// ISA Annual Convention . San Francisco: Citeseer, 2013: 1-49.

[7]	KIM H A, D’ORAZIO V, BRANDT P T, et al UTDEventData: an r package to access political event data[J]. Journal of Open Source Software, 2019, 4 (36): 1322 doi: 10.21105/joss.01322

[8]	WANG Q, MAO Z, WANG B, et al Knowledge graph embedding: a survey of approaches and applications[J]. IEEE Transactions on Knowledge and Data Engineering, 2017, 29 (12): 2724- 2743 doi: 10.1109/TKDE.2017.2754499

[9]	BORDES A, USUNIER N, GARCIA-DURAN A, et al Translating embeddings for modeling multi-relational data[J]. Advances in Neural Information Processing Systems, 2013, 26: 2787- 2795

[10]	WANG Z, ZHANG J, FENG J, et al. Knowledge graph embedding by translating on hyperplanes [C]// Proceedings of the AAAI Conference on Artificial Intelligence . Québec: AAAI, 2014: 752-786.

[11]	LIN Y, LIU Z, SUN M, et al. Learning entity and relation embeddings for knowledge graph completion [C]// Proceedings of the AAAI conference on Artificial Intelligence . Texas: AAAI, 2015: 762-816.

[12]	TROUILLON T, DANCE C R, GAUSSIER É, et al Knowledge graph completion via complex tensor factorization[J]. The Journal of Machine Learning Research, 2017, 18 (1): 4735- 4772

[13]	NICKEL M, TRESP V, KRIEGEL H P. A three-way model for collective learning on multi-relational data [C]// Proceedings of the 28th International Conference on International Conference on Machine Learning . [S. l. ]: IEEE, 2011: 809-816.

[14]	YANG B, YIH S W, HE X, et al. Embedding entities and relations for learning and inference in knowledge bases [C]// Proceedings of the International Conference on Learning Representations . San Diego: IEEE, 2015.

[15]	TROUILLON T, DANCE C R, GAUSSIER É, et al Knowledge graph completion via complex tensor factorization[J]. Journal of Machine Learning Research, 2017, 18 (130): 1- 38

[16]	DETTMERS T, MINERVINI P, STENETORP P, et al. Convolutional 2d knowledge graph embeddings [C]// Proceedings of the AAAI Conference on Artificial Intelligence . New Orleans: AAAI, 2018: 612-618.

[17]	NGUYEN T D, NGUYEN D Q, PHUNG D, et al. A novel embedding model for knowledge base completion based on convolutional neural network [C]// Proceedings of NAACL-HLT . New Orleans: ACL, 2018: 327-333.

[18]	SCHLICHTKRULL M, KIPF T N, BLOEM P, et al. Modeling relational data with graph convolutional networks [C]// The Semantic Web: 15th International Conference, ESWC 2018. Heraklion: Springer, 2018: 593-607.

[19]	VASHISHTH S, SANYAL S, NITIN V, et al. Composition-based multi-relational graph convolutional networks [C]// International Conference on Learning Representations . New Orleans: IEEE, 2019.

[20]	JIANG T, LIU T, GE T, et al. Encoding temporal information for time-aware link prediction [C]// Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing . Texas: ACL, 2016: 2350-2354.

[21]	GARCIA-DURAN A, DUMANČIĆ S, NIEPERT M. Learning sequence encoders for temporal knowledge graph completion [C]// Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing . Brussels: ACL, 2018: 4816-4821.

[22]	DASGUPTA S S, RAY S N, TALUKDAR P P. HyTE: hyperplane-based temporally aware knowledge graph embedding [C]// Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing . Brussels: ACL, 2018: 2001-2011.

[23]	XU C, NAYYERI M, ALKHOURY F, et al. Temporal knowledge graph embedding model based on additive time series decomposition [C]// International Semantic Web Conference . [S. l. ]: IEEE, 2020.

[24]	LEBLAY J, CHEKOL M W. Deriving validity time in knowledge graph [C]// Companion Proceedings of the Web Conference . Lyon: WWW, 2018: 1771-1776.

[25]	LACROIX T, OBOZINSKI G, USUNIER N. Tensor decompositions for temporal knowledge base completion [C]// International Conference on Learning Representations . New Orleans: IEEE, 2019.

[26]	李凤英, 范伟豪基于时序感知LR的动态知识图谱补全方法[J]. 计算机工程与应用, 2022, 58 (15): 202- 209 LI Fengying, FAN Weihao Temporal aware approach for dynamic knowledge graph completion[J]. Computer Engineering and Applications, 2022, 58 (15): 202- 209 doi: 10.3778/j.issn.1002-8331.2112-0057

[27]	GAO Y, FENG L, KAN Z, et al. Modeling precursors for temporal knowledge graph reasoning via auto-encoder structure [C]// Proceedings of the 31st International Joint Conference on Artificial Intelligence . Vienna: ACM, 2022: 23-29.

[28]	YU Y, SI X, HU C, et al A review of recurrent neural networks: LSTM cells and network architectures[J]. Neural Computation, 2019, 31 (7): 1235- 1270 doi: 10.1162/neco_a_01199

[29]	HAN Z, ZHANG G, MA Y, et al. Time-dependent entity embedding is not all you need: a re-evaluation of temporal knowledge graph completion models under a unified framework [C]// Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing . Punta Cana: ACL, 2021: 8104-8118.

[30]	DAI Y, WANG S, XIONG N N, et al A survey on knowledge graph embedding: approaches, applications and benchmarks[J]. Electronics, 2020, 9 (5): 750 doi: 10.3390/electronics9050750

[31]	LEBLAY J, CHEKOL M W, LIU X, et al. Towards temporal knowledge graph embeddings with arbitrary time precision [C]// Proceedings of the 29th ACM International Conference on Information and Knowledge Management. [S. l.]: ACM, 2020: 685-694.

[32]	KAZEMI S M, POOLE D Simple embedding for link prediction in knowledge graphs[J]. Advances in Neural Information Processing Systems, 2018, 31 (3): 4289- 4300

[33]	GOEL R, KAZEMI S M, BRUBAKER M, et al. Diachronic embedding for temporal knowledge graph completion [C]// Proceedings of the AAAI Conference on Artificial Intelligence . New York: AAAI, 2020, 34(4): 3988-3995.

[34]	XU C, NAYYERI M, ALKHOURY F, et al. TeRo: a time-aware knowledge graph embedding via temporal rotation [C]// Proceedings of the 28th International Conference on Computational Linguistics . Barcelona: ACL, 2020: 1583-1593.

[35]	JIN W, QU M, JIN X, et al. Recurrent event network: autoregressive structure inference over temporal knowledge graphs [C]// Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing . [S. l. ]: ACL, 2020: 6669-6683.

[36]	LI Z, JIN X, LI W, et al. Temporal knowledge graph reasoning based on evolutional representation learning [C]// Proceedings of the 44th International ACM SIGIR Conference on Research and Development in Information Retrieval . [S. l. ]: ACM, 2021: 408-417.

[1]	Canlin LI,Xinyue WANG,Lizhuang MA,Zhiwen SHAO,Wenjiao ZHANG. Image cartoonization incorporating attention mechanism and structural line extraction[J]. Journal of ZheJiang University (Engineering Science), 2024, 58(8): 1728-1737.

[2]	Shuhan WU,Dan WANG,Yuanfang CHEN,Ziyu JIA,Yueqi ZHANG,Meng XU. Attention-fused filter bank dual-view graph convolution motor imagery EEG classification[J]. Journal of ZheJiang University (Engineering Science), 2024, 58(7): 1326-1335.

[3]	Xianwei MA,Chaohui FAN,Weizhi NIE,Dong LI,Yiqun ZHU. Robust fault diagnosis method for failure sensors[J]. Journal of ZheJiang University (Engineering Science), 2024, 58(7): 1488-1497.

[4]	Jun YANG,Chen ZHANG. Semantic segmentation of 3D point cloud based on boundary point estimation and sparse convolution neural network[J]. Journal of ZheJiang University (Engineering Science), 2024, 58(6): 1121-1132.

[5]	Yuntang LI,Hengjie LI,Kun ZHANG,Binrui WANG,Shanyue GUAN,Yuan CHEN. Recognition of complex power lines based on novel encoder-decoder network[J]. Journal of ZheJiang University (Engineering Science), 2024, 58(6): 1133-1141.

[6]	Zhiwei XING,Shujie ZHU,Biao LI. Airline baggage feature perception based on improved graph convolutional neural network[J]. Journal of ZheJiang University (Engineering Science), 2024, 58(5): 941-950.

[7]	Yi LIU,Yidan CHEN,Lin GAO,Jiao HONG. Lightweight road extraction model based on multi-scale feature fusion[J]. Journal of ZheJiang University (Engineering Science), 2024, 58(5): 951-959.

[8]	Cuiting WEI,Weijian ZHAO,Bochao SUN,Yunyi LIU. Intelligent rebar inspection based on improved Mask R-CNN and stereo vision[J]. Journal of ZheJiang University (Engineering Science), 2024, 58(5): 1009-1019.

[9]	Hai HUAN,Yu SHENG,Chenxi GU. Global guidance multi-feature fusion network based on remote sensing image road extraction[J]. Journal of ZheJiang University (Engineering Science), 2024, 58(4): 696-707.

[10]	Song LI,Zhe WANG,Liping ZHANG. SL-tgStore: new temporal knowledge graph storage model[J]. Journal of ZheJiang University (Engineering Science), 2024, 58(3): 449-458.

[11]	Mingjun SONG,Wen YAN,Yizhao DENG,Junran ZHANG,Haiyan TU. Light-weight algorithm for real-time robotic grasp detection[J]. Journal of ZheJiang University (Engineering Science), 2024, 58(3): 599-610.

[12]	Xinhua YAO,Tao YU,Senwen FENG,Zijian MA,Congcong LUAN,Hongyao SHEN. Recognition method of parts machining features based on graph neural network[J]. Journal of ZheJiang University (Engineering Science), 2024, 58(2): 349-359.

[13]	Siyi QIN,Shaoyan GAI,Feipeng DA. Video object detection algorithm based on multi-level feature aggregation under mixed sampler[J]. Journal of ZheJiang University (Engineering Science), 2024, 58(1): 10-19.

[14]	Hai-feng LI,Xue-ying ZHANG,Shu-fei DUAN,Hai-rong JIA,Hui-zhi LIANG. Fusing generative adversarial network and temporal convolutional network for Mandarin emotion recognition[J]. Journal of ZheJiang University (Engineering Science), 2023, 57(9): 1865-1875.

[15]	Xiao-qiang ZHAO,Ze WANG,Zhao-yang SONG,Hong-mei JIANG. Image super-resolution reconstruction based on dynamic attention network[J]. Journal of ZheJiang University (Engineering Science), 2023, 57(8): 1487-1494.

Viewed

Full text

Abstract

Cited

Shared

Discussed