Please wait a minute...
浙江大学学报(工学版)
浙江大学学报(工学版)  2023, Vol. 57 Issue (1): 133-143    DOI: 10.3785/j.issn.1008-973X.2023.01.014
计算机技术、通信工程     
基于Transformer和知识图谱的新闻推荐新方法
凤丽洲1(),杨阳1,王友卫2,*(),杨贵军1
1. 天津财经大学 统计学院，天津 300222
2. 中央财经大学 信息学院，北京 100081
New method for news recommendation based on Transformer and knowledge graph
Li-zhou FENG1(),Yang YANG1,You-wei WANG2,*(),Gui-jun YANG1
1. School of Statistics, Tianjin University of Finance and Economics, Tianjin 300222, China
2. School of Information, Central University of Finance and Economics, Beijing 100081, China
 全文: PDF(1590 KB)   HTML
摘要：

为了增加新闻推荐的辅助信息并提高预测精度，提出基于Transformer和知识图谱的新闻推荐方法. 为了结合新闻语义信息和实体信息，利用自注意力机制获取新闻单词之间和新闻实体之间的联系，采用加法注意力机制捕捉单词和实体对新闻表示的影响. 考虑到用户对新闻的偏好具有时序性特点，引入Transformer以捕捉用户点击新闻间的关联信息及用户兴趣随时间的变化情况. 利用知识图谱中的高阶结构信息，融合候选新闻邻接实体，提升候选新闻嵌入向量所含信息的完整性. 在2个版本的MIND新闻数据集上与5个典型推荐方法的对比实验表明，注意力机制、Transformer和知识图谱的引入提高了算法在新闻推荐方面的表现.
关键词： 新闻推荐;  知识图谱;  注意力机制;  新闻实体;  高阶结构信息    
Abstract:

A news recommendation method based on Transformer and knowledge graph was proposed to increase the auxiliary information and improve the prediction accuracy. The self-attention mechanism was used to obtain the connection between news words and news entities in order to combine news semantic information and entity information. The additive attention mechanism was employed to capture the influence of words and entities on news representation. Transformer was introduced to pick up the correlation information between clicked news of user and capture the change of user interest over time by considering the time-series characteristics of user preference for news. High-order structural information in knowledge graphs was used to fuse adjacent entities of the candidate news and enhance the integrity of the information contained in the candidate news embedding vector. The comparison experiments with five typical recommendation methods on two versions of the MIND news dataset show that the introduction of attention mechanism, Transformer and knowledge graph can improve the performance of the algorithm on news recommendation.
Key words: news recommendation    knowledge graph    attention mechanism    news entity    high-order structural information
收稿日期: 2022-04-09 出版日期: 2023-01-17
CLC:  TP 391  
基金资助: 国家自然科学基金资助项目(61906220)；国家社科基金资助项目(18CTJ008)；教育部人文社科资助项目(19YJCZH178)；天津市自然科学基金资助项目(18JCQNJC69600)；内蒙古纪检监察大数据实验室2020—2021年度开放课题资助项目(IMDBD202002, IMDBD202004)
通讯作者: 王友卫     E-mail: flzvg@126.com;ywwang15@126.com
作者简介: 凤丽洲（1987—），女，副教授，博士，从事机器学习、数据挖掘的研究. orcid.org/0000-0002-1010-8539. E-mail： flzvg@126.com
服务　　
把本文推荐给朋友
加入引用管理器
E-mail Alert
作者相关文章　　
凤丽洲
杨阳
王友卫
杨贵军

引用本文:

凤丽洲,杨阳,王友卫,杨贵军. 基于Transformer和知识图谱的新闻推荐新方法[J]. 浙江大学学报(工学版), 2023, 57(1): 133-143.

Li-zhou FENG,Yang YANG,You-wei WANG,Gui-jun YANG. New method for news recommendation based on Transformer and knowledge graph. Journal of ZheJiang University (Engineering Science), 2023, 57(1): 133-143.

链接本文:

https://www.zjujournals.com/eng/CN/10.3785/j.issn.1008-973X.2023.01.014    或    https://www.zjujournals.com/eng/CN/Y2023/V57/I1/133

图 1  知识图谱的示例
图 2  TKGN整体框架
图 3  注意力机制的计算流程
图 4  候选新闻实体嵌入的计算流程
数据集 Sn Se Mn Mw Me Mr Ps
MIND-small 93 698 108 497 22.5 10.8 1.1 6.4 339 498
MIND-large 173 550 203 947 21.0 10.7 1.2 8.9 3 870 640
表 1  数据集的描述分析
图 5  超参数调节实验
方法 MIND-small MIND-large
AUC F1 P R AUC F1 P R
NRMS 0.618 0.539 0.618 0.480 0.656 0.611 0.627 0.596
DKN 0.603 0.523 0.549 0.501 0.619 0.571 0.630 0.528
KGCN 0.582 0.335 0.449 0.267 0.604 0.372 0.598 0.273
RippleNet 0.598 0.529 0.544 0.515 0.613 0.567 0.622 0.524
GNUD 0.602 0.530 0.616 0.466 0.627 0.602 0.631 0.580
TKGN 0.637 0.563 0.623 0.515 0.679 0.624 0.655 0.601
表 2  不同方法在数据集上的性能比较
图 6  迭代次数对不同算法的影响
方法 AUC F1 ΔAUC ΔF1
NRMS 0.592 0.511 ?0.026 ?0.028
DKN 0.578 0.493 ?0.025 ?0.029
KGCN 0.559 0.230 ?0.023 ?0.105
RippleNet 0.556 0.507 ?0.042 ?0.022
GNUD 0.582 0.495 ?0.020 ?0.035
TKGN 0.619 0.541 ?0.018 ?0.022
表 3  不同方法在冷启动数据上的性能表现
图 7  消融实验
1 LI L, CHU W, LANGFORD J, et al. A contextual-bandit approach to personalized news article recommendation [C]// Proceedings of the 19th International Conference on World Wide Web. Raleigh: ACM, 2010: 661-670.
2 KOREN Y, BELL R, VOLINSKY C Matrix factorization techniques for recommender systems[J]. Computer, 2009, 42 (8): 30- 37
doi: 10.1109/MC.2009.263
3 SUN Z, GUO Q, YANG J, et al Research commentary on recommendations with side information: a survey and research directions[J]. Electronic Commerce Research and Applications, 2019, 37 (1): 1- 30
4 WANG H, ZHANG F, XIE X, et al. DKN: deep knowledge-aware network for news recommendation [C]// Proceedings of the 2018 World Wide Web Conference. Lyon: ACM, 2018: 1835-1844.
5 WANG H, ZHANG F, ZHAO M, et al. Multi-task feature learning for knowledge graph enhanced recommendation [C]// Proceedings of the 2019 World Wide Web Conference. San Francisco: ACM, 2019: 2000-2010.
6 XIAN Y, FU Z, MUTHUKRISHNAN S, et al. Reinforcement knowledge graph reasoning for explainable recommendation [C]// Proceedings of the 42nd International ACM SIGIR Conference on Research and Development in Information Retrieval. Paris: ACM, 2019: 285-294.
7 宁泽飞, 孙静宇, 王欣娟 基于知识图谱和标签感知的推荐算法[J]. 计算机科学, 2021, 48 (11): 192- 198
NING Ze-fei, SUN Jing-yu, WANG Xin-juan Recommendation algorithm based on knowledge graph and tag-aware[J]. Computer Science, 2021, 48 (11): 192- 198
doi: 10.11896/jsjkx.201000085
8 WANG H, ZHAO M, XIE X, et al. Knowledge graph convolutional networks for recommender systems [C]// Proceedings of the 2019 World Wide Web Conference. San Francisco: ACM, 2019: 3307-3313.
9 WANG H, ZHANG F, WANG J, et al. Ripplenet: propagating user preferences on the knowledge graph for recommender systems [C]// Proceedings of the 27th ACM International Conference on Information and Knowledge Management. Torino: ACM, 2018: 417-426.
10 刘羽茜, 刘玉奇, 张宗霖, 等 注入注意力机制的深度特征融合新闻推荐模型[J]. 计算机应用, 2022, 42 (2): 426- 432
LIU Yu-xi, LIU Yu-qi, ZHANG Zong-lin, et al News recommendation model with deep feature fusion injecting attention mechanism[J]. Computer Applications, 2022, 42 (2): 426- 432
11 CHEN Q, ZHAO H, LI W, et al. Behavior sequence transformer for e-commerce recommendation in Alibaba [C]// Proceedings of the 1st International Workshop on Deep Learning Practice for High-Dimensional Sparse Data. Anchorage: ACM, 2019: 1-4.
12 TANG J, WANG K. Personalized top-n sequential recommendation via convolutional sequence embedding [C]// Proceedings of the 11th ACM International Conference on Web Search and Data Mining. Marina Del Rey: ACM, 2018: 565-573.
13 冯永, 张备, 强保华, 等 MN-HDRM: 长短兴趣多神经网络混合动态推荐模型[J]. 计算机学报, 2019, 42 (1): 16- 28
FENG Yong, ZHANG Bei, QIANG Bao-hua, et al MN-HDRM: a novel hybrid dynamic recommendation model based on long-short-term interests multiple neural networks[J]. Journal of Computer Science, 2019, 42 (1): 16- 28
14 VASWANI A, SHAZEER N, PARMAR N, et al. Attention is all you need [C]// Proceedings of the 31st International Conference on Neural Information Processing Systems. Long Beach: MIT Press, 2017: 6000-6010.
15 BANSAL T, DAS M, BHATTACHARYYA C. Content driven user profiling for comment-worthy recommendations of news and blog articles [C]// Proceedings of the 9th ACM Conference on Recommender Systems. Vienna: ACM, 2015: 195-202.
16 KUMAR V, KHATTAR D, GUPTA S, et al. Deep neural architecture for news recommendation [C]// Proceedings of the 2017 Conference and Labs of the Evaluation Forum. Dublin: [s. n. ], 2017: 1-19.
17 ZHANG Q, LI J, JIA Q, et al. UNBERT: user-news matching BERT for news recommendation [C]// Proceedings of the 30th International Joint Conference on Artificial Intelligence. Montreal: Morgan Kaufmann, 2021: 3356-3362.
18 WU C, WU F, QI T, et al. Feedrec: news feed recommendation with various user feedbacks [C]// Proceedings of the ACM Web Conference. Lyon: ACM, 2022: 2088-2097.
19 QI T, WU F, WU C, et al. Personalized news recommendation with knowledge-aware interactive matching [C]// Proceedings of the 44th International ACM SIGIR Conference on Research and Development in Information Retrieval. Canada: ACM, 2021: 61-70.
20 LIU D, LIAN J, LIU Z, et al. Reinforced anchor knowledge graph generation for news recommendation reasoning [C]// Proceedings of the 27th ACM SIGKDD Conference on Knowledge Discovery and Data Mining. Singapore: ACM, 2021: 1055-1065.
21 VRANDECIC D, KROTZSCH M Wikidata: a free collaborative knowledgebase[J]. Communications of the ACM, 2014, 57 (10): 78- 85
doi: 10.1145/2629489
22 XU B, XU Y, LIANG J, et al. CN-DBpedia: a never-ending Chinese knowledge extraction system [C]// Proceedings of the 30th International Conference on Industrial, Engineering and Other Applications of Applied Intelligent Systems. Arras: Springer, 2017: 428-438.
23 SUCHANEK F M, KASNECI G, WEIKUM G. Yago: a core of semantic knowledge [C]// Proceedings of the 16th International Conference on World Wide Web. Banff: ACM, 2007: 697-706.
24 WU C, WU F, GE S, et al. Neural news recommendation with multi-head self-attention [C]// Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing. Hong Kong: ACL, 2019: 6389-6394.
25 WU F, QIAO Y, CHEN J H, et al. Mind: a large-scale dataset for news recommendation [C]// Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics. Stroudsburg: ACL, 2020: 3597-3606.
26 MIKOLOV T, CHEN K, CORRADO G, et al. Efficient estimation of word representations in vector space [C]// Proceedings of the 1st International Conference on Learning Representations. Scottsdale: [s. n. ], 2013: 1-12.
27 HU L, XU S, LI C, et al. Graph neural news recommendation with unsupervised preference disentanglement [C]// Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics. Stroudsburg: ACL, 2020: 4255-4264.
[1] 王林涛,毛齐. 基于RGB与深度信息融合的管片抓取位置测量方法[J]. 浙江大学学报(工学版), 2023, 57(1): 47-54.
[2] 郝琨,王阔,王贝贝. 基于改进Mobilenet-YOLOv3的轻量级水下生物检测算法[J]. 浙江大学学报(工学版), 2022, 56(8): 1622-1632.
[3] 莫仁鹏,司小胜,李天梅,朱旭. 基于多尺度特征与注意力机制的轴承寿命预测[J]. 浙江大学学报(工学版), 2022, 56(7): 1447-1456.
[4] 王友卫,童爽,凤丽洲,朱建明,李洋,陈福. 基于图卷积网络的归纳式微博谣言检测新方法[J]. 浙江大学学报(工学版), 2022, 56(5): 956-966.
[5] 鞠晓臣,赵欣欣,钱胜胜. 基于自注意力机制的桥梁螺栓检测算法[J]. 浙江大学学报(工学版), 2022, 56(5): 901-908.
[6] 陈成,张皞,李永强,冯远静. 关系生成图注意力网络的知识图谱链接预测[J]. 浙江大学学报(工学版), 2022, 56(5): 1025-1034.
[7] 张雪芹,李天任. 基于Cycle-GAN和改进DPN网络的乳腺癌病理图像分类[J]. 浙江大学学报(工学版), 2022, 56(4): 727-735.
[8] 许萌,王丹,李致远,陈远方. IncepA-EEGNet: 融合Inception网络和注意力机制的P300信号检测方法[J]. 浙江大学学报(工学版), 2022, 56(4): 745-753, 782.
[9] 柳长源,何先平,毕晓君. 融合注意力机制的高效率网络车型识别[J]. 浙江大学学报(工学版), 2022, 56(4): 775-782.
[10] 陈巧红,裴皓磊,孙麒. 基于视觉关系推理与上下文门控机制的图像描述[J]. 浙江大学学报(工学版), 2022, 56(3): 542-549.
[11] 农元君,王俊杰,陈红,孙文涵,耿慧,李书悦. 基于注意力机制和编码-解码架构的施工场景图像描述方法[J]. 浙江大学学报(工学版), 2022, 56(2): 236-244.
[12] 刘英莉,吴瑞刚,么长慧,沈韬. 铝硅合金实体关系抽取数据集的构建方法[J]. 浙江大学学报(工学版), 2022, 56(2): 245-253.
[13] 袁天乐,袁巨龙,朱勇建,郑翰辰. 基于改进YOLOv5的推力球轴承表面缺陷检测算法[J]. 浙江大学学报(工学版), 2022, 56(12): 2349-2357.
[14] 于楠晶,范晓飚,邓天民,冒国韬. 基于多头自注意力的复杂背景船舶检测算法[J]. 浙江大学学报(工学版), 2022, 56(12): 2392-2402.
[15] 陈巧红,李妃玉,孙麒,贾宇波. 基于LSTM与衰减自注意力的答案选择模型[J]. 浙江大学学报(工学版), 2022, 56(12): 2436-2444.