面向节点分类的图神经网络节点嵌入增强模型

doi:10.3785/j.issn.1008-973X.2023.02.001

浙江大学学报(工学版)

2023, Vol. 57

Issue (2): 219-225 DOI: 10.3785/j.issn.1008-973X.2023.02.001

计算机技术

面向节点分类的图神经网络节点嵌入增强模型

曾菊香(

),王平辉*(

),丁益东,兰林,蔡林熹,管晓宏

西安交通大学电子与信息工程学部，陕西西安 710049

Graph neural network based node embedding enhancement model for node classification

Ju-xiang ZENG(

),Ping-hui WANG*(

),Yi-dong DING,Lin LAN,Lin-xi CAI,Xiao-hong GUAN

Faculty of Electronic and Information Engineering, Xi’an Jiaotong University, Xi’an 710049, China

全文: PDF(1473 KB) HTML

摘要：

考虑到实际的图结构往往是有噪的，可能包含实际不存在的边或者遗漏节点间实际存在的部分边，提出可微分相似度模型（DSM）. 通过挖掘节点间隐藏关系增强节点嵌入，以提高节点分类的准确度. DSM基于普通图神经网络方法（GNN）得到各节点的基础表征，根据节点表征相似度为目标节点选出相似节点集合，结合相似节点集合的基础表征对目标节点进行嵌入表征增强. 在数学上，DSM 是可微分的，可以将 DSM 作为插件与任意 GNN 相结合，以端到端的方式进行训练. DSM具有挖掘隐藏连接关系的能力，能促使GNNs学习到更具辨识性和鲁棒性的节点表征. 基于最常用的多个公开的节点分类数据集，开展实验验证. 结果表明，将已有GNNs与DSM结合能显著提升分类准确度，其中GAT-DSM相对GAT在数据集Cora和Citeseer上分别取得了2.9%、3.5%的提升．

关键词： 节点分类; 有监督节点分类; 图神经网络; 神经网络; 深度学习

Abstract:

In reality, the structure of most graphs could be noisy, i.e., including some noisy edges or ignoring some edges that exist between nodes in practice. To solve these challenges, a novel differentiable similarity module (DSM), which boosted node representations by digging implict association between nodes to improve the accuracy of node classification, was presented. Basic representation of each target node was learnt by DSM using an ordinary graph neural network (GNN), similar node sets were selected in terms of node representation similarity and the basic representation of the similar nodes was integrated to boost the target node’s representation. Mathematically, DSM is differentiable, so it is possible to combine DSM as plug-in with arbitrary GNNs and train them in an end-to-end fashion. DSM enables to exploit the implicit edges between nodes and make the learned representations more robust and discriminative. Experiments were conducted on several public node classification datasets. Results demonstrated that with GNNs equipped with DSM, the classification accuracy can be significantly improved, for example, GAT-DSM outperformed GAT by significant margins of 2.9% on Cora and 3.5% on Citeseer.

Key words: node classification supervised node classification graph neural network neural network deep learning

收稿日期: 2022-08-04 出版日期: 2023-02-28

CLC:

TP 29

基金资助: 国家重点研发计划资助项目（2021YFB1715600）；国家自然科学基金资助项目（61902305，61922067）；深圳基础研究基金资助项目（JCYJ20170816100819428）；“人工智能”教育部-中国移动建设资助项目（MCM20190701）

通讯作者: 王平辉 E-mail: jxzeng@sei.xjtu.edu.cn;phwang@xjtu.edu.cn

作者简介: 曾菊香（1995—），女，博士生，从事图神经网络研究. orcid.org/0000-0002-0935-0715. E-mail: jxzeng@sei.xjtu.edu.cn

	服务
	把本文推荐给朋友
	加入引用管理器
	E-mail Alert
	作者相关文章
	曾菊香
	王平辉
	丁益东
	兰林
	蔡林熹
	管晓宏

引用本文:

曾菊香,王平辉,丁益东,兰林,蔡林熹,管晓宏. 面向节点分类的图神经网络节点嵌入增强模型[J]. 浙江大学学报(工学版), 2023, 57(2): 219-225.

Ju-xiang ZENG,Ping-hui WANG,Yi-dong DING,Lin LAN,Lin-xi CAI,Xiao-hong GUAN. Graph neural network based node embedding enhancement model for node classification. Journal of ZheJiang University (Engineering Science), 2023, 57(2): 219-225.

链接本文:

https://www.zjujournals.com/eng/CN/10.3785/j.issn.1008-973X.2023.02.001 或 https://www.zjujournals.com/eng/CN/Y2023/V57/I2/219

图 1 GNN-DSM的整体结构

表 1 基准数据集的情况总结

表 2 Cora数据集上的分类准确度

表 3 Citeseer数据集上的分类准确度

表 4 Pubmed数据集上的分类准确度

图 2 5种不同节点表示学习方法在Cora、Citeseer、Pubmed数据集上的节点表征分布t-SNE图

图 3 Cora数据集上相似节点数量、超参数、损失平衡因子对准确度的影响

1	WU Z, PAN S, CHEN F, et al A comprehensive survey on graph neural networks[J]. IEEE Transactions on Neural Networks and Learning Systems, 2019, 32 (1): 4- 24
2	YAO L, MAO C, LUO Y. Graph convolutional networks for text classification [C]// Proceedings of the 33rd AAAI Conference on Artificial Intelligence. Menlo Park: AAAI, 2019: 7370-7377.
3	ZHANG Z, CUI P, ZHU W Deep learning on graphs: a survey[J]. IEEE Transactions on Knowledge and Data Engineering, 2022, 34 (1): 249- 270 doi: 10.1109/TKDE.2020.2981333
4	AKOGLU L, TONG H, KOUTRA D Graph based anomaly detection and description: a survey[J]. Data Mining and Knowledge Discovery, 2015, 29 (3): 626- 688 doi: 10.1007/s10618-014-0365-y
5	WANG P, XU B, XU J, et al Semantic expansion using word embedding clustering and convolutional neural network for improving short text classification[J]. Neurocomputing, 2016, 174: 806- 814 doi: 10.1016/j.neucom.2015.09.096
6	SHCHUR O, MUMME M, BOJCHEVSKI A, et al. Pitfalls of graph neural network evaluation [EB/OL]. [2022-04-02]. https://arxiv.org/abs/1811.05868.
7	ZHU D, ZHANG Z, CUI P, et al. Robust graph convolutional networks against adversarial attacks [C]// Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. New York: ACM, 2019.
8	HENAFF M, BRUNA J, LECUN Y. Deep convolutional networks on graph-structured data [EB/OL]. [2022-04-02]. http://arxiv.org/abs/1506.05163.
9	ZHANG Y, PAL S, COATES M J, et al. Bayesian graph convolutional neural networks for semi-supervised classification [C]// Proceedings of the 33rd AAAI Conference on Artificial Intelligence. Menlo Park: AAAI, 2019: 5829-5836.
10	HAMILTON W, YING Z, LESKOVEC J. Inductive Representation learning on large graphs [C]// Proceedings of the 31st International Conference on Neural Information Processing Systems. New York: Curran Associates, Inc., 2017: 1024-1034.
11	VELICKOVIC P, CUCURULL G, CASANOVA A, et al. Graph attention networks [C]// Proceedings of International Conference on Learning Representations 2018. Vancouver: [s.n.], 2018.
12	DUVENAUD D K, MACLAURIN D, IPARRAGUIRRE J, et al. Convolutional networks on graphs for learning molecular fingerprints [C]// Proceedings of the 29th International Conference on Neural Information Processing Systems. New York: Curran Associates, Inc., 2015: 2224-2232.
13	NIEPERT M, AHMED M, KUTZKOV K. Learning convolutional neural networks for graphs [C]// Proceedings of the 33rd International Conference on Machine Learning. New York: ACM, 2016: 2014-2023.
14	WANG X, JI H, SHI C, et al. Heterogeneous graph attention network [C]// Proceedings of the World Wide Web Conference 2019. New York: ACM, 2019: 2022-2032.
15	BRUNA J, ZAREMBA W, SZLAM A, et al. Spectral networks and locally connected networks on graphs [C]// Proceedings of the 31st International Conference on Machine Learning. New York: ACM, 2014.
16	SCARSELLI F, GORI M, TSOI A C, et al The graph neural network model[J]. IEEE Transactions on Neural Networks, 2009, 20 (1): 61- 80 doi: 10.1109/TNN.2008.2005605
17	DEFFERRARD M E L, BRESSON X, VANDERGHEYNST P. Convolutional neural networks on graphs with fast localized spectral filtering [C]// Proceedings of the 30th International Conference on Neural Information Processing Systems. New York: Curran Associates, Inc., 2016: 3844-3852.
18	KIPF T N, WELLING M. Semi-supervised classification with graph convolutional networks [C]// Proceedings of International Conference on Learning Representations 2017. Toulon: [s.n.], 2017.
19	SEN P, NAMATA G, BILGIC M, et al Collective classification in network data[J]. AI Magazine, 2008, 29 (3): 93- 106 doi: 10.1609/aimag.v29i3.2157
20	ABADI M I N, BARHAM P, CHEN J, et al. TensorFlow: a system for large-scale machine learning [C]// Proceedings of the 12th USENIX Conference on Operating Systems Design and Implementation. Berkeley: USENIX, 2016: 265-283.
21	PEROZZI B, AL-RFOU R, SKIENA S. DeepWalk: online learning of social representations [C]// Proceedings of the 20th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. New York: ACM, 2014: 701-710.
22	VAN DER MAATEN L, HINTON G E Visualizing data using t-SNE[J]. Journal of Machine Learning Research, 2008, 9 (86): 2579- 2605
23	VERMA V, QU M, KAWAGUCHI K, et al. GraphMix: improved training of GNNs for semi-supervised learning [C]// Proceedings of the AAAI Conference on Artificial Intelligence. [s.l.]: AAAI, 2021: 10024-10032.
24	ZENG J, WANG P, LAN L, et al. Accurate and scalable graph neural networks for billion-scale graphs [C]// Proceedings of the 38th International Conference on Data Engineering. Kuala Lumpur: IEEE, 2022: 110-122.

[1]	黄华,赵秋舸,何再兴,李嘉然. 基于LSTM与牛顿迭代的两轴系统轮廓误差控制[J]. 浙江大学学报(工学版), 2023, 57(1): 10-20.
[2]	叶晨,战洪飞,林颖俊,余军合,王瑞,钟武昌. 基于推理-情境感知激活模型的设计知识推荐[J]. 浙江大学学报(工学版), 2023, 57(1): 32-46.
[3]	戴理朝,曹威,易善昌,王磊. 基于WPT-SVD和GA-BPNN的混凝土结构损伤识别[J]. 浙江大学学报(工学版), 2023, 57(1): 100-110.
[4]	马骏驰,迪骁鑫,段宗涛,唐蕾. 程序表示学习综述[J]. 浙江大学学报(工学版), 2023, 57(1): 155-169.
[5]	刘近贞,陈飞,熊慧. 多尺度残差网络模型的开放式电阻抗成像算法[J]. 浙江大学学报(工学版), 2022, 56(9): 1789-1795.
[6]	王万良,王铁军,陈嘉诚,尤文波. 融合多尺度和多头注意力的医疗图像分割方法[J]. 浙江大学学报(工学版), 2022, 56(9): 1796-1805.
[7]	郝琨,王阔,王贝贝. 基于改进Mobilenet-YOLOv3的轻量级水下生物检测算法[J]. 浙江大学学报(工学版), 2022, 56(8): 1622-1632.
[8]	夏杰锋,唐武勤,杨强. 光伏航拍红外图像的热斑自动检测方法[J]. 浙江大学学报(工学版), 2022, 56(8): 1640-1647.
[9]	杨正银,胡健,姚建勇,沙英哲,宋秋雨. 基于自适应神经网络滑模观测器的容错控制[J]. 浙江大学学报(工学版), 2022, 56(8): 1656-1665.
[10]	贺俊,张雅声,尹灿斌. 基于深度学习的星载SAR工作模式鉴别[J]. 浙江大学学报(工学版), 2022, 56(8): 1676-1684.
[11]	牛伟宏,黄晓峰,祁伟,殷海兵,颜成钢. H.266/VVC分步全零块判决快速算法[J]. 浙江大学学报(工学版), 2022, 56(7): 1285-1293, 1319.
[12]	莫仁鹏,司小胜,李天梅,朱旭. 基于多尺度特征与注意力机制的轴承寿命预测[J]. 浙江大学学报(工学版), 2022, 56(7): 1447-1456.
[13]	赵永胜,李瑞祥,牛娜娜,赵志勇. 数字孪生驱动的机身形状控制方法[J]. 浙江大学学报(工学版), 2022, 56(7): 1457-1463.
[14]	温竹鹏,陈捷,刘连华,焦玲玲. 基于小波变换和优化CNN的风电齿轮箱故障诊断[J]. 浙江大学学报(工学版), 2022, 56(6): 1212-1219.
[15]	于军琪,杨思远,赵安军,高之坤. 基于神经网络的建筑能耗混合预测模型[J]. 浙江大学学报(工学版), 2022, 56(6): 1220-1231.

Viewed

Full text

Abstract

Cited

Shared

Discussed