|
|
|
| Graph neural network based node embedding enhancement model for node classification |
Ju-xiang ZENG( ),Ping-hui WANG*(),Yi-dong DING,Lin LAN,Lin-xi CAI,Xiao-hong GUAN |
| Faculty of Electronic and Information Engineering, Xi’an Jiaotong University, Xi’an 710049, China |
|
|
|
Abstract In reality, the structure of most graphs could be noisy, i.e., including some noisy edges or ignoring some edges that exist between nodes in practice. To solve these challenges, a novel differentiable similarity module (DSM), which boosted node representations by digging implict association between nodes to improve the accuracy of node classification, was presented. Basic representation of each target node was learnt by DSM using an ordinary graph neural network (GNN), similar node sets were selected in terms of node representation similarity and the basic representation of the similar nodes was integrated to boost the target node’s representation. Mathematically, DSM is differentiable, so it is possible to combine DSM as plug-in with arbitrary GNNs and train them in an end-to-end fashion. DSM enables to exploit the implicit edges between nodes and make the learned representations more robust and discriminative. Experiments were conducted on several public node classification datasets. Results demonstrated that with GNNs equipped with DSM, the classification accuracy can be significantly improved, for example, GAT-DSM outperformed GAT by significant margins of 2.9% on Cora and 3.5% on Citeseer.
|
|
Received: 04 August 2022
Published: 28 February 2023
|
|
|
| Fund: 国家重点研发计划资助项目（2021YFB1715600）；国家自然科学基金资助项目（61902305，61922067）；深圳基础研究基金资助项目（JCYJ20170816100819428）；“人工智能”教育部-中国移动建设资助项目（MCM20190701） |
|
Corresponding Authors:
Ping-hui WANG
E-mail: jxzeng@sei.xjtu.edu.cn;phwang@xjtu.edu.cn
|
面向节点分类的图神经网络节点嵌入增强模型
考虑到实际的图结构往往是有噪的，可能包含实际不存在的边或者遗漏节点间实际存在的部分边，提出可微分相似度模型（DSM）. 通过挖掘节点间隐藏关系增强节点嵌入，以提高节点分类的准确度. DSM基于普通图神经网络方法（GNN）得到各节点的基础表征，根据节点表征相似度为目标节点选出相似节点集合，结合相似节点集合的基础表征对目标节点进行嵌入表征增强. 在数学上，DSM 是可微分的，可以将 DSM 作为插件与任意 GNN 相结合，以端到端的方式进行训练. DSM具有挖掘隐藏连接关系的能力，能促使GNNs学习到更具辨识性和鲁棒性的节点表征. 基于最常用的多个公开的节点分类数据集，开展实验验证. 结果表明，将已有GNNs与DSM结合能显著提升分类准确度，其中GAT-DSM相对GAT在数据集Cora和Citeseer上分别取得了2.9%、3.5%的提升．
关键词：
节点分类,
有监督节点分类,
图神经网络,
神经网络,
深度学习
|
|
| [1] |
WU Z, PAN S, CHEN F, et al A comprehensive survey on graph neural networks[J]. IEEE Transactions on Neural Networks and Learning Systems, 2019, 32 (1): 4- 24
|
|
|
| [2] |
YAO L, MAO C, LUO Y. Graph convolutional networks for text classification [C]// Proceedings of the 33rd AAAI Conference on Artificial Intelligence. Menlo Park: AAAI, 2019: 7370-7377.
|
|
|
| [3] |
ZHANG Z, CUI P, ZHU W Deep learning on graphs: a survey[J]. IEEE Transactions on Knowledge and Data Engineering, 2022, 34 (1): 249- 270
doi: 10.1109/TKDE.2020.2981333
|
|
|
| [4] |
AKOGLU L, TONG H, KOUTRA D Graph based anomaly detection and description: a survey[J]. Data Mining and Knowledge Discovery, 2015, 29 (3): 626- 688
doi: 10.1007/s10618-014-0365-y
|
|
|
| [5] |
WANG P, XU B, XU J, et al Semantic expansion using word embedding clustering and convolutional neural network for improving short text classification[J]. Neurocomputing, 2016, 174: 806- 814
doi: 10.1016/j.neucom.2015.09.096
|
|
|
| [6] |
SHCHUR O, MUMME M, BOJCHEVSKI A, et al. Pitfalls of graph neural network evaluation [EB/OL]. [2022-04-02]. https://arxiv.org/abs/1811.05868.
|
|
|
| [7] |
ZHU D, ZHANG Z, CUI P, et al. Robust graph convolutional networks against adversarial attacks [C]// Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. New York: ACM, 2019.
|
|
|
| [8] |
HENAFF M, BRUNA J, LECUN Y. Deep convolutional networks on graph-structured data [EB/OL]. [2022-04-02]. http://arxiv.org/abs/1506.05163.
|
|
|
| [9] |
ZHANG Y, PAL S, COATES M J, et al. Bayesian graph convolutional neural networks for semi-supervised classification [C]// Proceedings of the 33rd AAAI Conference on Artificial Intelligence. Menlo Park: AAAI, 2019: 5829-5836.
|
|
|
| [10] |
HAMILTON W, YING Z, LESKOVEC J. Inductive Representation learning on large graphs [C]// Proceedings of the 31st International Conference on Neural Information Processing Systems. New York: Curran Associates, Inc., 2017: 1024-1034.
|
|
|
| [11] |
VELICKOVIC P, CUCURULL G, CASANOVA A, et al. Graph attention networks [C]// Proceedings of International Conference on Learning Representations 2018. Vancouver: [s.n.], 2018.
|
|
|
| [12] |
DUVENAUD D K, MACLAURIN D, IPARRAGUIRRE J, et al. Convolutional networks on graphs for learning molecular fingerprints [C]// Proceedings of the 29th International Conference on Neural Information Processing Systems. New York: Curran Associates, Inc., 2015: 2224-2232.
|
|
|
| [13] |
NIEPERT M, AHMED M, KUTZKOV K. Learning convolutional neural networks for graphs [C]// Proceedings of the 33rd International Conference on Machine Learning. New York: ACM, 2016: 2014-2023.
|
|
|
| [14] |
WANG X, JI H, SHI C, et al. Heterogeneous graph attention network [C]// Proceedings of the World Wide Web Conference 2019. New York: ACM, 2019: 2022-2032.
|
|
|
| [15] |
BRUNA J, ZAREMBA W, SZLAM A, et al. Spectral networks and locally connected networks on graphs [C]// Proceedings of the 31st International Conference on Machine Learning. New York: ACM, 2014.
|
|
|
| [16] |
SCARSELLI F, GORI M, TSOI A C, et al The graph neural network model[J]. IEEE Transactions on Neural Networks, 2009, 20 (1): 61- 80
doi: 10.1109/TNN.2008.2005605
|
|
|
| [17] |
DEFFERRARD M E L, BRESSON X, VANDERGHEYNST P. Convolutional neural networks on graphs with fast localized spectral filtering [C]// Proceedings of the 30th International Conference on Neural Information Processing Systems. New York: Curran Associates, Inc., 2016: 3844-3852.
|
|
|
| [18] |
KIPF T N, WELLING M. Semi-supervised classification with graph convolutional networks [C]// Proceedings of International Conference on Learning Representations 2017. Toulon: [s.n.], 2017.
|
|
|
| [19] |
SEN P, NAMATA G, BILGIC M, et al Collective classification in network data[J]. AI Magazine, 2008, 29 (3): 93- 106
doi: 10.1609/aimag.v29i3.2157
|
|
|
| [20] |
ABADI M I N, BARHAM P, CHEN J, et al. TensorFlow: a system for large-scale machine learning [C]// Proceedings of the 12th USENIX Conference on Operating Systems Design and Implementation. Berkeley: USENIX, 2016: 265-283.
|
|
|
| [21] |
PEROZZI B, AL-RFOU R, SKIENA S. DeepWalk: online learning of social representations [C]// Proceedings of the 20th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. New York: ACM, 2014: 701-710.
|
|
|
| [22] |
VAN DER MAATEN L, HINTON G E Visualizing data using t-SNE[J]. Journal of Machine Learning Research, 2008, 9 (86): 2579- 2605
|
|
|
| [23] |
VERMA V, QU M, KAWAGUCHI K, et al. GraphMix: improved training of GNNs for semi-supervised learning [C]// Proceedings of the AAAI Conference on Artificial Intelligence. [s.l.]: AAAI, 2021: 10024-10032.
|
|
|
| [24] |
ZENG J, WANG P, LAN L, et al. Accurate and scalable graph neural networks for billion-scale graphs [C]// Proceedings of the 38th International Conference on Data Engineering. Kuala Lumpur: IEEE, 2022: 110-122.
|
|
|
|
Viewed |
|
|
|
Full text
|
|
|
|
|
Abstract
|
|
|
|
|
Cited |
|
|
|
|
| |
Shared |
|
|
|
|
| |
Discussed |
|
|
|
|
