|
|
3D point cloud super-resolution with dynamic residual graph convolutional networks |
Fan ZHONG( ),Zheng-yao BAI*( ) |
School of Information Science and Engineering, Yunnan University, Kunming 650500, China |
|
|
Abstract A 3D point cloud super-resolution network with dynamic residual graph convolution (PSR-DRGCN) was proposed to efficiently extract of local information from 3D point clouds of non-European data in super-resolution. The network includes feature extraction module, DRGCN module and upsampling module. For the input point cloud, the feature extraction module locates k nearest points of each point in 3D space by k-NN algorithm and then converts the local geometry information into the high dimensional feature space through a multi-layer pointwise convolution. The DRGCN module converts the local geometry feature of each point into the semantic feature through a multi-layer graph convolution. It dynamically adjusts the neighbor space of the point in each layer to increase the receptive field range and effectively fuse the semantic information of different levels through residual connection, which makes the extraction of local geometric information efficient. The upsampling module adds the number of points and maps them from feature space to 3D space. The results showed that at 2× magnification of the high-resolution point cloud generated by PSR-DRGCN, the similarity indexes CD, EMD and F-score compared with the second network were increased by 10.00%, 4.76% and 16.84% respectively. Compared with the second network, the similarity indexes at 6× magnification were increased by 2.35%, 40.00% and 0.58% respectively. In all cases, the optimal effect was achieved on the mean and the std indicators and the generated high-resolution point cloud quality was high.
|
Received: 29 November 2021
Published: 02 December 2022
|
|
Fund: 云南省重大科技专项计划资助项目(202002AD080001) |
Corresponding Authors:
Zheng-yao BAI
E-mail: zffhost@mail.ynu.edu.cn;baizhy@ynu.edu.cn
|
采用动态残差图卷积的3D点云超分辨率
为了在超分辨率中对非欧数据的3D点云进行局部信息高效提取,提出采用动态残差图卷积的3D点云超分辨率网络(PSR-DRGCN),该网络包括特征提取模块、DRGCN模块及上采样模块. 对于输入的3D点云, 特征提取模块采用k-近邻(k-NN)算法在3D空间中找到每个点对应的k个邻居,通过逐点卷积把局部几何信息转换到高维特征空间中;DRGCN模块利用多层图卷积操作将3D空间中每个点的局部几何特征抽象为语义特征,在每一层对点的近邻空间进行动态调整以增加感受野范围,并通过残差连接融合多层次语义信息,从而对局部几何信息高效提取;上采样模块将特征空间中的点进行上采样并转换到3D空间中. 实验结果表明,PSR-DRGCN生成的高分辨率点云在放大尺度为2倍时,相似性指标CD、EMD、F-score相比第2网络分别优化了10.00%,4.76%,16.84%;当放大尺度为6倍时,相似性指标相比第2网络分别优化了2.35%,40.00%,0.58%;在所有情况下的均值与标准差指标上达到最优效果,生成的高分辨率点云质量高.
关键词:
3D点云,
超分辨率,
动态图卷积网络,
语义特征,
深度学习
|
|
[1] |
YANG J, WRIGHT J, HUANG T S, et al Image super-resolution via sparse representation[J]. IEEE Transactions on Image Processing, 2010, 19 (11): 2861- 2873
doi: 10.1109/TIP.2010.2050625
|
|
|
[2] |
QIAO C, LI D, GUO Y, et al Evaluation and development of deep neural networks for image super-resolution in optical microscopy[J]. Nature Methods, 2021, 18 (2): 194- 202
doi: 10.1038/s41592-020-01048-5
|
|
|
[3] |
段然, 周登文, 赵丽娟, 等 基于多尺度特征映射网络的图像超分辨率重建[J]. 浙江大学学报: 工学版, 2019, 53 (7): 1331- 1339 DUAN Ran, ZHOU Deng-Wen, ZHAO Li-Juan, et al Image super-resolution reconstruction based on multi-scale feature mapping network[J]. Journal of Zhejiang University: Engineering Science, 2019, 53 (7): 1331- 1339
|
|
|
[4] |
DONG C, LOY C C, HE K, et al Image super-resolution using deep convolutional networks[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2015, 38 (2): 295- 307
|
|
|
[5] |
CHEN Y, LIU L, PHONEVILAY V, et al Image super-resolution reconstruction based on feature map attention mechanism[J]. Applied Intelligence, 2021, (8): 1- 14
|
|
|
[6] |
WU H, ZHANG J, HUANG K. Point cloud super resolution with adversarial residual graph networks [EB/OL]. [2019-08-06]. https://arxiv.org/pdf/1908.0211.pdf .
|
|
|
[7] |
YU L, LI X, FU C W, et al. Pu-net: point cloud upsampling network [C]// Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Salt Lake City: IEEE, 2018: 2790-2799.
|
|
|
[8] |
LI R, LI X, FU C W, et al. Pu-gan: a point cloud upsampling adversarial network [C]// Proceedings of the IEEE/CVF International Conference on Computer Vision. Seoul: IEEE, 2019: 7203-7212.
|
|
|
[9] |
QI C R, SU H, MO K, et al. Pointnet: deep learning on point sets for 3d classification and segmentation [C]// Proceedings of the IEEE conference on computer vision and pattern recognition. Hawaii: IEEE, 2017: 652-660.
|
|
|
[10] |
LI R, LI X, HENG P A, et al. Point cloud upsampling via Disentangled Refinement [C]// Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. Online: IEEE, 2021: 344-353.
|
|
|
[11] |
QI C R, YI L, SU H, et al. Pointnet++: deep hierarchical feature learning on point sets in a metric space [C]// Proceedings of Neural Information Processing Systems. Long Beach: NIPS, 2017: 5099-5108.
|
|
|
[12] |
彭秀平, 仝其胜, 林洪彬, 等 一种面向散乱点云语义分割的深度残差特征金字塔网络框架[J]. 自动化学报, 2021, 47 (12): 2831- 2840 PENG Xiu-Ping, TONG Qi-Sheng, LIN Hong-Bin, et al A deep residual-feature pyramid network for scattered point cloud semantic segmentation[J]. Acta Automatica Sinica, 2021, 47 (12): 2831- 2840
|
|
|
[13] |
MATURANA D, SCHERER S. Voxnet: a 3d convolutional neural network for real-time object recognition [C]// Proceedings of IEEE/RSJ International Conference on Intelligent Robots and Systems. Hamburg: IEEE, 2015: 922-928.
|
|
|
[14] |
MAO J, XUE Y, NIU M, et al. Voxel transformer for 3d object detection [C]// Proceedings of the IEEE/CVF International Conference on Computer Vision. Online: IEEE, 2021: 3164-3173.
|
|
|
[15] |
WANG G, XU G, WU Q, et al Two-stage point cloud super resolution with local interpolation and readjustment via outer-product neural network[J]. Journal of Systems Science and Complexity, 2021, 34 (1): 68- 82
doi: 10.1007/s11424-020-9266-x
|
|
|
[16] |
SIMONOYSKY M, KOMODAKIS N. Dynamic edge-conditioned flters in convolutional neural networks on graphs [C]// Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Hawaii: IEEE, 2017: 3693-3702.
|
|
|
[17] |
TE G, HU W, ZHENG A, et al. Rgcnn: regularized graph cnn for point cloud segmentation [C]// Proceedings of the 26th ACM International Conference on Multimedia. Lisboa: ACM, 2018: 746-754.
|
|
|
[18] |
QIAN G, ABUALSHOUR A, LI G, et al. Pu-gcn: point cloud upsampling using graph convolutional networks [C]// Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. Online: IEEE, 2021: 11683-11692.
|
|
|
[19] |
GUO M H, CAI J X, LIU Z N, et al Pct: point cloud transformer[J]. Computational Visual Media, 2021, 7 (2): 187- 199
doi: 10.1007/s41095-021-0229-5
|
|
|
[20] |
YU X, RAO Y, WANG Z, et al. Pointr: diverse point cloud completion with geometry-aware transformers [C]// Proceedings of the IEEE/CVF International Conference on Computer Vision. Online: IEEE, 2021: 12498-12507.
|
|
|
[21] |
HE K, ZHANG X, REN S, et al. Deep residual learning for image recognition [C]// Proceedings of the IEEE conference on computer vision and pattern recognition. Las Vegas: IEEE, 2016: 770-778.
|
|
|
[22] |
KIPF T N, WELLING M. Semi-supervised classification with graph convolutional networks [EB/OL]. [2016-09-09]. https://arxiv.org/pdf/1609.02907.pdf.
|
|
|
[23] |
LUO C, CHEN Y, WANG N, et al. Spectral feature transformation for person re-identification [C]// Proceedings of the IEEE/CVF International Conference on Computer Vision. Seoul: IEEE, 2019: 4976-4985.
|
|
|
[24] |
HU X, MU H, ZHANG X, et al. Meta-SR: a magnification-arbitrary network for super-resolution [C]// Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. Long Beach: IEEE, 2019: 1575-1584.
|
|
|
[25] |
BUTT M A, MARAGOS P Optimum design of chamfer distance transforms[J]. IEEE Transactions on Image Processing, 1998, 7 (10): 1477- 1484
doi: 10.1109/83.718487
|
|
|
[26] |
CUTURI M. Sinkhorn distances: lightspeed computation of optimal transport [C]// Proceedings of the 26th International Conference on Neural Information Processing Systems. Lake Tahoe: NIPS, 2013: 2292-2300.
|
|
|
[27] |
YE S, CHEN D, HAN S, et al Meta-PU: an arbitrary-scale upsampling network for point cloud[J]. IEEE Transactions on Visualization and Computer Graphics, 2022, 28 (9): 3206- 3218
doi: 10.1109/TVCG.2021.3058311
|
|
|
[28] |
BOLTZ S, NIELSEN F, SOATTO S. Earth mover distance on superpixels [C]// IEEE International Conference on Image Processing. Hong Kong: IEEE, 2010: 4597-4600.
|
|
|
[29] |
SOKOLOVA M, JAPKOWICZ N, SZPAKOWICZ S. Beyond accuracy, F-score and ROC: a family of discriminant measures for performance evaluation [C]// Australasian Joint Conference on Artificial Intelligence. Berlin: Springer, 2006: 1015-1021.
|
|
|
|
Viewed |
|
|
|
Full text
|
|
|
|
|
Abstract
|
|
|
|
|
Cited |
|
|
|
|
|
Shared |
|
|
|
|
|
Discussed |
|
|
|
|