Please wait a minute...
Journal of ZheJiang University (Engineering Science)  2024, Vol. 58 Issue (2): 279-287    DOI: 10.3785/j.issn.1008-973X.2024.02.006
    
Variable rate compression of point cloud based on scene flow
Zhaoyi JIANG(),Wenqin ZOU,Shenghao ZHENG,Chao SONG,Bailin YANG*()
School of Computer Science and Technology, Zhejiang Gongshang University, Hangzhou 310018, China
Download: HTML     PDF(1060KB) HTML
Export: BibTeX | EndNote (RIS)      

Abstract  

A variable-rate dynamic point cloud compression network framework based on scene flow was proposed in order to address the problem of training multiple network models for existing dynamic point cloud compression neural networks. The raw dynamic point cloud was taken as input, and the scene flow network was utilized to estimate motion vectors. A channel gain module was introduced to evaluate and scale the latent vector channels while compressing motion vectors and residuals, achieving variable-rate control. A new joint training loss function was designed to end-to-end train the entire network framework by comprehensively considering the motion vector loss and rate-distortion loss. A human body dataset with motion vector labels was created based on the AMASS dataset for network training in order to solve the problem of lack of real motion information labels in dynamic point cloud datasets. The experimental results show that the compression bit rate of the method decreases by several orders of magnitude compared with existing deep learning-based dynamic point cloud compression methods. The method has a 5%~10% improvement compared with the reconstruction effect of static compression networks processing each frame separately.



Key wordsdynamic point cloud compression      variable rate      joint loss function      scene flow network     
Received: 18 August 2023      Published: 23 January 2024
CLC:  TP 37  
Fund:  国家自然科学基金资助项目(62172366); 浙江省自然科学基金资助项目(LY21F020013, LY22F020013).
Corresponding Authors: Bailin YANG     E-mail: zyjiang@zjgsu.edu.cn;ybl@zjgsu.edu.cn
Cite this article:

Zhaoyi JIANG,Wenqin ZOU,Shenghao ZHENG,Chao SONG,Bailin YANG. Variable rate compression of point cloud based on scene flow. Journal of ZheJiang University (Engineering Science), 2024, 58(2): 279-287.

URL:

https://www.zjujournals.com/eng/10.3785/j.issn.1008-973X.2024.02.006     OR     https://www.zjujournals.com/eng/Y2024/V58/I2/279


基于场景流的可变速率动态点云压缩

针对现有的动态点云压缩神经网络需要训练多个网络模型的问题,提出基于场景流的可变速率动态点云压缩网络框架. 网络以原始动态点云为输入, 利用场景流网络进行运动向量估计, 在压缩运动向量和残差的同时,引入通道增益模块对隐向量通道进行评估和缩放, 实现可变速率控制. 通过综合考虑运动向量损失和率失真损失, 设计新的联合训练损失函数, 用来端到端地训练整个网络框架. 为了解决动态点云数据集缺少真实运动信息标签的问题, 基于AMASS数据集制作带有运动向量标签的人体数据集,用于网络的训练. 实验结果显示, 与现有的基于深度学习动态点云压缩方法相比, 该方法的压缩比特率下降了几个数量级, 与静态压缩网络单独处理每帧的重构效果相比,该方法有5%~10%的提升.


关键词: 动态点云压缩,  可变速率,  联合损失函数,  场景流网络 
Fig.1 Overall framework of dynamic point cloud compression
Fig.2 Structure diagram of scene flow network
Fig.3 Schematic diagram of motion compensation network
Fig.4 Comparison of compression performance
方法bpp/bitsPSNR/dBte/std/s
Draco3.0636.0
VPCC34.741.9630.60
DDPCC24.535.60.450.17
Static0.13635.30.260.09
本文方法0.12237.50.30.12
Tab.1 Compression performance comparison results of different methods
Fig.5 Comparison of visualization results of different compression methods
数据集本文方法DDPCC
bpp/bitsPSNR/dBbpp/bitsPSNR/dB
Soldier0.12541.524.835.0
Loot0.11839.424.435.2
Longdress0.12235.424.335.7
Redandblack0.12036.124.435.6
Tab.2 Comparison of results with D-DPCC on 8i dataset
Fig.6 Comparison chart of loss function ablation
$ \beta $PSNR/dB
10028.8
135.7
0.137.3
0.0136.2
Tab.3 Experimental results of different sparse ablation
网络模型模型参数量/kB计算复杂度/MB
可变比特率41.272.8
非可变比特率206364
Tab.4 Complexity comparison of variable rate and unvariable rate models
[1]   KATHARIYA B, LI L, LI Z, et al. Lossless dynamic point cloud geometry compression with inter compensation and traveling salesman prediction [C]// Data Compression Conference . Boston: IEEE, 2018: 414.
[2]   THANOU D, CHOU P A, FROSSARD P Graph-based compression of dynamic 3D point cloud sequences[J]. IEEE Transactions on Image Processing, 2016, 25 (4): 1765- 1778
doi: 10.1109/TIP.2016.2529506
[3]   DE QUEIROZ R L, CHOU P A Motion-compensated compression of dynamic voxelized point clouds[J]. IEEE Transactions on Image Processing, 2017, 26 (8): 3886- 3895
doi: 10.1109/TIP.2017.2707807
[4]   KAMMERL J, BLODOW N, RUSU R B, et al. Real-time compression of point cloud streams [C]// IEEE International Conference on Robotics and Automation . [S. l.]: IEEE, 2012: 778-785.
[5]   SUN X, MA H, SUN Y, et al A novel point cloud compression algorithm based on clustering[J]. IEEE Robotics and Automation Letters, 2019, 4 (2): 2132- 2139
doi: 10.1109/LRA.2019.2900747
[6]   BISWAS S, LIU J, WONG K, et al Muscle: multi sweep compression of lidar using deep entropy models[J]. Advances in Neural Information Processing Systems, 2020, 33 (2): 22170- 22181
[7]   YU S, SUN S, YAN W, et al A method based on curvature and hierarchical strategy for dynamic point cloud compression in augmented and virtual reality system[J]. Sensors, 2022, 22 (3): 1262
doi: 10.3390/s22031262
[8]   FAN T, GAO L, XU Y, et al. D-DPCC: deep dynamic point cloud compression via 3D motion prediction [C]// Proceedings of the 31st International Joint Conference on Artificial Intelligence . Vienna: [s. n.], 2022: 898-904.
[9]   GEORGE T, SEAN M, SUNG J, et al. Variable rate image compression with recurrent neural networks [C]// International Conference on Learning Representations . Puerto Rico: [s. n.], 2016.
[10]   CHEN T, MA Z. Variable bitrate image compression with quality scaling factors [C]// IEEE International Conference on Acoustics, Speech and Signal Processing . Barcelona: IEEE, 2020: 2163-2167.
[11]   YANG F, HERRANZ L, WEIJER J, et al. Variable rate deep image compression with modulated autoencoder [J]. IEEE Signal Processing Letters , 2020, 27: 331-335.
[12]   CUI Z, WANG J, GAO S, et al. Asymmetric gained deep image compression with continuous rate adaptation [C]// Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition . Baltimore: IEEE, 2021: 10532-10541.
[13]   MUZADDID M A, BEKSI W J. Variable rate compression for raw 3D point clouds [C]// International Conference on Robotics and Automation . Philadelphia: [s. n.], 2022: 8748-8755.
[14]   QI C R, YI L, SU H, et al. Pointnet++: deep hierarchical feature learning on point sets in a metric space [C]// Advances in Neural Information Processing Systems . Long Beach: [s. n.], 2017: 30.
[15]   WU W, WANG Z Y, LI Z, et al. Pointpwc-net: cost volume on point clouds for (self-) supervised scene flow estimation [C]// 16th European Conference on Computer Vision. Glasgow: Springer, 2020.
[16]   WU W, QI Z, FUXIN L. Pointconv: deep convolutional networks on 3d point clouds [C]// Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition . Seattle: IEEE, 2019: 9621-9630.
[17]   ENGEL N, BELAGIANNIS V, DIETMAYER K Point transformer[J]. IEEE Access, 2021, 9 (1): 134826- 134840
[18]   HE Y, REN X, TANG D, et al. Density-preserving deep point cloud compression [C] // Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition . New Orleans: IEEE, 2022: 2333-2342.
[19]   LU G, OUYANG W, XU D, et al. Dvc: an end-to-end deep video compression framework [C]// Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition . Seattle: IEEE, 2019: 11006-11015.
[20]   WU Y, HE K. Group normalization [C]// Proceedings of the European Conference on Computer Vision . Munich: [s. n.], 2018: 3-19.
[21]   HUANG L, WANG S, WONG K, et al. Octsqueeze: octree-structured entropy model for lidar compression [C] // Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition . Seattle: IEEE, 2020: 1313-1323.
[22]   MAHMOOD N, GHORBANI N, TROJE N F, et al. AMASS: archive of motion capture as surface shapes [C]// Proceedings of the IEEE/CVF International Conference on Computer Vision . Seoul: IEEE, 2019: 5442-5451.
[23]   SEBASTIAN S, MARIUS P, VITTORIO B, et al Emerging mpeg standards for point cloud compression[J]. IEEE Journal on Emerging and Selected Topics in Circuits and Systems, 2019, 9 (1): 133- 148
doi: 10.1109/JETCAS.2018.2885981
[24]   ZHANG Z. Improved Adam optimizer for deep neural networks [C]// IEEE/ACM 26th International Symposium on Quality of Service . Banff: IEEE, 2018: 1-2.
[25]   ZHANG J, LIU G, DING D, et al. Transformer and upsampling-based point cloud compression [C]// ACM MM Workshops on Advances in Point Cloud Compression, Processing and Analysis. Lisbon: ACM, 2022: 33-39.
[26]   Google. Draco 3D data compression [EB/OL]. [2023-08-01]. https://github.com/google/draco.
[1] Fuxin MAO,Xu YANG,Jiaqiang CHENG,Tao PENG. Open-set 3D model retrieval algorithm based on multi-modal fusion[J]. Journal of ZheJiang University (Engineering Science), 2024, 58(1): 61-70.
[2] Jing-hui CHU,Li-dong SHI,Pei-guang JING,Wei LV. Context-aware knowledge distillation network for object detection[J]. Journal of ZheJiang University (Engineering Science), 2022, 56(3): 503-509.
[3] YANG Bo, FENG Hua jun, XU Zhi hai, LI Qi, CHNE Kuo. Automatical content-adapting stretched distortion correction algorithm for wide-angle images[J]. Journal of ZheJiang University (Engineering Science), 2016, 50(9): 1711-1717.
[4] PIAO Yun, DONG Li-da, DING Li. Novel localization method for mobile sensor node under
local NLOS environment
[J]. Journal of ZheJiang University (Engineering Science), 2011, 45(7): 1147-1153.
[5] CA Wei-Guang, TAO Qiang-Dong, LIU Feng, et al. Optimization of data forwarding based on early write-back strategy[J]. Journal of ZheJiang University (Engineering Science), 2010, 44(1): 75-80.