Please wait a minute...
浙江大学学报(工学版)  2024, Vol. 58 Issue (2): 279-287    DOI: 10.3785/j.issn.1008-973X.2024.02.006
计算机技术、通信技术     
基于场景流的可变速率动态点云压缩
江照意(),邹文钦,郑晟豪,宋超,杨柏林*()
浙江工商大学 计算机科学与技术学院,浙江 杭州 310018
Variable rate compression of point cloud based on scene flow
Zhaoyi JIANG(),Wenqin ZOU,Shenghao ZHENG,Chao SONG,Bailin YANG*()
School of Computer Science and Technology, Zhejiang Gongshang University, Hangzhou 310018, China
 全文: PDF(1060 KB)   HTML
摘要:

针对现有的动态点云压缩神经网络需要训练多个网络模型的问题,提出基于场景流的可变速率动态点云压缩网络框架. 网络以原始动态点云为输入, 利用场景流网络进行运动向量估计, 在压缩运动向量和残差的同时,引入通道增益模块对隐向量通道进行评估和缩放, 实现可变速率控制. 通过综合考虑运动向量损失和率失真损失, 设计新的联合训练损失函数, 用来端到端地训练整个网络框架. 为了解决动态点云数据集缺少真实运动信息标签的问题, 基于AMASS数据集制作带有运动向量标签的人体数据集,用于网络的训练. 实验结果显示, 与现有的基于深度学习动态点云压缩方法相比, 该方法的压缩比特率下降了几个数量级, 与静态压缩网络单独处理每帧的重构效果相比,该方法有5%~10%的提升.

关键词: 动态点云压缩可变速率联合损失函数场景流网络    
Abstract:

A variable-rate dynamic point cloud compression network framework based on scene flow was proposed in order to address the problem of training multiple network models for existing dynamic point cloud compression neural networks. The raw dynamic point cloud was taken as input, and the scene flow network was utilized to estimate motion vectors. A channel gain module was introduced to evaluate and scale the latent vector channels while compressing motion vectors and residuals, achieving variable-rate control. A new joint training loss function was designed to end-to-end train the entire network framework by comprehensively considering the motion vector loss and rate-distortion loss. A human body dataset with motion vector labels was created based on the AMASS dataset for network training in order to solve the problem of lack of real motion information labels in dynamic point cloud datasets. The experimental results show that the compression bit rate of the method decreases by several orders of magnitude compared with existing deep learning-based dynamic point cloud compression methods. The method has a 5%~10% improvement compared with the reconstruction effect of static compression networks processing each frame separately.

Key words: dynamic point cloud compression    variable rate    joint loss function    scene flow network
收稿日期: 2023-08-18 出版日期: 2024-01-23
CLC:  TP 37  
基金资助: 国家自然科学基金资助项目(62172366); 浙江省自然科学基金资助项目(LY21F020013, LY22F020013).
通讯作者: 杨柏林     E-mail: zyjiang@zjgsu.edu.cn;ybl@zjgsu.edu.cn
作者简介: 江照意(1975—),男,副教授,硕导,从事计算机图形学、数字几何处理、机器学习等研究. orcid.org/0000-0001-5347-7935. E-mail:zyjiang@zjgsu.edu.cn
服务  
把本文推荐给朋友
加入引用管理器
E-mail Alert
作者相关文章  
江照意
邹文钦
郑晟豪
宋超
杨柏林

引用本文:

江照意,邹文钦,郑晟豪,宋超,杨柏林. 基于场景流的可变速率动态点云压缩[J]. 浙江大学学报(工学版), 2024, 58(2): 279-287.

Zhaoyi JIANG,Wenqin ZOU,Shenghao ZHENG,Chao SONG,Bailin YANG. Variable rate compression of point cloud based on scene flow. Journal of ZheJiang University (Engineering Science), 2024, 58(2): 279-287.

链接本文:

https://www.zjujournals.com/eng/CN/10.3785/j.issn.1008-973X.2024.02.006        https://www.zjujournals.com/eng/CN/Y2024/V58/I2/279

图 1  动态点云压缩的整体框架图
图 2  场景流网络的结构图
图 3  运动补偿网络的示意图
图 4  压缩性能的对比
方法bpp/bitsPSNR/dBte/std/s
Draco3.0636.0
VPCC34.741.9630.60
DDPCC24.535.60.450.17
Static0.13635.30.260.09
本文方法0.12237.50.30.12
表 1  不同方法的压缩性能比较
图 5  不同压缩方法的可视化结果对比
数据集本文方法DDPCC
bpp/bitsPSNR/dBbpp/bitsPSNR/dB
Soldier0.12541.524.835.0
Loot0.11839.424.435.2
Longdress0.12235.424.335.7
Redandblack0.12036.124.435.6
表 2  在8i数据集上与D-DPCC的结果对比
图 6  损失函数消融的对比图
$ \beta $PSNR/dB
10028.8
135.7
0.137.3
0.0136.2
表 3  不同稀疏消融的实验结果
网络模型模型参数量/kB计算复杂度/MB
可变比特率41.272.8
非可变比特率206364
表 4  可变比特率和非可变比特率模型的复杂度比较
1 KATHARIYA B, LI L, LI Z, et al. Lossless dynamic point cloud geometry compression with inter compensation and traveling salesman prediction [C]// Data Compression Conference . Boston: IEEE, 2018: 414.
2 THANOU D, CHOU P A, FROSSARD P Graph-based compression of dynamic 3D point cloud sequences[J]. IEEE Transactions on Image Processing, 2016, 25 (4): 1765- 1778
doi: 10.1109/TIP.2016.2529506
3 DE QUEIROZ R L, CHOU P A Motion-compensated compression of dynamic voxelized point clouds[J]. IEEE Transactions on Image Processing, 2017, 26 (8): 3886- 3895
doi: 10.1109/TIP.2017.2707807
4 KAMMERL J, BLODOW N, RUSU R B, et al. Real-time compression of point cloud streams [C]// IEEE International Conference on Robotics and Automation . [S. l.]: IEEE, 2012: 778-785.
5 SUN X, MA H, SUN Y, et al A novel point cloud compression algorithm based on clustering[J]. IEEE Robotics and Automation Letters, 2019, 4 (2): 2132- 2139
doi: 10.1109/LRA.2019.2900747
6 BISWAS S, LIU J, WONG K, et al Muscle: multi sweep compression of lidar using deep entropy models[J]. Advances in Neural Information Processing Systems, 2020, 33 (2): 22170- 22181
7 YU S, SUN S, YAN W, et al A method based on curvature and hierarchical strategy for dynamic point cloud compression in augmented and virtual reality system[J]. Sensors, 2022, 22 (3): 1262
doi: 10.3390/s22031262
8 FAN T, GAO L, XU Y, et al. D-DPCC: deep dynamic point cloud compression via 3D motion prediction [C]// Proceedings of the 31st International Joint Conference on Artificial Intelligence . Vienna: [s. n.], 2022: 898-904.
9 GEORGE T, SEAN M, SUNG J, et al. Variable rate image compression with recurrent neural networks [C]// International Conference on Learning Representations . Puerto Rico: [s. n.], 2016.
10 CHEN T, MA Z. Variable bitrate image compression with quality scaling factors [C]// IEEE International Conference on Acoustics, Speech and Signal Processing . Barcelona: IEEE, 2020: 2163-2167.
11 YANG F, HERRANZ L, WEIJER J, et al. Variable rate deep image compression with modulated autoencoder [J]. IEEE Signal Processing Letters , 2020, 27: 331-335.
12 CUI Z, WANG J, GAO S, et al. Asymmetric gained deep image compression with continuous rate adaptation [C]// Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition . Baltimore: IEEE, 2021: 10532-10541.
13 MUZADDID M A, BEKSI W J. Variable rate compression for raw 3D point clouds [C]// International Conference on Robotics and Automation . Philadelphia: [s. n.], 2022: 8748-8755.
14 QI C R, YI L, SU H, et al. Pointnet++: deep hierarchical feature learning on point sets in a metric space [C]// Advances in Neural Information Processing Systems . Long Beach: [s. n.], 2017: 30.
15 WU W, WANG Z Y, LI Z, et al. Pointpwc-net: cost volume on point clouds for (self-) supervised scene flow estimation [C]// 16th European Conference on Computer Vision. Glasgow: Springer, 2020.
16 WU W, QI Z, FUXIN L. Pointconv: deep convolutional networks on 3d point clouds [C]// Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition . Seattle: IEEE, 2019: 9621-9630.
17 ENGEL N, BELAGIANNIS V, DIETMAYER K Point transformer[J]. IEEE Access, 2021, 9 (1): 134826- 134840
18 HE Y, REN X, TANG D, et al. Density-preserving deep point cloud compression [C] // Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition . New Orleans: IEEE, 2022: 2333-2342.
19 LU G, OUYANG W, XU D, et al. Dvc: an end-to-end deep video compression framework [C]// Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition . Seattle: IEEE, 2019: 11006-11015.
20 WU Y, HE K. Group normalization [C]// Proceedings of the European Conference on Computer Vision . Munich: [s. n.], 2018: 3-19.
21 HUANG L, WANG S, WONG K, et al. Octsqueeze: octree-structured entropy model for lidar compression [C] // Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition . Seattle: IEEE, 2020: 1313-1323.
22 MAHMOOD N, GHORBANI N, TROJE N F, et al. AMASS: archive of motion capture as surface shapes [C]// Proceedings of the IEEE/CVF International Conference on Computer Vision . Seoul: IEEE, 2019: 5442-5451.
23 SEBASTIAN S, MARIUS P, VITTORIO B, et al Emerging mpeg standards for point cloud compression[J]. IEEE Journal on Emerging and Selected Topics in Circuits and Systems, 2019, 9 (1): 133- 148
doi: 10.1109/JETCAS.2018.2885981
24 ZHANG Z. Improved Adam optimizer for deep neural networks [C]// IEEE/ACM 26th International Symposium on Quality of Service . Banff: IEEE, 2018: 1-2.
25 ZHANG J, LIU G, DING D, et al. Transformer and upsampling-based point cloud compression [C]// ACM MM Workshops on Advances in Point Cloud Compression, Processing and Analysis. Lisbon: ACM, 2022: 33-39.
26 Google. Draco 3D data compression [EB/OL]. [2023-08-01]. https://github.com/google/draco.
[1] 毛福新,杨旭,程嘉强,彭涛. 基于多模态融合的开放域三维模型检索算法[J]. 浙江大学学报(工学版), 2024, 58(1): 61-70.
[2] 褚晶辉,史李栋,井佩光,吕卫. 适用于目标检测的上下文感知知识蒸馏网络[J]. 浙江大学学报(工学版), 2022, 56(3): 503-509.
[3] 杨波,冯华君,徐之海,李奇,陈阔. 适应内容的广角图像拉伸畸变自动校正算法[J]. 浙江大学学报(工学版), 2016, 50(9): 1711-1717.
[4] 朴云,董利达,丁力. 能够克服局部NLOS影响的自主移动节点定位方法[J]. J4, 2011, 45(7): 1147-1153.
[5] 蔡卫光, 姚庆栋, 刘鹏, 等. 基于提前写回策略的数据转发优化方法[J]. J4, 2010, 44(1): 75-80.