Please wait a minute...
浙江大学学报(工学版)
Journal of ZheJiang University (Engineering Science)  2021, Vol. 55 Issue (12): 2342-2351    DOI: 10.3785/j.issn.1008-973X.2021.12.014
    
Deep 3D point cloud classification network based on competitive attention fusion
Han-juan CHEN1,2(),Fei-peng DA1,2,3,*(),Shao-yan GAI1,2
1. School of Automation, Southeast University, Nanjing 210096, China
2. Key Laboratory of Measurement and Control of Complex Systems of Engineering, Ministry of Education, Southeast University, Nanjing 210096, China
3. Shenzhen Research Institute, Southeast University, Shenzhen 518063, China
Download: HTML     PDF(1181KB) HTML
Export: BibTeX | EndNote (RIS)      

Abstract  

A competitive attention fusion block that can be transferred to different classification networks was proposed, in order to improve the 3D point cloud deep network classification model’s ability to extract and express global features, and enhance the model’s robustness to noise interference. The global representation of multi-hierarchical features and the internal similarity of intermediate features were learned. The weights of the intermediate feature channels were re-allocated. The proposed block was embedded in the benchmark networks Pointnet++ and PointASNL for experiments. Results show that the proposed block is independent and transferable, focusing on the core and backbone features that are more conducive to 3D point cloud shape classification. Compared with the benchmark network, the proposed block enhances the model’s anti-interference ability to point cloud disturbance noise, outlier noise and random noise without decreasing the classification accuracy. The proposed method achieves the accuracy was 93.2%, 92.9%, 85.7%, 78.2%, 63.5% in the case of the number of random noises was 0, 10, 50, 100, 200, respectively. Compared with the traditional filtering method, end-to-end learning reduces the pre-processing steps and manual intervention process, and has better anti-noise performance.

Key words： point cloud object classification      3D point cloud      deep learning      neural network      attention mechanism      competitive fusion     
Received: 04 January 2021      Published: 31 December 2021
CLC:  TP 181  
Fund:  国家自然科学基金资助项目(51475092)；江苏省前沿引领技术基础研究专项资助项目(BK20192004C)；江苏省自然基金资助项目(BK20181269)；深圳市科技创新委员会资助项目(JCYJ20180306174455080)
Corresponding Authors: Fei-peng DA     E-mail: 220181486@seu.edu.cn;dafp@seu.edu.cn
Service
E-mail this article
Add to my bookshelf
Add to citation manager
E-mail Alert
RSS
Articles by authors
Han-juan CHEN
Fei-peng DA
Shao-yan GAI
Cite this article:

Han-juan CHEN,Fei-peng DA,Shao-yan GAI. Deep 3D point cloud classification network based on competitive attention fusion. Journal of ZheJiang University (Engineering Science), 2021, 55(12): 2342-2351.

URL:

https://www.zjujournals.com/eng/10.3785/j.issn.1008-973X.2021.12.014     OR     https://www.zjujournals.com/eng/Y2021/V55/I12/2342


基于竞争注意力融合的深度三维点云分类网络

为了提高三维点云深度网络分类模型对全局特征的提取与表达能力，增强模型对噪声干扰的鲁棒性，提出可迁移应用于不同分类网络的竞争性注意力融合模块，学习多层级特征的全局表征和中间特征内在相似度，对中间特征通道权值重分配. 在基准网络Pointnet++和PointASNL中嵌入所提模块并进行实验，结果显示：所提模块具有独立性和可迁移性，聚焦更利于三维点云形状分类的核心骨干特征. 与基准网络相比，所提模块在保持分类精度稳定不下降的情况下，模型对点云扰动噪声、离群点噪声和随机噪声的抗干扰能力增强，在随机噪声数分别为0、10、50、100、200的情况下，准确度分别达到93.2%、92.9%、85.7%、78.2%、63.5%. 与传统滤波方法相比，端到端的学习减少预处理步骤和人工干预过程，同时具有更优的抗噪性能.


关键词： 点云物体分类,  三维点云,  深度学习,  神经网络,  注意力机制,  竞争性融合 
Fig.1 Diagram of competitive attention fusion block
Fig.2 Structure of 2D and 3D squeeze-excitation block
Fig.3 Structure of feature inner connection self-attention block
Fig.4 3D point cloud classification network framework based on CAF block
方法 输入 Nin/103 Acc/%
PointNet[4] Pnt 1 89.2
SO-Net[8] Pnt,Noml 2 90.9
PointNet++[5] Pnt,Noml 5 91.9
PointCNN[10] Pnt 1 92.2
Point2Sequence[12] Pnt 1 92.6
A-CNN[13] Pnt,Noml 1 92.6
PointASNL[21] Pnt 1 92.9(92.85)
PointASNL[21] Pnt,Noml 1 93.2(93.15)
本研究 Pnt 1 92.9(92.88)
本研究 Pnt,Noml 1 93.2(93.19)
Tab.1 Average classification accuracy on ModelNet40 dataset
Fig.5 Anti-jamming performance of CAF block against different noise types
Fig.6 Influence of CAF block on model robustness
Fig.7 Comparison of anti-jamming performance between CAF block and traditional filtering
Fig.8 Influence of independent sub blocks on model robustness
n Acc
Base Base+MFSE Base+FICSA Base+CAF
%
0 93.2 93.2 92.6 93.2
1 92.1 91.7 91.8 92.3
10 88.3 86.5 89.8 89.1
50 78.1 74.0 80.1 81.9
100 71.1 60.6 72.5 74.8
Tab.2 Influence of independent sub blocks on anti-jamming performance of model
方法 mIoU IoU
areo bag cap car chair ear phone guitar knife lamp laptop motor mug pistol rocket skate board table
PointNet[4] 83.7 83.4 78.7 82.5 74.9 89.6 73.0 91.5 85.9 80.8 95.3 65.2 93.0 81.2 57.9 72.8 80.6
SO-Net[8] 84.9 82.8 77.8 88.0 77.3 90.6 73.5 90.7 83.9 82.8 94.8 69.1 94.2 80.9 53.1 72.9 83.0
PointNet++[5] 85.1 82.4 79.0 87.7 77.3 90.8 71.8 91.0 85.9 83.7 95.3 71.6 94.1 81.3 58.7 76.4 82.6
P2Sequence[12] 85.2 82.6 81.8 87.5 77.3 90.8 77.1 91.1 86.9 83.9 95.7 70.8 94.6 79.3 58.1 75.2 82.8
PointCNN[10] 86.1 84.1 86.5 86.0 80.8 90.6 79.7 92.3 88.4 85.3 96.1 77.2 95.2 84.2 64.2 80.0 83.0
PointASNL[21] 86.1 84.1 84.7 87.9 79.7 92.2 73.7 91.0 87.2 84.2 95.8 74.4 95.2 81.0 63.0 76.3 83.2
本研究 85.9 84.2 83.2 87.4 79.2 91.9 74.3 91.5 86.4 84.3 95.7 73.7 95.4 82.6 62.4 75.0 82.7
Tab.3 Part segmentation performance on ShapeNetPart dataset %
方法 OA mAcc mIoU IoU
ceiling floor wall beam column window door table chair sofa bookcase board clutter
PointNet[4] 78.5 66.2 47.6 88.0 88.7 69.3 42.4 23.1 47.5 51.6 42.0 54.1 38.2 9.6 29.4 35.2
A-CNN[13] 87.3 ? 62.9 92.4 96.4 79.2 59.5 34.2 56.3 65.0 66.5 78.0 28.5 56.9 48.0 56.8
PointCNN[10] 88.1 75.6 65.4 94.8 97.3 75.8 63.3 51.7 58.4 57.2 71.6 69.1 39.1 61.2 52.2 58.6
PointWeb[14] 87.3 76.2 66.7 93.5 94.2 80.8 52.4 41.3 64.9 68.1 71.4 67.1 50.3 62.7 62.2 58.5
PointASNL[21] 88.8 79.0 68.7 95.3 97.9 81.9 47.0 48.0 67.3 70.5 71.3 77.8 50.7 60.4 63.0 62.8
本研究 88.2 78.7 68.3 95.1 97.3 81.2 47.4 45.8 67.0 69.1 72.1 77.5 50.6 60.8 62.4 61.6
Tab.4 Semantic segmentation performance on S3DIS dataset with 6-fold cross validation %
[1]   BU S, LIU Z, HAN J, et al Learning high-level feature by deep belief networks for 3-D model retrieval and recognition[J]. IEEE Transactions on Multimedia, 2014, 16 (8): 2154- 2167
doi: 10.1109/TMM.2014.2351788
[2]   SU H, MAJI S, KALOGERAKIS E, et al. Multi-view convolutional neural networks for 3D shape recognition [C]// 2015 IEEE International Conference on Computer Vision. Santiago: IEEE, 2015: 945-953.
[3]   WU Z, SONG S, KHOSLA A, et al. 3D shapeNets: a deep representation for volumetric shapes [C]// 2015 IEEE Conference on Computer Vision and Pattern Recognition. Boston: IEEE, 2015: 1912-1920.
[4]   QI C R, SU H, MO K, et al. PointNet: deep learning on point sets for 3D classification and segmentation [C]// 2017 IEEE Conference on Computer Vision and Pattern Recognition. Honolulu: IEEE, 2017: 77-85.
[5]   QI C R, YI L, SU H, et al. PointNet++: deep hierarchical feature learning on point sets in a metric space [C]// Advances in Neural Information Processing Systems. Long Beach: MIT Press, 2017: 5099-5108.
[6]   GUERRERO P, KLEIMAN Y, OVSJANIKOV M, et al PCPNET learning local shape properties from raw point clouds[J]. Computer Graphics Forum, 2018, 37 (2): 75- 85
doi: 10.1111/cgf.13343
[7]   SHEN Y, FENG C, YANG Y, et al. Mining point cloud local structures by kernel correlation and graph pooling [C]// 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Salt Lake City: IEEE, 2018: 4548-4557.
[8]   LI J, CHEN B M, LEE G H. SO-Net: self-organizing network for point cloud analysis [C]// 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Salt Lake City: IEEE, 2018: 9397-9406.
[9]   QI C R, LIU W, WU C, et al. Frustum PointNets for 3D object detection from RGB-D data [C]// 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Salt Lake City: IEEE, 2018: 918-927.
[10]   LI Y, BU R, SUN M, et al. PointCNN: convolution on Χ-transformed points [C]// Advances in Neural Information Processing Systems. Montreal: MIT Press, 2018: 828-838.
[11]   LIU Y, FAN B, MENG G, et al. DensePoint: learning densely contextual representation for efficient point cloud processing [C]// 2019 IEEE/CVF International Conference on Computer Vision. Seoul: IEEE, 2019: 5238-5247.
[12]   LIU X, HAN Z, LIU Y S, et al. Point2Sequence: learning the shape representation of 3D point clouds with an attention-based sequence to sequence network [C]// Proceedings of the AAAI conference on Artificial Intelligence. Honolulu: AAAI, 2019: 8778-8785.
[13]   KOMARICHEV A, ZHONG Z, HUA J. A-CNN: annularly convolutional neural networks on point clouds [C]// 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Long Beach: IEEE, 2019: 7413-7422.
[14]   ZHAO H, JIANG L, FU C W, et al. PointWeb: enhancing local neighborhood features for point cloud processing [C]// 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Long Beach: IEEE, 2019: 5560-5568.
[15]   WANG C, SAMARI B, SIDDIQI K. Local spectral graph convolution for point set feature learning [C]// 15th European Conference on Computer Vision. Munich: Springer, 2018: 56-71.
[16]   TE G, HU W, GUO Z, et al. RGCNN: regularized graph CNN for point cloud segmentation [C]// Proceedings of the 26th ACM international conference on Multimedia. Seoul: ACM, 2018: 746-754.
[17]   LANDRIEU L, SIMONOVSKY M. Large-scale point cloud semantic segmentation with superpoint graphs [C]// 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Salt Lake City: IEEE, 2018: 4558-4567.
[18]   VASWANI A, SHAZEER N, PARMAR N, et al. Attention is all you need [C]// Advances in Neural Information Processing Systems. Long Beach: MIT Press, 2017: 5998-6008.
[19]   HU J, SHEN L, SUN G. Squeeze-and-excitation networks [C]// 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Salt Lake City: IEEE, 2018: 7132-7141.
[20]   YOU H, FENG Y, JI R, et al. PVNet: a joint convolutional network of point cloud and multi-view for 3D shape recognition [C]// Proceedings of the 26th ACM international conference on Multimedia. Seoul: ACM, 2018: 1310-1318.
[21]   YAN X, ZHENG C, LI Z, et al. PointASNL: robust point clouds processing using nonlocal neural networks with adaptive sampling [C]// 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Seattle: IEEE, 2020: 5588-5597.
[22]   HU Y, WEN G, LUO M, et al. Competitive inner-imaging squeeze and excitation for residual network [EB/OL]. (2018-12-23)[2020-12-29]. https://arxiv.org/abs/1807.08920.
[23]   YI L, KIM V G, CEYLAN D, et al A scalable active framework for region annotation in 3D shape collections[J]. ACM Transactions on Graphics, 2016, 35 (6): 210
[1] Xiao-chen JU,Xin-xin ZHAO,Sheng-sheng QIAN. Self-attention mechanism based bridge bolt detection algorithm[J]. Journal of ZheJiang University (Engineering Science), 2022, 56(5): 901-908.
[2] Li HE,Shan-min PANG. Face reconstruction from voice based on age-supervised learning and face prior information[J]. Journal of ZheJiang University (Engineering Science), 2022, 56(5): 1006-1016.
[3] You-wei WANG,Shuang TONG,Li-zhou FENG,Jian-ming ZHU,Yang LI,Fu CHEN. New inductive microblog rumor detection method based on graph convolutional network[J]. Journal of ZheJiang University (Engineering Science), 2022, 56(5): 956-966.
[4] Xue-qin ZHANG,Tian-ren LI. Breast cancer pathological image classification based on Cycle-GAN and improved DPN network[J]. Journal of ZheJiang University (Engineering Science), 2022, 56(4): 727-735.
[5] Yun-hao WANG,Ming-hui SUN,Yi XIN,Bo-xuan ZHANG. Robot tactile recognition system based on piezoelectric film sensor[J]. Journal of ZheJiang University (Engineering Science), 2022, 56(4): 702-710.
[6] Ze-kang WU,Shan ZHAO,Hong-wei LI,Yi-rui JIANG. Spatial global context information network for semantic segmentation of remote sensing image[J]. Journal of ZheJiang University (Engineering Science), 2022, 56(4): 795-802.
[7] Meng XU,Dan WANG,Zhi-yuan LI,Yuan-fang CHEN. IncepA-EEGNet: P300 signal detection method based on fusion of Inception network and attention mechanism[J]. Journal of ZheJiang University (Engineering Science), 2022, 56(4): 745-753, 782.
[8] Chang-yuan LIU,Xian-ping HE,Xiao-jun BI. Efficient network vehicle recognition combined with attention mechanism[J]. Journal of ZheJiang University (Engineering Science), 2022, 56(4): 775-782.
[9] Yang-zhao CHEN,Wei-na YUAN. Deep learning aided multi-user detection for up-link grant-free NOMA[J]. Journal of ZheJiang University (Engineering Science), 2022, 56(4): 816-822.
[10] Ke-wen ZHANG,Bai-song PAN. Control design of spacecraft autonomous rendezvous using nonlinear models with uncertainty[J]. Journal of ZheJiang University (Engineering Science), 2022, 56(4): 833-842.
[11] Qiao-hong CHEN,Hao-lei PEI,Qi SUN. Image caption based on relational reasoning and context gate mechanism[J]. Journal of ZheJiang University (Engineering Science), 2022, 56(3): 542-549.
[12] Jing-hui CHU,Li-dong SHI,Pei-guang JING,Wei LV. Context-aware knowledge distillation network for object detection[J]. Journal of ZheJiang University (Engineering Science), 2022, 56(3): 503-509.
[13] Ruo-ran CHENG,Xiao-li ZHAO,Hao-jun ZHOU,Han-chen YE. Review of Chinese font style transfer research based on deep learning[J]. Journal of ZheJiang University (Engineering Science), 2022, 56(3): 510-519, 530.
[14] Ting WANG,Xiao-fei ZHU,Gu TANG. Knowledge-enhanced graph convolutional neural networks for text classification[J]. Journal of ZheJiang University (Engineering Science), 2022, 56(2): 322-328.
[15] Pei-zhi WEN,Jun-mou CHEN,Yan-nan XIAO,Ya-yuan WEN,Wen-ming HUANG. Underwater image enhancement algorithm based on GAN and multi-level wavelet CNN[J]. Journal of ZheJiang University (Engineering Science), 2022, 56(2): 213-224.