Please wait a minute...
Journal of ZheJiang University (Engineering Science)  2024, Vol. 58 Issue (5): 941-950    DOI: 10.3785/j.issn.1008-973X.2024.05.007
    
Airline baggage feature perception based on improved graph convolutional neural network
Zhiwei XING(),Shujie ZHU,Biao LI
College of Electronic Information and Automation, Civil Aviation University of China, Tianjin 300300, China
Download: HTML     PDF(2439KB) HTML
Export: BibTeX | EndNote (RIS)      

Abstract  

An airline baggage feature perception network model was designed with PointNet++ as the benchmark and incorporating graph convolutional neural network and self-attention mechanism aiming at the problem that the configuration feature perception capability of airline baggage was inadequate under the demand of automatic baggage stacking handling. The local spatial attention module was introduced in the feature abstraction layer of the backbone network to extract associated spatial structure features of neighboring points in aviation baggage point cloud in order to perceive the intrinsic connection of its region feature space. Correlation between local features of airline baggage point cloud was learned through the global feature aggregation module to adaptively aggregate local features so as to form global contextual information. The recycling maxpooling layer was applied to recycle features from some discard points in the feature reduction process and collect baggage information at multiple levels, reducing information redundancy while retaining local and global feature activations with stark intensity. The experimental results showed that the average and overall accuracy of airline baggage classification were 94.68% and 96.32%, which were 6.53% and 5.07% improved over PointNet++, respectively. The airline baggage feature perception performance of the network model is better than other existing intelligent algorithms, which can provide accurate, reliable and effective input for airline baggage stacking space optimization and control.



Key wordsair transport      baggage feature perception      three-dimensional point cloud      graph convolutional neural network      self-attention mechanism     
Received: 04 May 2023      Published: 26 April 2024
CLC:  TP 391  
Fund:  国家重点研发计划资助项目(2018YFB1601200);中国民航大学研究生科研创新项目(2022YJS023).
Cite this article:

Zhiwei XING,Shujie ZHU,Biao LI. Airline baggage feature perception based on improved graph convolutional neural network. Journal of ZheJiang University (Engineering Science), 2024, 58(5): 941-950.

URL:

https://www.zjujournals.com/eng/10.3785/j.issn.1008-973X.2024.05.007     OR     https://www.zjujournals.com/eng/Y2024/V58/I5/941


基于改进图卷积神经网络的航空行李特征感知

针对航空行李自动化码放处理需求下构型特征感知能力不足的问题,设计以PointNet++为基准,融入图卷积神经网络和自注意力机制的航空行李特征感知网络模型. 在骨干网络的特征抽象层中引入局部空间注意力模块,提取航空行李点云中相邻点的关联空间结构特征,感知区域特征空间的内在联系. 通过全局特征聚合模块学习行李点云局部特征间的相关性,自适应聚合航空行李局部特征,形成点云全局上下文信息. 利用循环最大池化层回收特征降维中丢弃点的特征,在多个层次上收集航空行李的特征信息,在减少信息冗余的同时,保留强度鲜明的局部、全局特征激活. 实验结果表明,航空行李分类的平均精度和整体精度分别为94.68%和96.32%,比PointNet++分别提高了6.53%和5.07%. 该网络模型的航空行李特征感知性能优于现有的其他智能算法,能够为航空行李码放空间优化及控制提供准确、可靠、有效的输入.


关键词: 航空运输,  行李特征感知,  三维点云,  图卷积神经网络,  自注意力机制 
Fig.1 Experiment system of airline baggage feature collection
Fig.2 Operation status of airline baggage feature collection system
Fig.3 Classification of airline baggage
类别表面材质形状特点
方箱纸板、塑料规则长方体
硬箱金属、塑料光滑曲面、类长方体
软箱织物、皮革凹凸曲面、类长方体
硬包金属、塑料光滑曲面、类椭球体
软包织物、皮革凹凸曲面、类椭球体
异形塑料保护套球形、柱形不规则物体
Tab.1 Basis for classification of airline baggage
Fig.4 Structure of GACP network
Fig.5 Structure of local spatial attention module
Fig.6 Structure of global feature aggregation module
Fig.7 Schematic diagram of recycling maxpooling
软硬件名称软硬件配置参数数值
Operation systemWindows 10Batch size8
CPUIntel Xeon E5-2680 v4Input point1 024
GPUNVIDIA RTX 3080 TiEpoch200
RAM64 GBLearning rate0.001
Python+Pytorch3.7. 0+1.7.1Gamma0.7
CUDA+CUDNN11.0+8.1.0OptimizerAdam
Tab.2 Experimental configuration of airline baggage feature perception
Fig.8 Production method of extended dataset
算法方箱硬箱软箱硬包软包异形mAcc/%OA/%t/s
PointNet86.9286.1487.4780.7685.3389.4586.0189.67124.5
PointNet++88.6188.2789.9482.7588.0391.3188.1591.25267.3
DGCNN90.5692.7394.4384.6787.2694.6890.7293.06192.4
PointVGG91.2393.5294.4887.5489.7394.8191.8994.18216.7
GACP(本文方法)93.7295.6496.3991.1692.5398.6694.6896.32283.2
Tab.3 Comparision of classification accuracy for different models
Fig.9 Classification accuracy curves of different models
组数LSAGFARMPmAcc/%OA/%
A×××88.1591.25
B××90.0792.61
C×93.4595.73
D94.6896.32
Tab.4 Ablation experiments of different modules
Fig.10 Perceptual results of PointNet++
Fig.11 Perceptual results of GACP
Fig.12 Anomalies in luggage point cloud data
Fig.13 Comparison of robustness experiment results
[1]   DAVID L. Baggage IT insights 2022 [R]. Brussels: SITA, 2022.
[2]   GAO Q, LIANG P Airline baggage appearance transportability detection based on a novel dataset and sequential hierarchical sampling CNN model[J]. IEEE Access, 2021, 9 (2): 41833- 41843
[3]   PENG C, GAO Q, LUO Q. Method on 3d reconstruction of airline baggage based on active laser projection of improved encoding [C]// IEEE CSAA Guidance, Navigation and Control Conference . Xiamen: IEEE, 2018: 1-7.
[4]   TANG C, ZHANG X, YU X, et al Target detection and classification based on LiDAR[J]. American Academic Scientific Research Journal for Engineering, Technology, and Sciences, 2018, 49 (1): 28- 39
[5]   MAZZEO P L, LIBETTA C, SPAGNOLO P, et al A siamese neural network for non-invasive baggage re-identification[J]. Journal of Imaging, 2020, 6 (11): 126
doi: 10.3390/jimaging6110126
[6]   CHEN R, ZHANG H, LI C, et al. LSDNN: local-salient deep neural network for baggage re-identification with material discerning [C]// Chinese Automation Congress . Shanghai: IEEE, 2020: 6344-6349.
[7]   ZHAO Q, MA H, LU R, et al. MVAD-Net: learning view-aware and domain-invariant representation for baggage re-identification [C]// Pattern Recognition and Computer Vision . Beijing: Springer, 2021: 142-153.
[8]   WU H, LUO Z, CAO D, et al. Attention and multi-grained feature learning for baggage re-identification[C]// Computer Supported Cooperative Work and Social Computing . Xiangtan: Springer, 2022: 460-472.
[9]   YANG H, CHU X, ZHANG L, et al QuadNet: quadruplet loss for multi-view learning in baggage re-identification[J]. Pattern Recognition, 2022, 126 (5): 108- 109
[10]   行李处理系统 带式输送机: MH/T 6123.1—2021 [S]. 北京: 中国民用航空局, 2021.
[11]   ZHANG Z, LI D, WU J, et al. MVB: a large-scale dataset for baggage re-identification and merged Siamese networks [C]// Pattern Recognition and Computer Vision . Xi’an: Springer, 2019: 84-96.
[12]   CAZAN C. Identification of surface material of baggage for self-service bag drop system [D]. Hong Kong: The University of Hong Kong, 2020.
[13]   QI C R, YI L, SU H, et al PointNet++: deep hierarchical feature learning on point sets in a metric space[J]. Advances in Neural Information Processing Systems, 2017, 30 (5): 78- 92
[14]   GAO Z, YAN J, ZHAI G, et al. Learning local neighboring structure for robust 3d shape representation [C]// Proceedings of the AAAI Conference on Artificial Intelligence , 2021, 35(2): 1397-1405.
[15]   VASWANI A, SHAZEER N, PARMAR N, et al Attention is all you need[J]. Advances in Neural Information Processing Systems, 2017, 30 (8): 5998- 6008
[16]   DU Z, YE H, CAO F A novel local-global graph convolutional method for point cloud semantic segmentation[J]. IEEE Transactions on Neural Networks and Learning Systems, 2022, 15 (6): 1- 15
[17]   NIU Z, ZHONG G, YUE G, et al Recurrent attention unit: a new gated recurrent unit for long-term memory of important parts in sequential data[J]. Neurocomputing, 2022, 517 (4): 1- 9
[18]   CHEN J, KAKILLIOGLU B, REN H, et al. Why discard if you can recycle?: a recycling max pooling module for 3d point cloud analysis [C]// IEEE/CVF Conference on Computer Vision and Pattern Recognition . New Orleans: IEEE, 2022: 559-567.
[19]   LI R, LI X, HENG P A, et al. PointAugment: an auto-augmentation framework for point cloud classification [J]. IEEE/CVF Conference on Computer Vision and Pattern Recognition . Seattle: IEEE, 2020: 6378-6387.
[20]   CHARLES R Q, HAO S, MO K C, et al. PointNet: deep learning on point sets for 3d classification and segmentation [C]// IEEE Conference on Computer Vision and Pattern Recognition . Honolulu: IEEE, 2017: 77-85.
[21]   WANG Y, SUN Y, LIU Z, et al Dynamic graph CNN for learning on point clouds[J]. ACM Transactions on Graphics, 2019, 38 (5): 146- 152
[1] Chuan-hua ZHOU,Li-chun CAO,Jia-yi ZHOU,Feng ZHAN. Identification of critical nodes in temporal networks based on graph convolution union computing[J]. Journal of ZheJiang University (Engineering Science), 2023, 57(5): 930-938.
[2] Chao LIU,Bing KONG,Guo-wang DU,Li-hua ZHOU,Hong-mei CHEN,Chong-ming BAO. Deep clustering via high-order mutual information maximization and pseudo-label guidance[J]. Journal of ZheJiang University (Engineering Science), 2023, 57(2): 299-309.
[3] Tian-le YANG,Ling-xia LI,Wei ZHANG. Dual-branch crowd counting algorithm based on self-attention mechanism[J]. Journal of ZheJiang University (Engineering Science), 2023, 57(10): 1955-1965.
[4] Xiao-chen JU,Xin-xin ZHAO,Sheng-sheng QIAN. Self-attention mechanism based bridge bolt detection algorithm[J]. Journal of ZheJiang University (Engineering Science), 2022, 56(5): 901-908.
[5] Ying-li LIU,Rui-gang WU,Chang-hui YAO,Tao SHEN. Construction method of extraction dataset of Al-Si alloy entity relationship[J]. Journal of ZheJiang University (Engineering Science), 2022, 56(2): 245-253.
[6] Nan-jing YU,Xiao-biao FAN,Tian-min DENG,Guo-tao MAO. Ship detection algorithm in complex backgrounds via multi-head self-attention[J]. Journal of ZheJiang University (Engineering Science), 2022, 56(12): 2392-2402.