Please wait a minute...
浙江大学学报(工学版)
Journal of ZheJiang University (Engineering Science)  2021, Vol. 55 Issue (4): 626-638    DOI: 10.3785/j.issn.1008-973X.2021.04.004
    
End context-adaptative deep sensing model with edge-end collaboration
Hong-li WANG(),Bin GUO*(),Si-cong LIU,Jia-qi LIU,Yun-gang WU,Zhi-wen YU
College of Computer Science, Northwestern Polytechnical University, Xi’an 710072, China
Download: HTML     PDF(1756KB) HTML
Export: BibTeX | EndNote (RIS)      

Abstract  

The end context adaptative of deep models with edge-end collaboration was analyzed. The partition and alternating direction method of multiplier method (X-ADMM) was proposed. The model compression was employed to simplify the model structure, and the model was partitioned at layer granularity to find the best partition point. The model can collaborate with edge-end devices to improve model operation efficiency. The graph based adaptive DNN surgery algorithm (GADS) was proposed in order to realize the dynamic adaptation of model partition. The model will preferentially search for the partition point that best meets resource constraints among surrounding partition states to achieve rapid adaptation when the running context (e.g., storage, power, bandwidth) of the model changes. The experimental results showed that the model realized the adaptive tuning of model partition point in an average of 0.1 ms. The total running latency was reduced by 56.65% at the highest with no more than 2.5% accuracy loss.

Key words： deep learning      edge intelligence      model compression      model partition      adaptive perception     
Received: 26 January 2021      Published: 07 May 2021
CLC:  TP 391  
Fund:  国家重点研发计划资助项目（2019YFB1703901）；国家自然科学基金资助项目（61772428，61725205）
Corresponding Authors: Bin GUO     E-mail: wanghongli@mail.nwpu.edu.cn;guob@nwpu.edu.cn
Service
E-mail this article
Add to my bookshelf
Add to citation manager
E-mail Alert
RSS
Articles by authors
Hong-li WANG
Bin GUO
Si-cong LIU
Jia-qi LIU
Yun-gang WU
Zhi-wen YU
Cite this article:

Hong-li WANG,Bin GUO,Si-cong LIU,Jia-qi LIU,Yun-gang WU,Zhi-wen YU. End context-adaptative deep sensing model with edge-end collaboration. Journal of ZheJiang University (Engineering Science), 2021, 55(4): 626-638.

URL:

http://www.zjujournals.com/eng/10.3785/j.issn.1008-973X.2021.04.004     OR     http://www.zjujournals.com/eng/Y2021/V55/I4/626


边端融合的终端情境自适应深度感知模型

研究边端融合的深度模型终端情境自适应问题. 提出边端融合增强的模型压缩方法（X-ADMM），利用模型压缩技术简化模型结构，以层为粒度寻找模型最佳分割点，协同边端设备提高运行效率. 为了实现模型分割的动态自适应，提出基于图的自适应深度模型手术刀算法（GADS）. 当模型运行情境（如存储、电量、带宽等）发生变化时，优先在邻近分割状态中快速搜索最能满足资源约束的分割点，实现快速自适应调整. 实验结果表明，该模型平均在0.1 ms内实现了模型分割点的自适应调优，在保证模型精度下降不超过2.5%的情况下，运行总时延最高下降了56.65%.


关键词： 深度学习,  边缘智能,  模型压缩,  模型分割,  自适应感知 
Fig.1 Generic flow of model compression
Fig.2 Illustration of model compression framework
Fig.3 Neighbor partition effect of edge-end model
Fig.4 Algorithm of graph based adaptive DNN surgery
Fig.5 Inference latency for AlexNet with different bandwidths
Fig.6 Output data size of each layer in AlexNet
Fig.7 Inference latency and output data size under different partitions of CNNs
Fig.8 Construction of model partition state graph on two devices
Fig.9 Part of model partition state graph on two devices
Fig.10 Example of constructing KD（K-dimensional）tree
Fig.11 Drop ratio of inference latency and accuracy of models after compression
网络 A0 /% T0 /ms γ TRAP-ADMM[19] /ms ARAP-ADMM /% TX-ADMM /ms AX-ADMM /%
Alexnet 85.82 46.8 14.7 38.78 84.21 36.3 84.17
GoogleNet 87.48 943.6 15.6 883.95 84.91 532.6 84.60
Resnet-18 91.60 285.5 15.4 267.70 90.01 213.5 89.80
VGG-16 91.66 203.7 16.0 186.90 89.59 88.3 89.20
MobileNet 89.60 219.2 2.0 207.89 87.96 179.2 87.90
MobileNet 89.60 219.2 4.0 196.69 80.60 160.3 80.83
ShuffleNet 88.14 202.8 4.0 183.79 84.25 153.4 84.19
Tab.1 Inference latency and accuracy of typical deep learning models with different approaches
Fig.12 Comparison of KNN（K-nearest neighbor）and KD trees search approaches
算法 ${L_{ p{\rm{a} } } }$ ${S_{\rm{t}}}$ /ms
Neurosurgeon 0.0822 0.4412
GADS-KNN 0.0452 0.9927
GADS-KD树（无s） 0.0452 0.1331
GADS 0.0581 0.0997
Tab.2 Effectiveness of KD tree search
情境变化 M1 /MB M2 /MB E1 /J E2 /J T ′ /s B /（MB·s?1） 变化前分割状态集 变化后分割状态集
移动端存储减少 230 9 0.02 0.02 0.01 6 8，10，9 7，8，10
移动端存储减少 230 4.5 0.02 0.02 0.01 6
移动端电量降低 230 9 0.02 0.02 0.01 6 8，10，9 6，7，8
移动端电量降低 230 9 0.02 0.005 0.01 6
网络带宽降低 230 9 0.02 0.02 0.01 6 8，10，9 10，8，9
网络带宽降低 230 9 0.02 0.02 0.01 1
Tab.3 Experiments on adaptive ability of GADS
Fig.13 Memory and energy consumption required at different partition points on AlexNet
Fig.14 Inference latency of different partition points and output data size of each layer on AlexNet
[1]   KRIZHEVSKY A, SUTSKEVER I, HINTON G E Imagenet classification with deep convolutional neural networks[J]. Advances in Neural Information Processing Systems, 2012, 25: 1097- 1105
[2]   HE K, ZHANG X, REN S, et al. Deep residual learning for image recognition [C]// Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Las Vegas: IEEE, 2016: 770-778.
[3]   ALOM M Z, TAHA T M, YAKOPCIC C, et al A state-of-the-art survey on deep learning theory and architectures[J]. Electronics, 2019, 8 (3): 292
doi: 10.3390/electronics8030292
[4]   ZHOU Z, CHEN X, LI E, et al Edge intelligence: paving the last mile of artificial intelligence with edge computing[J]. Proceedings of the IEEE, 2019, 107 (8): 1738- 1762
doi: 10.1109/JPROC.2019.2918951
[5]   LUO P, ZHU Z, LIU Z, et al. Face model compression by distilling knowledge from neurons [C]// Proceedings of the 30th AAAI Conference on Artificial Intelligence. Arizona: AAAI, 2016: 3560-3566.
[6]   HAN S, POOL J, TRAN J, et al. Learning both weights and connections for efficient neural networks [C]// Proceedings of the 28th International Conference on Neural Information Processing Systems. Montreal: MIT Press, 2015: 1135-1143.
[7]   LUO J H, WU J, LIN W. Thinet: a filter level pruning method for deep neural network compression [C]// Proceedings of the IEEE International Conference on Computer Vision. Venice: IEEE, 2017: 5058-5066.
[8]   LIU S, LIN Y, ZHOU Z, et al. On-demand deep model compression for mobile devices: a usage-driven model selection framework [C]// Proceedings of the 16th Annual International Conference on Mobile Systems, Applications, and Services. Munich: ACM, 2018: 389-400.
[9]   ZHAO Z, BARIJOUGH K M, GERSTLAUER A DeepThings: distributed adaptive deep learning inference on resource-constrained IoT edge clusters[J]. IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems, 2018, 37 (11): 2348- 2359
doi: 10.1109/TCAD.2018.2858384
[10]   KANG Y, HAUSWALD J, GAO C, et al Neurosurgeon: collaborative intelligence between the cloud and mobile edge[J]. ACM SIGARCH Computer Architecture News, 2017, 45 (1): 615- 629
doi: 10.1145/3093337.3037698
[11]   LI H, HU C, JIANG J, et al. JALAD: joint accuracy-and latency-aware deep structure decoupling for edge-cloud execution [C]// 2018 IEEE 24th International Conference on Parallel and Distributed Systems. Singapore: IEEE, 2018: 671-678.
[12]   MAO J, CHEN X, NIXON K W, et al. Modnn: local distributed mobile computing system for deep neural network [C]// Design, Automation and Test in Europe Conference and Exhibition. Lausanne: IEEE, 2017: 1396-1401.
[13]   KO J H, NA T, AMIR M F, et al. Edge-host partitioning of deep neural networks with feature space encoding for resource-constrained internet-of-things platforms [C]// 2018 15th IEEE International Conference on Advanced Video and Signal Based Surveillance. New Zealand: IEEE, 2018: 1-6.
[14]   LIU N, MA X, XU Z, et al. AutoCompress: an automatic DNN structured pruning framework for ultra-high compression rates [C]// Proceedings of the AAAI Conference on Artificial Intelligence. New York: AAAI, 2020, 34(04): 4876-4883.
[15]   HE Y, ZHANG X, SUN J. Channel pruning for accelerating very deep neural networks [C]// Proceedings of the IEEE International Conference on Computer Vision. Venice: IEEE, 2017: 1389-1397.
[16]   HU C, BAO W, WANG D, et al. Dynamic adaptive DNN surgery for inference acceleration on the edge [C]// IEEE Conference on Computer Communications. Paris: IEEE, 2019: 1423-1431.
[17]   CHEN Y H, EMER J, SZE V Eyeriss: a spatial architecture for energy-efficient dataflow for convolutional neural networks[J]. ACM SIGARCH Computer Architecture News, 2016, 44 (3): 367- 379
doi: 10.1145/3007787.3001177
[18]   YANG T J, CHEN Y H, SZE V. Designing energy-efficient convolutional neural networks using energy-aware pruning [C]// Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Hawaii: IEEE, 2017: 5687-5695.
[1] Jia-hui XU,Jing-chang WANG,Ling CHEN,Yong WU. Surface water quality prediction model based on graph neural network[J]. Journal of ZheJiang University (Engineering Science), 2021, 55(4): 601-607.
[2] Teng ZHANG,Xin-long JIANG,Yi-qiang CHEN,Qian CHEN,Tao-mian MI,Piu CHAN. Wrist attitude-based Parkinson's disease ON/OFF state assessment after medication[J]. Journal of ZheJiang University (Engineering Science), 2021, 55(4): 639-647.
[3] Li-feng XU,Hai-fan HUANG,Wei-long DING,Yu-lei FAN. Detection of small fruit target based on improved DenseNet[J]. Journal of ZheJiang University (Engineering Science), 2021, 55(2): 377-385.
[4] Hao-can XU,Ji-tuo LI,Guo-dong LU. Reconstruction of three-dimensional human bodies from single image by LeNet-5[J]. Journal of ZheJiang University (Engineering Science), 2021, 55(1): 153-161.
[5] Yi-peng HUANG,Ji-su HU,Xu-sheng QIAN,Zhi-yong ZHOU,Wen-lu ZHAO,Qi MA,Jun-kang SHEN,Ya-kang DAI. SE-Mask-RCNN: segmentation method for prostate cancer on multi-parametric MRI[J]. Journal of ZheJiang University (Engineering Science), 2021, 55(1): 203-212.
[6] Qiao-hong CHEN,YI CHEN,Wen-shu Li,Yu-bo JIA. Clothing image classification based on multi-scale SE-Xception[J]. Journal of ZheJiang University (Engineering Science), 2020, 54(9): 1727-1735.
[7] Pu ZHENG,Hong-yang BAI,Wei LI,Hong-wei GUO. Small target detection algorithm in complex background[J]. Journal of ZheJiang University (Engineering Science), 2020, 54(9): 1777-1784.
[8] Deng-wen ZHOU,Jin-yue TIAN,Lu-yao MA,Xiu-xiu SUN. Lightweight image semantic segmentation based on multi-level feature cascaded network[J]. Journal of ZheJiang University (Engineering Science), 2020, 54(8): 1516-1524.
[9] Tao MING,Dan WANG,Ji-chang GUO,Qiang LI. Breast cancer histopathological image classification using multi-scale channel squeeze-and-excitation model[J]. Journal of ZheJiang University (Engineering Science), 2020, 54(7): 1289-1297.
[10] Xu YAN,Xiao-liang FAN,Chuan-pan ZHENG,Yu ZANG,Cheng WANG,Ming CHENG,Long-biao CHEN. Urban traffic flow prediction algorithm based on graph convolutional neural networks[J]. Journal of ZheJiang University (Engineering Science), 2020, 54(6): 1147-1155.
[11] Zhou-fei WANG,Wei-na YUAN. Channel estimation and detection method for multicarrier system based on deep learning[J]. Journal of ZheJiang University (Engineering Science), 2020, 54(4): 732-738.
[12] Bing YANG,Wen-bo MO,Jin-liang YAO. 3D palmprint recognition by using local features and deep learning[J]. Journal of ZheJiang University (Engineering Science), 2020, 54(3): 540-545.
[13] Yan-jia HONG,Tie-bao MENG,Hao-jiang LI,Li-zhi LIU,Li LI,Shuo-yu XU,Sheng-wen GUO. Deep segmentation method of tumor boundaries from MR images of patients with nasopharyngeal carcinoma using multi-modality and multi-dimension fusion[J]. Journal of ZheJiang University (Engineering Science), 2020, 54(3): 566-573.
[14] Zi-yu JIA,You-fang LIN,Hong-jun ZHANG,Jing WANG. Sleep stage classification model based ondeep convolutional neural network[J]. Journal of ZheJiang University (Engineering Science), 2020, 54(10): 1899-1905.
[15] Wan-liang WANG,Xiao-han YANG,Yan-wei ZHAO,Nan GAO,Chuang LV,Zhao-juan ZHANG. Image enhancement algorithm with convolutional auto-encoder network[J]. Journal of ZheJiang University (Engineering Science), 2019, 53(9): 1728-1740.