Please wait a minute...
Journal of ZheJiang University (Engineering Science)  2020, Vol. 54 Issue (9): 1727-1735    DOI: 10.3785/j.issn.1008-973X.2020.09.008
    
Clothing image classification based on multi-scale SE-Xception
Qiao-hong CHEN(),YI CHEN,Wen-shu Li,Yu-bo JIA
School of Information Science and Technology, Zhejiang Sci-Tech University, Hangzhou 310018, China
Download: HTML     PDF(971KB) HTML
Export: BibTeX | EndNote (RIS)      

Abstract  

The current newer and better convolutional neural network Xception was used as the foundation network structures. Multi-scale depth separable convolution was employed to improve the richness of model feature information. The SE-Net model was embedded in the model to enhance the useful feature channels and weaken the useless feature channels. The experimental results show that the multi-scale SE-Xception model achieved good performance in two different noise clothing datasets. The average classification accuracy of ACS dataset was 78.34%, which was higher than VGG-16, ResNet-50 and Xception models by 8.52%, 4.81% and 3.69%, respectively. Therefore, it’s verified that the multi-scale SE-Xception model has better ability to extract features, can obtain more clothing information, and thus improve the clothing image classification effect. To some extent, a multi-scale SE-Xception network is conducive to solve the single feature scale and low information richness problems.



Key wordsclothing image classification      multi-scale SE-Xception      image identification      deep learning      machine learning      convolutional neural network (CNN)     
Received: 14 August 2019      Published: 22 September 2020
CLC:  TP 181  
Cite this article:

Qiao-hong CHEN,YI CHEN,Wen-shu Li,Yu-bo JIA. Clothing image classification based on multi-scale SE-Xception. Journal of ZheJiang University (Engineering Science), 2020, 54(9): 1727-1735.

URL:

http://www.zjujournals.com/eng/10.3785/j.issn.1008-973X.2020.09.008     OR     http://www.zjujournals.com/eng/Y2020/V54/I9/1727


多尺度SE-Xception服装图像分类

应用当前较新颖且分类性能靠前的卷积神经网络Xception作为基础网络结构,尝试采用多尺度的深度可分离卷积来提升模型特征信息的丰富度,在模型中嵌入SE-Net模块增强有用特征通道,减弱无用特征通道. 实验结果表明:提出的多尺度SE-Xception模型在2种噪声程度不同的服装数据集中均取得不错表现;ACS数据集的平均分类准确率为78.34%,分别高于VGG-16、ResNet-50和Xception模型8.52%、4.81%、3.69%;验证了多尺度SE-Xception模型具有更好的特征提取能力,能够提取到更多的服装信息,从而提高服装图像分类效果,一定程度上解决了特征尺度单一、信息丰富度低的问题.


关键词: 服装图像分类,  多尺度SE-Xception,  图像识别,  深度学习,  机器学习,  卷积神经网络(CNN) 
Fig.1 Diagram of main network module of Xception model
Fig.2 Architecture of depth separable convolution
Fig.3 Flowchart of SE-Net module
Fig.4 Architecture of multi-scale depth separable convolution
层级序号 残差链接 循环 卷积操作 输出尺寸
Conv_1 ? ? Conv 32,3×3,stride = 2 111×111×32
Conv_2 ? ? Conv 64,3×3 109×109×64
Conv_3 Conv 1×1,stride = 2 ? 多尺度深度可分离卷积 128
多尺度深度可分离卷积 128
最大池化 3×3,stride = 2,padding=1
55×55×128
Conv_4 Conv 1×1,stride = 2 ? 多尺度深度可分离卷积 256
多尺度深度可分离卷积 256
最大池化 3×3,stride = 2,padding=1
28×28×256
Conv_5 Conv 1×1,stride = 2 ? 多尺度深度可分离卷积 728
多尺度深度可分离卷积 728
最大池化 3×3,stride = 2,padding=1
14×14×728
Conv_6_x 直连 ×8 多尺度深度可分离卷积 728
多尺度深度可分离卷积 728
多尺度深度可分离卷积 728
14×14×728
Conv_7 Conv 1×1,stride = 2 ? 多尺度深度可分离卷积 728
多尺度深度可分离卷积 1024
最大池化 3×3,stride = 2,padding=1
7×7×1 024
Conv_8 ? ? 多尺度深度可分离卷积 1536
SE-Net模块
7×7×1 024
Conv_9 ? ? 多尺度深度可分离卷积 2048
SE-Net模块
7×7×2 048
GAP_10 ? ? 全局均值池化 1×1×2 048
FC_11 ? ? 全连接层 1×1×7
Tab.1 Framework of multi-scale SE-Xception model
Fig.5 Sample graph of ACS dataset and DeepFashion dataset
Fig.6 Comparison of loss change curves of different models on ACS dataset
Fig.7 Comparison of accuracy rate change curves of different models on ACS datasets
卷积核组合 R/%
$3 \times 3$ 74.65
$3 \times 3$$5 \times 5$ 76.12
$1 \times 1$$3 \times 3$,最大池化 75.70
$1 \times 1$$5 \times 5$,最大池化 75.58
$1 \times 1$$3 \times 3$$5 \times 5$,最大池化 78.34
Tab.2 Comparison of experimental results of convolution kernel with different combinations
CNN模型 RDF RACS RDEC
Xception_Net 78.68 74.65 4.03
Multi_SE_Xception_Net 80.14 78.34 1.80
Tab.3 Comparison of experimental results between Xception model and multi-scale SE-Xception model in different datasets %
CNN模型 R/%
VGG16_Net 69.82
Resnet50_Net 73.53
Xception_Net 74.65
Multi_Xception_Net 77.58
Multi_SE_Xception_Net 78.34
Tab.4 Comparison of average classification accuracy of different models on ACS datasets
[1]   中国电子商务研究中心. 2015—2016年度中国服装电商行业报告[R/OL]. (2016-12-26)[2018-12-17]. http://www.100ec.cn/zt/fzyxbg/.
[2]   OJALA T, PIETIKAINEN M, HARWOOD D. Performance evaluation of texture measures with classification based on Kullback discrimination of distributions [C] // Proceedings of 12th International Conference on Pattern Recognition. Jerusalem: IEEE, 1994: 582-585.
[3]   LOWE D G Distinctive image features from scale-invari-ant keypoints[J]. International Journal of Computer Vision, 2004, 60 (2): 91- 110
doi: 10.1023/B:VISI.0000029664.99615.94
[4]   DALAL N, TRIGGS B. Histograms of oriented gradients for human detection [C] // Computer Vision and Pattern Re-cognition. San Diego: IEEE, 2005: 886-893.
[5]   BOSSARD L, DANTONE M, LEISTNER C, et al. Apparel classification with style [C] // Asian Conference on Com-puter Vision. Daejeon: Springer, 2012: 321-335.
[6]   HUO P, WANG Y, LIU Q. A part-based and feature fus-ion method for clothing classification [C] // Pacific Rim Conference on Multimedia. Xi’an: Springer, 2016: 231-241.
[7]   SURAKARIN W, CHONGSTITVATANA P. Predicting types of clothing using SURF and LDP based on bag of features [C] // International Conference on Electrical Engineering/Electronics. Hua Hin: IEEE, 2015: 1-5.
[8]   吴苗苗, 刘骊, 付晓东, 等 款式特征描述符的服装图像细粒度分类方法[J]. 计算机辅助设计与图形学学报, 2019, 31 (5): 780- 791
WU Miao-miao, LIU Li, FU Xiao-dong, et al Fine-grained clothing image classification by style feature description[J]. Journal of Computer-aided Design and Computer Graphics, 2019, 31 (5): 780- 791
doi: 10.3724/SP.J.1089.2019.17380
[9]   LIU Z, LUO P, QIU S, et al. DeepFashion: Powering ro-bust clothes recognition and retrieval with rich annotations [C] // 2016 the IEEE Conference on Computer Vision and Pattern Recognition. Las Vegas: IEEE, 2016: 1096-1104.
[10]   DONG C Y, SHI Y Q, TAO R. Convolutional neural ne-tworks for clothing image style recognition [C] // 2018 International Conference on Computational, Modeling, Simulation and Mathematical Statistics. Pennsylvania: CMSMS, 2018: 592-597.
[11]   包青平, 孙志锋 基于度量学习的服装图像分类和检索[J]. 计算机应用与软件, 2017, 34 (4): 255- 259
BAO Qing-ping, SUN Zhi-feng Clothing image classification and retrieval based on metric learning[J]. Computer Applications and Software, 2017, 34 (4): 255- 259
doi: 10.3969/j.issn.1000-386x.2017.04.043
[12]   张振焕, 周彩兰, 梁媛 基于残差的优化卷积神经网络服装分类算法[J]. 计算机工程与科学, 2018, 40 (2): 354- 360
ZHANG Zhen-huan, ZHOU Cai-lan, LIANG Yuan An optimized clothing classification algorithm based on residual convolutional neural network[J]. Computer Engineering and Science, 2018, 40 (2): 354- 360
doi: 10.3969/j.issn.1007-130X.2018.02.023
[13]   厉智, 孙玉宝, 王枫, 等 基于深度卷积神经网络的服装图像分类检索算法[J]. 计算机工程, 2016, 42 (11): 309- 315
LI Zhi, SUN Yu-bao, WANG Feng, et al Clothing image classification and retrieval algorithm based on deep convolutional neural network[J]. Computer Engineering, 2016, 42 (11): 309- 315
doi: 10.3969/j.issn.1000-3428.2016.11.053
[14]   KRIZHEVSKY A, SUTSKEVER I, HINTON G E. ImageNet classification with deep convolutional neural networks [C] // Advances in Neural Information Processing Systems 25 (NIPS 2012). Nevada: NIPS, 2012: 1097-1105.
[15]   SIMONYAN K, ZISSERMAN A. Very deep convolutional networks for large-scale image recognition [EB/OL]. (2015-04-10)[2018-12-17]. https://arxiv.org/abs/1409.1556.
[16]   SZEGEDY C, LIU W, JIA Y, et al. Going deeper with convolutions [C] // 2015 the IEEE Conference on Computer Vision and Pattern Recognition. Boston: IEEE, 2015: 1-9.
[17]   HE K, ZHANG X, REN S, et al. Deep residual learning for image recognition [C] // 2016 the IEEE Conference on Computer Vision and Pattern Recognition. Las Vegas: IEEE, 2016: 770-778.
[18]   CHOLLET F. Xception: deep learning with depthwise separable convolutions [C] // Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Honolulu: IEEE, 2017: 1251-1258.
[1] Jia-hui XU,Jing-chang WANG,Ling CHEN,Yong WU. Surface water quality prediction model based on graph neural network[J]. Journal of ZheJiang University (Engineering Science), 2021, 55(4): 601-607.
[2] Hong-li WANG,Bin GUO,Si-cong LIU,Jia-qi LIU,Yun-gang WU,Zhi-wen YU. End context-adaptative deep sensing model with edge-end collaboration[J]. Journal of ZheJiang University (Engineering Science), 2021, 55(4): 626-638.
[3] Teng ZHANG,Xin-long JIANG,Yi-qiang CHEN,Qian CHEN,Tao-mian MI,Piu CHAN. Wrist attitude-based Parkinson's disease ON/OFF state assessment after medication[J]. Journal of ZheJiang University (Engineering Science), 2021, 55(4): 639-647.
[4] You ZHAN,Qiang LI,Xiao-tian MA,Chen-ping WANG,Yan-jun QIU. Macro and micro texture based prediction of pavement surface friction[J]. Journal of ZheJiang University (Engineering Science), 2021, 55(4): 684-694.
[5] Yong YU,Jing-yuan XUE,Sheng DAI,Qiang-wei BAO,Gang ZHAO. Quality prediction and process parameter optimization method for machining parts[J]. Journal of ZheJiang University (Engineering Science), 2021, 55(3): 441-447.
[6] Li-feng XU,Hai-fan HUANG,Wei-long DING,Yu-lei FAN. Detection of small fruit target based on improved DenseNet[J]. Journal of ZheJiang University (Engineering Science), 2021, 55(2): 377-385.
[7] Hao-can XU,Ji-tuo LI,Guo-dong LU. Reconstruction of three-dimensional human bodies from single image by LeNet-5[J]. Journal of ZheJiang University (Engineering Science), 2021, 55(1): 153-161.
[8] Yi-peng HUANG,Ji-su HU,Xu-sheng QIAN,Zhi-yong ZHOU,Wen-lu ZHAO,Qi MA,Jun-kang SHEN,Ya-kang DAI. SE-Mask-RCNN: segmentation method for prostate cancer on multi-parametric MRI[J]. Journal of ZheJiang University (Engineering Science), 2021, 55(1): 203-212.
[9] Pu ZHENG,Hong-yang BAI,Wei LI,Hong-wei GUO. Small target detection algorithm in complex background[J]. Journal of ZheJiang University (Engineering Science), 2020, 54(9): 1777-1784.
[10] Deng-wen ZHOU,Jin-yue TIAN,Lu-yao MA,Xiu-xiu SUN. Lightweight image semantic segmentation based on multi-level feature cascaded network[J]. Journal of ZheJiang University (Engineering Science), 2020, 54(8): 1516-1524.
[11] Tao MING,Dan WANG,Ji-chang GUO,Qiang LI. Breast cancer histopathological image classification using multi-scale channel squeeze-and-excitation model[J]. Journal of ZheJiang University (Engineering Science), 2020, 54(7): 1289-1297.
[12] Xu YAN,Xiao-liang FAN,Chuan-pan ZHENG,Yu ZANG,Cheng WANG,Ming CHENG,Long-biao CHEN. Urban traffic flow prediction algorithm based on graph convolutional neural networks[J]. Journal of ZheJiang University (Engineering Science), 2020, 54(6): 1147-1155.
[13] Ping YANG,Dan WANG,Zi-jian KAGN,Tong LI,Li-hua FU,Yue-ren YU. Prediction model of paroxysmal atrial fibrillation based on pattern recognition and ensemble CNN-LSTM[J]. Journal of ZheJiang University (Engineering Science), 2020, 54(5): 1039-1048.
[14] Zhou-fei WANG,Wei-na YUAN. Channel estimation and detection method for multicarrier system based on deep learning[J]. Journal of ZheJiang University (Engineering Science), 2020, 54(4): 732-738.
[15] Bing YANG,Wen-bo MO,Jin-liang YAO. 3D palmprint recognition by using local features and deep learning[J]. Journal of ZheJiang University (Engineering Science), 2020, 54(3): 540-545.