Multi-source unsupervised domain adaption method based on self-supervised task
Lan WU1(),Han WANG1,Bin-quan LI1,Chong-yang LI1,Fan-shi KONG2
1. School of Electrical Engineering, Henan University of Technology, Zhengzhou 450001, China 2. Zhengzhou Railway Vocational and Technical College, Zhengzhou 450001, China
A multi-source unsupervised domain adaptation method based on self-supervised tasks was proposed aiming at the problem of low classification accuracy due to the difficulty of simultaneously aligning domain-invariant features under multi-source aggregation. The method introduced three self-supervised auxiliary tasks of rotation, horizontal flip and position prediction, and performed adaption alignment optimization on unlabeled data through pseudo-labeling and consistency of semantic information. A new optimized loss was built, and the classification variance of multi-domain common classes was reduced. Dynamic weight parameters were defined to improve the classification performance of the model based on the principle of few samples and large weights for the problem of class-imbalance. Experiments were compared with the existing mainstream methods on the two benchmark data sets, Office-31 and Office-Caltech10. The experimental results show that the classification accuracy can be improved by up to 6.8% in the two cases of class balance and imbalance.
Lan WU,Han WANG,Bin-quan LI,Chong-yang LI,Fan-shi KONG. Multi-source unsupervised domain adaption method based on self-supervised task. Journal of ZheJiang University (Engineering Science), 2022, 56(4): 754-763.
Fig.3Schematic diagram of semantic consistency of self-supervised tasks
Fig.4Transfer principle under class-imbalance
Fig.5Loss function relationship diagram
Fig.6Example diagrams of two data sets
标准
方法
A/%
AW-D
AD-W
DW-A
平均值
单源迁移
DAN[2]
99.0
96.0
54.0
83.0
单源迁移
ADDA[3]
99.4
95.3
54.6
83.1
单源迁移
RevGrad[4]
99.2
96.4
53.4
83.0
源域组合
DAN[2]
98.8
96.2
54.9
83.3
源域组合
JAN[5]
99.4
95.9
54.6
83.3
源域组合
MCD[6]
99.5
96.2
54.4
83.4
源域组合
RevGrad[4]
98.8
96.2
54.6
83.2
多源迁移
DCTN[7]
99.6
96.9
54.9
83.8
多源迁移
M3SDA[10]
99.4
96.2
55.4
83.7
多源迁移
MDAN[8]
99.2
95.4
55.2
83.3
多源迁移
MDDA[11]
99.2
97.1
56.2
84.2
多源迁移
本文方法
99.6
97.2
56.4
84.4
Tab.1Comparison and analysis of accuracy on Office-31 data set
标准
方法
A/%
ADC-W
ACW-D
AWD-C
DWC-A
平均值
源域组合
Source only
99.0
98.3
87.8
86.1
92.8
源域组合
DAN[2]
99.3
98.2
89.7
94.8
95.5
多源迁移
Source only
99.1
98.2
85.4
88.7
92.9
多源迁移
FADA[13]
88.1
87.1
88.7
84.2
87.1
多源迁移
DAN[2]
99.5
99.1
89.2
91.6
94.9
多源迁移
DCTN[7]
99.4
99.0
90.2
92.7
95.3
多源迁移
M3SDA[10]
99.4
99.2
91.5
94.1
96.1
多源迁移
本文方法
99.5
99.2
90.2
93.3
95.5
Tab.2Comparison and analysis of accuracy on Office-Caltech10 data set
Fig.7Analysis of impact of loss function on model performance in Office-31 data set
Fig.8Analysis of impact of loss function on model performance in Office-Caltech10 data set
Fig.9Analysis of training effects for self-supervised tasks in Office-31 data set
Fig.10Effects of each loss function on classification accuracy in inter-domain difference loss
Fig.11Comparison of sample size of each source domain in Office-31 data set
Fig.12Comparison of sample size of each source domain in Office-Caltech10 data set
Fig.13Figure of weight setting for each sample on Office-31
Fig.14Figure of weight setting for each sample on Office-Caltech10
类型
A/%
AD-W
AW-D
DW-A
平均值
类别不均衡
81.0
82.6
42.4
68.7
类别不均衡（权重）
87.8
85.5
42.7
72.0
Tab.3Comparison and analysis of accuracy on Office-31 data set under uneven samples
类型
A/%
ADC-W
DWC-A
AWD-C
ACW-D
平均值
类别不均衡
75.3
67.8
63.7
79.3
71.5
类别不均衡（权重）
83.5
76.4
67.3
83.0
77.6
Tab.4Comparison and analysis of accuracy on Office-Caltech10 data set under uneven samples
[1]
沈宗礼, 余建波 基于迁移学习与深度森林的晶圆图缺陷识别[J]. 浙江大学学报: 工学版, 2020, 54 (6): 1228- 1239 SHEN Zong-li, YU Jian-bo Wafer map defect recognition based on transfer learning and deep forest[J]. Journal of Zhejiang University: Engineering Science, 2020, 54 (6): 1228- 1239
[2]
LONG M, CAO Y, WANG J, et al. Learning transferable features with deep adaptation networks [C]// International Conference on Machine Learning. Lille: [s. n.], 2015: 97-105.
[3]
TZENG E, HOFFMAN J, SAENKO K, et al. Adversarial discriminative domain adaptation[C]// Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Honolulu: IEEE, 2017: 7167–7176.
[4]
GANIN Y, LEMPITSKY V. Unsupervised domain adaptation by backpropagation [C]// International Conference on Machine Learning. Lille: PMLR, 2015: 1180–1189.
[5]
LONG M, ZHU H, WANG J, et al. Deep transfer learning with joint adaptation networks[C]// International Conference on Machine Learning. Sydney: PMLR, 2017: 2208–2217.
[6]
SAITO K, WATANABE K, USHIKU Y, et al. Maximum classifier discrepancy for unsupervi-sed domain adaptation [C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Salt Lake City: IEEE, 2018: 3723–3732.
[7]
XU R, CHEN Z, ZUO W, et al. Deep cockt-ail network: multi-source unsupervised domain adaptation with category shift [C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Salt Lake City: IEEE, 2018: 3964–3973.
[8]
ZHAO H, ZHANG S, WU G. Adversarial multiple source domain adaptation [C]// Proceedings of the 32nd International Conference on Neural Information Processing Systems. Montreal: MIT Press, 2018: 8568–8579.
[9]
李威, 王蒙. 基于渐进多源域迁移的无监督跨域目标检测[EB/OL]. （2020-03-20）. http://kns.cnki.net/kcms/detail/11.2109.TP.20200320.1044.003.html. LI Wei, WANG Meng. Unsupervised cross-domain object detection based on progressive multi-source transfer [EB/OL]. (2020-03-20). http://kns.cnki.net/kcms/detail/11.2109.TP.20200320.1044.003.html.
[10]
PENG X, BAI Q, XIA X, et al. Moment matching for multi-source domain adaptation [C]// Proceedings of the IEEE/CVF International Conference on Computer Vision. Seoul: IEEE, 2019: 1406–1415.
[11]
ZHAO S, WANG G, ZHANG S, et al. Multi-source distilling domain adaptation [C]// Proceedings of the AAAI Conference on Artificial Intelligence. New York: AAAI, 2020: 1297–1298.
[12]
ZHU Y, ZHUANG F, WANG D. Aligning domain specific distribution and classifier for cross-domain classification from multiple sources [C]// Proceedings of the AAAI Conference on Artificial Intelligence. Hawaii: AAAI, 2019: 5989–5996.
[13]
PENG X, HUANG Z, ZHU Y, et al. Federatedadversarial domain adaptation [C]// International Conference on Learning Representations. Addis Ababa: [s. n.], 2020.
[14]
ZHANG R, ISOLA P, ALEXEI A. Colorful image colorization [C]// European Conference on Computer Vision. Cham: Springer, 2016: 649–666.
[15]
GUSTAV L A, MICHAEL M, GREG O, et al. Learning representations for automatic colorization [C]// European Conference on Computer Vision. Amsterdam: [s. n.], 2016: 577–593.
[16]
CARL V, ABHINAV S, ALIREZA F, et al. Tracking emerges by colorizing videos [C]// Proceedings of the European Conference on Computer Vision. Munich: [s. n.], 2018: 391–408.
[17]
NOROOZI M, FAVARO P. Unsupervised learning of visual representations by solving jigsaw puzzles [C]// European Conference on Computer Vision. Amsterdam: [s. n.], 2016: 69–84.
[18]
CARL D, ABHINAV G, ALEXEI A. Unsupervised visual representation learning by conte-xt prediction [C]// Proceedings of the IEEE International Conference on Computer Vision. [S. l.]: IEEE, 2015: 1422–1430.
[19]
IMON J, PAOLO F. Self-supervised feature learning by learning to spot artifacts [C]// Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Salt Lake City: IEEE, 2018: 2733–2742.
[20]
SPYROS G, PRAVEER S, NIKOS K. Unsupervised representation learning by predicting image rotations [EB/OL].(2018-03-21). https://doi.org/10.48550/arXiv.1803.07728.
[21]
DEEPAK P, PHILIPP K, JEFF D, et al. Context encoders: feature learning by inpainting [C]// Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Las Vegas: IEEE, 2016: 2536–2544.
