Please wait a minute...
浙江大学学报(工学版)
浙江大学学报(工学版)  2023, Vol. 57 Issue (9): 1876-1884    DOI: 10.3785/j.issn.1008-973X.2023.09.019
计算机技术     
基于伪标签细化和语义对齐的异构域自适应
吴兰(),崔全龙
河南工业大学 电气工程学院，河南 郑州 450001
Heterogeneous domain adaptation based on pseudo label refinement and semantic alignment
Lan WU(),Quan-long CUI
College of Electrical Engineering, Henan University of Technology, Zhengzhou 450001, China
 全文: PDF(848 KB)   HTML
摘要：

在进行跨域知识迁移时，现有异构域自适应方法忽略了伪标签和类别预测中语义属性的重要性，导致分类精度不高，为此提出新的异构域自适应方法. 该方法匹配源域和目标域的条件分布和边缘分布，考虑源数据和目标数据在公共特征子空间的相似性，通过细化空间相似性的伪标签来增强目标域伪标签的置信度，使模型的分类精度提高. 考虑同类的样本经过分类器输出后有相似的预测分布，构造语义预测空间中的域鉴别器，使模型的泛化性提升. 不同特征表示的文本和图像的分类任务实验结果成功地验证了所提方法的优越性.
关键词： 异构域自适应;  伪标签细化;  语义预测空间;  域鉴别器;  语义对齐    
Abstract:

A new heterogeneous domain adaptive method was proposed for the existing methods ignoring the importance of semantic attributes in pseudo-labeling and category prediction, resulting in poor classification accuracy. The conditional and the marginal distributions of the source and the target domains were matched, and the similarity of the source and the target data in the common feature subspace was considered. The classification accuracy of a model was increased, enhancing the confidence of the pseudo-labels of the target domain pseudo-labels by refining the pseudo-labels of the spatial similarity. The domain discriminator in the semantic prediction space was constructed by the similar predictive distributions of the similar samples after the classifier output, and the generalization of the model was improved. Experimental results of the classification task for text and images with different feature representations successfully validate the superiority of the proposed method.
Key words: heterogeneous domain adaptation    pseudo-label refinement    semantic prediction space    domain discriminator    semantic alignment
收稿日期: 2022-11-24 出版日期: 2023-10-16
CLC:  TP 391.41  
基金资助: 国家自然科学基金资助项目（61973103）；河南省优秀青年基金资助项目（222300420039）；郑州市科技创新协同专项重点项目（21ZZXTCX01）
作者简介: 吴兰（1981—），女，教授，博士，从事深度学习方面的研究. orcid.org/0000-0002-2497-6556. E-mail： wulan@haut.edu.cn
服务　　
把本文推荐给朋友
加入引用管理器
E-mail Alert
作者相关文章　　
吴兰
崔全龙

引用本文:

吴兰,崔全龙. 基于伪标签细化和语义对齐的异构域自适应[J]. 浙江大学学报(工学版), 2023, 57(9): 1876-1884.

Lan WU,Quan-long CUI. Heterogeneous domain adaptation based on pseudo label refinement and semantic alignment. Journal of ZheJiang University (Engineering Science), 2023, 57(9): 1876-1884.

链接本文:

https://www.zjujournals.com/eng/CN/10.3785/j.issn.1008-973X.2023.09.019    或    https://www.zjujournals.com/eng/CN/Y2023/V57/I9/1876

图 1  异构域自适应的网络结构图
图 2  伪标签分配原理图
方法 Acc% 方法 Acc%
NNt 67.68 LG 77.54
CDLS 70.96 SDA-PPLS 76.22
STN 78.46 本研究 80.54
SSAN 80.22 — —
表 1  不同方法在 ImageNet+NUS-WIDE数据集上进行文本到图像异构迁移的分类结果
方法 ${{\rm{Acc}}_ {{\rm{SD}}}} $/% ${{{\rm{Acc}}}_ {{\rm{DS}}}} $/%
A→A C→C W→W 平均值 A→A C→C W→W 平均值
SVMt 88.66 77.31 89.32 85.10 43.03 30.15 55.28 42.82
NNt 90.00 79.56 91.42 86.99 42.82 31.33 60.87 45.01
MMDT 89.30 80.30 87.30 85.63 40.50 30.60 59.10 43.40
G-JDA 92.30 86.70 89.40 89.47 50.30 33.70 63.80 49.27
CDLS 91.70 81.80 95.20 89.57 46.40 31.80 63.10 47.10
STN 92.19 82.92 95.43 90.18 47.62 30.83 64.71 47.72
SSAN 92.45 87.01 96.66 92.04 52.91 37.24 69.81 53.32
LG 92.36 84.15 96.04 90.85 44.70 31.20 63.10 46.33
SDA-PPLS 92.85 87.34 96.31 92.17 54.88 33.18 71.72 53.26
本研究 93.70 89.03 97.29 93.34 54.40 39.09 72.83 55.44
表 2  不同方法在 Office+Caltech-256数据集上进行同域跨特征迁移的分类结果
方法 Acc%
A→C A→W A→D C→A C→D C→W W→A W→C W→D 平均值
SVMt 79.64 89.34 92.60 89.13 92.60 89.34 89.13 79.64 92.60 87.68
NNt 81.03 91.13 92.99 89.60 92.99 91.13 89.60 81.03 92.99 88.69
MMDT 75.62 89.28 91.65 87.06 91.46 89.11 87.00 75.44 91.77 86.49
CDLS 78.73 91.57 94.45 86.34 90.43 88.60 87.51 77.30 92.72 87.52
SGW 79.88 90.26 93.43 89.03 93.43 90.26 89.02 79.85 93.43 88.73
G-JDA 86.60 94.09 90.67 92.49 88.62 92.64 92.28 84.82 95.87 90.90
TNT 85.79 91.26 92.04 92.35 92.67 92.98 92.99 86.28 94.09 91.17
STN 88.21 96.68 96.42 93.03 96.06 96.38 93.11 87.22 96.38 93.72
SSAN 88.30 96.31 97.00 93.16 97.44 95.87 93.54 88.18 97.64 94.17
SDA-PPLS 86.29 95.42 95.68 92.16 95.42 95.62 91.15 85.40 97.24 92.71
本研究 89.07 97.17 98.03 93.49 98.27 96.67 93.95 89.56 98.58 94.98
表 3  不同方法在 Office+Caltech-256数据集上进行异域跨特征迁移的分类结果（SURF→DeCAF6)
方法 Acc%
A→C A→W A→D C→A C→D C→W W→A W→C W→D 平均值
SVMt 19.99 78.60 84.37 34.26 84.37 78.60 34.26 19.99 84.37 57.65
NNt 20.22 81.51 84.25 34.81 84.25 81.51 34.81 20.22 84.25 58.43
CDLS 21.32 80.38 85.03 34.69 86.61 78.11 35.66 20.31 82.68 58.31
STN 21.64 83.77 85.20 34.16 86.59 84.21 35.18 20.71 83.74 59.47
SSAN 24.07 88.47 91.81 41.71 91.22 86.24 37.36 23.50 93.62 64.22
本研究 27.93 90.68 94.63 42.79 94.51 90.16 39.68 27.00 94.88 66.92
表 4  不同方法在 Office+Caltech-256数据集上进行异域跨特征迁移的分类结果（SURF→ ResNet50)
方法 Acc%
E→S F→S G→S I→S 平均值
SVMt 67.70 67.70 67.70 67.70 67.70
NNt 68.20 68.20 68.20 68.20 68.00
NNst 67.72 67.23 67.53 67.76 67.56
MMDT 68.21 65.66 67.24 67.13 67.06
SHFA 70.33 70.32 70.33 70.33 70.33
G-JDA 68.53 69.51 69.62 69.63 69.32
CDLS 68.36 68.54 68.61 68.50 68.50
DDASL 70.15 70.92 70.23 70.50 70.45
STN 73.82 74.55 73.71 74.22 74.08
SSAN 76.16 76.61 76.72 77.16 76.66
LG 73.21 71.94 72.53 66.98 71.17
SDA-PPLS 75.55 74.38 74.33 75.71 74.99
本研究 77.04 77.74 78.18 78.05 77.75
表 5  不同方法在 Multilingual Reuters Collection数据集上进行文本到文本异构迁移的分类结果
方法 Acc%
A→D C→W W→A
PLR-SAs 95.28 95.28 92.96
PLR-SA(β=0) 94.49 92.83 89.01
PLR-SAst 95.16 94.91 92.35
PLR-SA(γ=0) 94.88 95.85 92.56
PLR-SA(q=Q) 97.71 96.11 93.24
PLR-SA 98.03 96.67 93.95
表 6  本研究所提方法在Office+Caltech-256数据集上的消融实验
图 3  本研究所提方法在不同迁移任务上的参数敏感性分析结果
1 沈宗礼, 余建波 基于迁移学习与深度森林的晶圆图缺陷识别[J]. 浙江大学学报: 工学版, 2020, 54 (6): 1228- 1239
SHEN Zong-li, YU Jian-bo Wafer map defect recognition based on transfer learning and deep forest[J]. Journal of Zhejiang University: Engineering Science, 2020, 54 (6): 1228- 1239
2 DAY O, KHOSHGOFTAAR T M A survey on heterogeneous transfer learning[J]. Journal of Big Data, 2017, 4: 29
doi: 10.1186/s40537-017-0089-0
3 LUO Y, WEN Y, TAO D Heterogeneous multitask metric learning across multiple domains[J]. IEEE Transactions on Neural Networks and Learning Systems, 2018, 29 (9): 4051- 4064
doi: 10.1109/TNNLS.2017.2750321
4 REN C X, FENG J, DAI D Q, et al Heterogeneous domain adaptation via covariance structured feature translators[J]. IEEE Transactions on Cybernetics, 2021, 51 (4): 2166- 2177
doi: 10.1109/TCYB.2019.2957033
5 LI H, PAN S J, WAN R, et al. Heterogeneous transfer learning via deep matrix completion with adversarial kernel embedding [C]// Proceedings of the Thirty-Third AAAI Conference on Artificial Intelligence and Thirty-First Innovative Applications of Artificial Intelligence Conference and Ninth AAAI Symposium on Educational Advances in Artificial Intelligence. [S.l.]: AAAI Press, 2019: 8602-8609.
6 TSAI Y H H, YEH Y R, WANG Y C F. Learning cross-domain landmarks for heterogeneous domain adaptation [C]// Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Las Vegas: IEEE, 2016: 5081-5090.
7 HSIEH Y T, TAO S Y, TSAI Y H H, et al. Recognizing heterogeneous cross-domain data via generalized joint distribution adaptation [C]// 2016 IEEE International Conference on Multimedia and Expo. Seattle: IEEE, 2016: 1-6.
8 YAO Y, ZHANG Y, LI X, et al. Heterogeneous domain adaptation via soft transfer network [C]// Proceedings of the 27th ACM International Conference on Multimedia. [S.l.]: ACM, 2019: 1578-1586.
9 WANG C, MAHADEVAN S. Heterogeneous domain adaptation using manifold alignment [C]// Proceedings of the Twenty-Second International Joint Conference on Artificial Intelligence. [S.l.]: AAAI Press, 2011: 1541-1546.
10 DUAN L, XU D, TSANG I W. Learning with augmented features for heterogeneous domain adaptation [C]// Proceedings of the 29th International Conference on International Conference on Machine Learning. [S.l.]: Omnipress, 2012: 667-674.
11 LI W, DUAN L, XU D, et al Learning with augmented features for supervised and semi-supervised heterogeneous domain adaptation[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2014, 36 (6): 1134- 1148
doi: 10.1109/TPAMI.2013.167
12 HOFFMAN J, RODNER E, DONAHUE J, et al. Efficient learning of domain-invariant image representations[EB/OL]. [2022-10-02]. https://arxiv.org/pdf/1301.3224.pdf.
13 ZHOU J T, TSANG I W, PAN S J, et al. Heterogeneous domain adaptation for multiple classes [C]// Proceedings of the 17th International Conference on Artificial Intelligence and Statistics. Reykjavik: [s.n.], 2014: 1095-1103.
14 YAN Y, LI W, WU H, et al. Semi-supervised optimal transport for heterogeneous domain adaptation [C]// Proceedings of the 27th International Joint Conference on Artificial Intelligence. [S.l.]: AAAI Press, 2018: 2969-2975.
15 CHEN W Y, HSU T M H, TSAI Y H H, et al. Transfer neural trees for heterogeneous domain adaptation [C]// European Conference on Computer Vision. [S.l.]: Springer, 2016: 399-414.
16 LI S, XIE B, WU J, et al. Simultaneous semantic alignment network for heterogeneous domain adaptation [C]// Proceedings of the 28th ACM International Conference on Multimedia. [S.l.]: ACM, 2020: 3866-3874.
17 ZHOU Z, WANG Y, NIU C, et al Label-guided heterogeneous domain adaptation[J]. Multimedia Tools and Applications, 2022, 81: 20105- 20126
doi: 10.1007/s11042-022-12483-1
18 ALIPOUR N, TAHMORESNEZHAD J Heterogeneous domain adaptation with statistical distribution alignment and progressive pseudo label selection[J]. Applied Intelligence, 2022, 52: 8038- 8055
doi: 10.1007/s10489-021-02756-x
[1] 张艳,孙晶雪,孙叶美,刘树东,王传启. 基于分割注意力与线性变换的轻量化目标检测[J]. 浙江大学学报(工学版), 2023, 57(6): 1195-1204.
[2] 袁天乐,袁巨龙,朱勇建,郑翰辰. 基于改进YOLOv5的推力球轴承表面缺陷检测算法[J]. 浙江大学学报(工学版), 2022, 56(12): 2349-2357.
[3] 于楠晶,范晓飚,邓天民,冒国韬. 基于多头自注意力的复杂背景船舶检测算法[J]. 浙江大学学报(工学版), 2022, 56(12): 2392-2402.
[4] 徐铸业,赵小强,蒋红梅. 基于点分布模型的3D模型拟合方法[J]. 浙江大学学报(工学版), 2021, 55(12): 2373-2381.
[5] 冯毅雄,李康杰,高一聪,郑浩. 面向视觉伺服的工业机器人轮廓曲线角点识别[J]. 浙江大学学报(工学版), 2020, 54(8): 1449-1456.
[6] 沈宗礼,余建波. 基于迁移学习与深度森林的晶圆图缺陷识别[J]. 浙江大学学报(工学版), 2020, 54(6): 1228-1239.
[7] 唐有宝, 卜巍, 邬向前. 多层次MSER自然场景文本检测[J]. 浙江大学学报(工学版), 2016, 50(6): 1134-1140.
[8] 吴一全,殷骏,朱丽,袁永明. 基于蜂群优化或分解的二维Arimoto灰度熵阈值分割[J]. 浙江大学学报(工学版), 2015, 49(9): 1625-1633.
[9] 王媛媛, 郭延恩, 施国全, 韦俊霞, 夏顺仁. 基于声图像的海底地形边界提取算法[J]. 浙江大学学报(工学版), 2015, 49(2): 376-383.
[10] 胡祝华, 赵瑶池, 程杰仁, 彭金莲. 基于改进DRLSE的运动目标分割方法[J]. 浙江大学学报(工学版), 2014, 48(8): 1488-1495.
[11] 杨玉婷, 史玉回, 夏顺仁. 基于讨论机制的头脑风暴优化算法[J]. J4, 2013, 47(10): 1705-1711.
[12] 朱晓恩, 郝欣, 夏顺仁. 基于Levy flight的特征选择算法[J]. J4, 2013, 47(4): 638-643.
[13] 周水琴, 应义斌, 商德胜. 基于形态学的香梨褐变核磁共振成像无损检测[J]. J4, 2012, 46(12): 2141-2145.
[14] 孙创日,甄帅,夏顺仁. 基于吸引区域的多模态脑磁共振图像仿射配准[J]. J4, 2012, 46(9): 1722-1728.
[15] 谢迪, 童若锋, 唐敏, 冯阳. 具有高区分度的视频火焰检测方法[J]. J4, 2012, 46(4): 698-704.