Please wait a minute...
浙江大学学报(工学版)  2026, Vol. 60 Issue (10): 2215-2226    DOI: 10.3785/j.issn.1008-973X.2026.10.014
计算机技术与控制工程     
面向去噪与用户偏好建模的多模态推荐模型
马文煜1(),夏鸿斌1,2,*(),王晓锋1,3
1. 江南大学 人工智能与计算机学院,江苏 无锡 214122
2. 江南大学 人机融合软件与媒体技术江苏省高校重点实验室,江苏 无锡 214122
3. 鹏城实验室,广东 深圳 518055
Multi-modal recommendation model for denoising and user preference modeling
Wenyu MA1(),Hongbin XIA1,2,*(),Xiaofeng WANG1,3
1. School of Artificial Intelligence and Computer Science, Jiangnan University, Wuxi 214122, China
2. Jiangsu Key University Laboratory of Software and Media Technology under Human-Computer Cooperation, Jiangnan University, Wuxi 214122, China
3. Pengcheng Laboratory, Shenzhen 518055, China
 全文: PDF(4149 KB)   HTML
摘要:

多模态推荐通过融合图像与文本信息提升推荐性能,但仍面临模态噪声干扰与用户偏好刻画不足2大挑战. 针对上述问题,提出面向去噪与用户偏好建模的多模态推荐模型(DPRec). 模型构建多模态双重去噪机制,在频域对信号级噪声进行自适应滤除,同时引入基于ID引导的门控单元抑制语义级冗余信息,从而提升模态特征表达质量与稳定性. 在结构建模方面,引入物品-物品图与用户-物品图以刻画高阶关联关系,增强多模态语义传播能力. 在偏好建模方面,设计用户动态偏好推断机制,融合长期兴趣表示与上下文信息,对不同模态分配自适应权重,实现个性化表达更新. 同时结合对比学习策略约束协同行为表示与偏好表示在统一语义空间内对齐,增强特征一致性与表示鲁棒性. 实验结果表明,DPRec在4个公开数据集上优于基线方法,NDCG@20平均提升约4.22%.

关键词: 多模态推荐图神经网络模态去噪用户偏好建模对比学习    
Abstract:

Multimodal recommendation improves performance by integrating visual and textual information, but it still faces two major challenges: modality noise interference and insufficient user preference modeling. To address these issues, a denoising and preference-aware multimodal recommendation model, DPRec, was proposed. A dual denoising mechanism was constructed to enhance feature quality. In the frequency domain, signal-level noise was adaptively filtered, while an ID-guided gating unit was introduced to suppress semantic-level redundancy, thereby improving the stability and expressiveness of multimodal representations. For structural modeling, both item-item and user-item graphs were incorporated to capture higher-order relationships and strengthen multimodal semantic propagation. For preference modeling, a dynamic user preference inference mechanism was designed. Long-term interest representations were combined with contextual information, and adaptive weights were assigned to different modalities to achieve personalized representation updates. In addition, a contrastive learning strategy was applied to align collaborative behavior representations and preference representations within a unified semantic space, which enhanced feature consistency and robustness. Experimental results showed that DPRec outperformed baseline methods on four public datasets, achieving an average improvement of approximately 4.22% in NDCG@20.

Key words: multimodal recommendation    graph neural networks    modality denoising    user preference modeling    contrastive learning
收稿日期: 2025-08-16 出版日期: 2026-07-28
CLC:  TP 391  
基金资助: 国家自然科学基金资助项目(61972182).
通讯作者: 夏鸿斌     E-mail: 6233115016@stu.jiangnan.edu.cn;hbxia@163.com
作者简介: 马文煜(2000—),男,硕士生,从事推荐系统研究. orcid.org/0009-0008-4579-2124. E-mail:6233115016@stu.jiangnan.edu.cn
服务  
把本文推荐给朋友
加入引用管理器
E-mail Alert
作者相关文章  
马文煜
夏鸿斌
王晓锋

引用本文:

马文煜,夏鸿斌,王晓锋. 面向去噪与用户偏好建模的多模态推荐模型[J]. 浙江大学学报(工学版), 2026, 60(10): 2215-2226.

Wenyu MA,Hongbin XIA,Xiaofeng WANG. Multi-modal recommendation model for denoising and user preference modeling. Journal of ZheJiang University (Engineering Science), 2026, 60(10): 2215-2226.

链接本文:

https://www.zjujournals.com/eng/CN/10.3785/j.issn.1008-973X.2026.10.014        https://www.zjujournals.com/eng/CN/Y2026/V60/I10/2215

图 1  DPRec模型结构
图 2  特征增强模块
数据集$ |U| $$ |I| $$ |E| $spa/%
Baby19445705016079299.88
Sports355981835729633799.95
Clothing393872303327867799.91
MicroLens981291722870517499.97
表 1  亚马逊数据集信息
数据集BabySportsClothingMicroLens
R@10R@20N@10N@20R@10R@20N@10N@20R@10R@20N@10N@20R@10R@20N@10N@20
BPR0.03570.05750.01920.02490.04320.06530.02410.02980.01870.02790.01030.01260.06240.09590.03220.0408
LightGCN0.04790.07540.02570.03280.05690.08640.03130.03870.03400.05260.01880.02360.07200.10750.03760.0467
VBPR0.04230.06630.02230.02840.05580.08560.03070.03840.04230.06630.02230.02840.06770.10260.03510.0441
MMGCN0.03780.06150.02000.02610.03700.06050.01930.02540.03780.06150.02000.02610.04210.07010.02070.0279
BM30.05640.08830.03010.03830.06560.09800.03550.04380.04220.06210.02310.02810.06060.09810.03040.0400
MGCN0.06200.09640.03390.04270.07290.11060.03970.04960.06410.09450.03470.04280.07560.11340.03870.0484
DiffMM0.06250.09750.03230.04110.06810.10170.03700.04580.05230.07770.02760.03440.07300.11410.03680.0472
LGMRec0.06440.10020.03490.04400.07200.10680.03900.04800.05550.08280.03020.03710.07480.11320.03900.0489
DA-MRS0.06500.09940.03460.04350.07510.11250.04020.04980.06470.09630.03530.04330.07580.11660.03880.0492
TMLP0.06710.10160.03600.04490.07690.11520.04160.05150.06510.09630.03480.04310.07700.11790.03940.0498
DPRec0.06810.10610.03660.04680.07890.11640.04330.05310.06930.10110.03710.04520.08160.12230.04200.0523
表 2  各方法在4个数据集上的性能对比
图 3  DPRec和5个变体的模块消融
图 4  DPRec和2个变体的模态消融
数据集BabySportsClothingMicrolens
Memory/GBTime/(s·epoch?1)Memory/GBTime/(s·epoch?1)Memory/GBTime/(s·epoch?1)Memory/GBTime/(s·epoch?1)
BPR0.860.171.510.351.740.421.521.06
LightGCN0.870.241.520.531.760.581.531.92
BM31.240.352.461.283.021.382.152.27
DiffMM1.982.813.719.604.4210.453.9416.24
DPRec1.861.213.343.573.823.513.3711.87
表 3  DPRec与基线模型运行效率比较
LBabySportsClothing
R@20N@20R@20N@20R@20N@20
10.09800.04310.10880.04910.09680.0437
20.10610.04680.11640.05310.10110.0452
30.10330.04580.11090.04100.10040.0455
40.10290.04530.11010.04990.09730.0426
表 4  用户-物品图层数对模型性能的影响
kBabySportsClothing
R@20N@20R@20N@20R@20N@20
100.10410.04610.11610.05230.10080.0445
200.10300.04590.11250.05050.09880.0442
300.10320.04550.10940.04880.09310.0415
400.10280.04530.10800.04830.09190.0409
表 5  物品-物品图 Top-k参数对模型性能的影响
图 5  Baby和Clothing数据集上不同对比学习权重的性能对比
图 6  LightGCN学到的用户与物品表示
图 7  DPRec学到的用户与物品表示
1 ADOMAVICIUS G, TUZHILIN A Toward the next generation of recommender systems: a survey of the state-of-the-art and possible extensions[J]. IEEE Transactions on Knowledge and Data Engineering, 2005, 17 (6): 734- 749
doi: 10.1109/TKDE.2005.99
2 HUANG L W, JIANG B T, LV S Y, et al Survey on deep learning based recommender systems[J]. Chinese Journal of Computers, 2018, 41 (7): 1619- 1647
3 DING Y, LAI Z, MOK P Y, et al Computational technologies for fashion recommendation: a survey[J]. ACM Computing Surveys, 2024, 56 (5): 1- 45
4 XU B B, CEN K T, HUANG J J, et al A survey on graph convolutional neural network[J]. Chinese Journal of Computers, 2020, 43 (5): 755- 780
doi: 10.31219/osf.io/9wk3y
5 HE R, MCAULEY J. VBPR: visual Bayesian personalized ranking from implicit feedback [C]// Proceedings of the AAAI Conference on Artificial Intelligence. Phoenix: AAAI Press, 2016: 1444–1449.
6 HE X, DENG K, WANG X, et al. LightGCN: simplifying and powering graph convolution network for recommendation [C]// Proceedings of the 43rd International ACM SIGIR Conference on Research and Development in Information Retrieval. [S.l.]: ACM, 2020: 639–648.
7 WEI Y, WANG X, NIE L, et al. MMGCN: multi-modal graph convolution network for personalized recommendation of micro-video [C]// Proceedings of the 27th ACM International Conference on Multimedia. Nice: ACM, 2019: 1437–1445.
8 WANG Q, WEI Y, YIN J, et al DualGNN: dual graph neural network for multimedia recommendation[J]. IEEE Transactions on Multimedia, 2023, 25: 1074- 1084
doi: 10.1109/TMM.2021.3138298
9 ZHANG J, ZHU Y, LIU Q, et al. Mining latent structures for multimedia recommendation [C]// Proceedings of the 29th ACM International Conference on Multimedia. [S.l.]: ACM, 2021: 3872–3880.
10 LIU F, CHEN H, CHENG Z, et al. Semantic-guided feature distillation for multimodal recommendation [C]// Proceedings of the 31st ACM International Conference on Multimedia. Ottawa: ACM, 2023: 6567–6575.
11 XV G, LI X, XIE R, et al. Improving multi-modal recommender systems by denoising and aligning multi-modal content and user feedback [C]// Proceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining. Barcelona: ACM, 2024: 3645–3656.
12 JIANG Y, XIA L, WEI W, et al. DiffMM: multi-modal diffusion model for recommendation [C]// Proceedings of the 32nd ACM International Conference on Multimedia. Melbourne: ACM, 2024: 7591–7599.
13 YU P, TAN Z, LU G, et al. Mind individual information! principal graph learning for multimedia recommendation [C]// Proceedings of the AAAI Conference on Artificial Intelligence. Philadelphia: AAAI Press, 2025: 13096–13105.
14 YU P, TAN Z, LU G, et al. Multi-view graph convolutional network for multimedia recommendation [C]// Proceedings of the 31st ACM International Conference on Multimedia. Ottawa: ACM, 2023: 6576–6585.
15 BAI H, WU L, HOU M, et al. Multimodality invariant learning for multimedia-based new item recommendation [C]// Proceedings of the 47th International ACM SIGIR Conference on Research and Development in Information Retrieval. Washington DC: ACM, 2024: 677–686.
16 WU J, WANG X, FENG F, et al. Self-supervised graph learning for recommendation [C]// Proceedings of the 44th International ACM SIGIR Conference on Research and Development in Information Retrieval. [S.l.]: ACM, 2021: 726–735.
17 YU J, YIN H, XIA X, et al. Are graph augmentations necessary? simple graph contrastive learning for recommendation [C]// Proceedings of the 45th International ACM SIGIR Conference on Research and Development in Information Retrieval. New York: ACM, 2022: 1294–1303.
18 TAO Z, LIU X, XIA Y, et al Self-supervised learning for multimedia recommendation[J]. IEEE Transactions on Multimedia, 2023, 25: 5107- 5116
doi: 10.1109/TMM.2022.3187556
19 ZHOU X, ZHOU H, LIU Y, et al. Bootstrap latent representations for multi-modal recommendation [C]// Proceedings of the ACM Web Conference 2023. Austin: ACM, 2023: 845–854.
20 XU J, CHEN Z, YANG S, et al MENTOR: multi-level self-supervised learning for multimodal recommendation[J]. Proceedings of the AAAI Conference on Artificial Intelligence, 2025, 39 (12): 12908- 12917
doi: 10.1609/aaai.v39i12.33408
21 LIN X, LIU R, CAO Y, et al Contrastive modality-disentangled learning for multimodal recommendation[J]. ACM Transactions on Information Systems, 2025, 43 (3): 1- 31
22 VAN DEN OORD A, LI Y, VINYALS O. Representation learning with contrastive predictive coding [EB/OL]. (2018–07–10) [2026–04–11]. https://arxiv.org/abs/1807.03748.
23 RENDLE S, FREUDENTHALER C, GANTNER Z, et al. BPR: Bayesian personalized ranking from implicit feedback [EB/OL]. (2012–05–11) [2026–04–11]. https://arxiv.org/abs/1205.2618.
24 HE R, MCAULEY J. Ups and downs: modeling the visual evolution of fashion trends with one-class collaborative filtering [C]// Proceedings of the 25th International Conference on World Wide Web. New York: ACM 2016: 507–517.
25 NI Y, CHENG Y, LIU X, et al. A content-driven micro-video recommendation dataset at scale [EB/OL]. (2023–09–27) [2026–04–11]. https://arxiv.org/abs/2309.15379.
26 ZHOU X. MMRec: simplifying multimodal recommendation [C]// Proceedings of the 5th ACM International Conference on Multimedia in Asia Workshops. Tainan: ACM, 2023: 1–2.
27 GUO Z, LI J, LI G, et al LGMRec: local and global graph learning for multimodal recommendation[J]. Proceedings of the AAAI Conference on Artificial Intelligence, 2024, 38 (8): 8454- 8462
[1] 高帅帅,魏诚,惠飞,张竟成,宋汉辰. 基于时空特征与交互关系建模的换道意图预测[J]. 浙江大学学报(工学版), 2026, 60(9): 2031-2041.
[2] 范业博,李逸成,刘勇,张薇. 面向图神经网络谣言检测器的高效攻击模型[J]. 浙江大学学报(工学版), 2026, 60(8): 1739-1748.
[3] 刘慧雪,刘新倩,王帅,赵川,丁建国. 融合日志语义与图对抗学习的APT检测框架[J]. 浙江大学学报(工学版), 2026, 60(8): 1749-1759.
[4] 郑文军,黎志昆,韩守飞. 知识增强图Transformer的方面级情感分析[J]. 浙江大学学报(工学版), 2026, 60(6): 1269-1276.
[5] 张雪梅,孙颖,张雪英. 基于无监督图对比学习的语音情感识别[J]. 浙江大学学报(工学版), 2026, 60(4): 782-790.
[6] 陈文强,冯琳越,王东丹,顾玉磊,赵轩. 融合动态风险图与多变量注意力机制的车辆轨迹预测模型[J]. 浙江大学学报(工学版), 2026, 60(3): 455-467.
[7] 王彦乐,张瑞峰,李锵. 融合全局信息和对比学习的图神经网络推荐模型[J]. 浙江大学学报(工学版), 2026, 60(2): 351-359.
[8] 谢涛,葛慧丽,陈宁,汪晓锋,李延松,黄晓峰. 知识嵌入增强的对比推荐模型[J]. 浙江大学学报(工学版), 2026, 60(1): 90-98.
[9] 黄文湖,赵邢,谢亮,梁浩然,梁荣华. 基于对比学习的声源定位引导视听分割模型[J]. 浙江大学学报(工学版), 2025, 59(9): 1803-1813.
[10] 侯越,李前辉,袁鹏,张鑫,王甜甜,郝紫微. 基于对比学习的可扩展交通图像自动标注方法[J]. 浙江大学学报(工学版), 2025, 59(8): 1634-1643.
[11] 凤丽洲,白至纯,王友卫. 融合用户行为与评论关系的双通道电商欺诈检测方法[J]. 浙江大学学报(工学版), 2025, 59(10): 2164-2174.
[12] 陈珂,张文浩. 基于对比学习的零样本对象谣言检测[J]. 浙江大学学报(工学版), 2024, 58(9): 1790-1800.
[13] 刘议丹,朱小飞,尹雅博. 基于异质图卷积神经网络的论点对抽取模型[J]. 浙江大学学报(工学版), 2024, 58(5): 900-907.
[14] 姚鑫骅,于涛,封森文,马梓健,栾丛丛,沈洪垚. 基于图神经网络的零件机加工特征识别方法[J]. 浙江大学学报(工学版), 2024, 58(2): 349-359.
[15] 张晗. 用于多元时间序列预测的图神经网络模型[J]. 浙江大学学报(工学版), 2024, 58(12): 2500-2509.