Please wait a minute...
Journal of ZheJiang University (Engineering Science)  2026, Vol. 60 Issue (10): 2215-2226    DOI: 10.3785/j.issn.1008-973X.2026.10.014
    
Multi-modal recommendation model for denoising and user preference modeling
Wenyu MA1(),Hongbin XIA1,2,*(),Xiaofeng WANG1,3
1. School of Artificial Intelligence and Computer Science, Jiangnan University, Wuxi 214122, China
2. Jiangsu Key University Laboratory of Software and Media Technology under Human-Computer Cooperation, Jiangnan University, Wuxi 214122, China
3. Pengcheng Laboratory, Shenzhen 518055, China
Download: HTML     PDF(4149KB) HTML
Export: BibTeX | EndNote (RIS)      

Abstract  

Multimodal recommendation improves performance by integrating visual and textual information, but it still faces two major challenges: modality noise interference and insufficient user preference modeling. To address these issues, a denoising and preference-aware multimodal recommendation model, DPRec, was proposed. A dual denoising mechanism was constructed to enhance feature quality. In the frequency domain, signal-level noise was adaptively filtered, while an ID-guided gating unit was introduced to suppress semantic-level redundancy, thereby improving the stability and expressiveness of multimodal representations. For structural modeling, both item-item and user-item graphs were incorporated to capture higher-order relationships and strengthen multimodal semantic propagation. For preference modeling, a dynamic user preference inference mechanism was designed. Long-term interest representations were combined with contextual information, and adaptive weights were assigned to different modalities to achieve personalized representation updates. In addition, a contrastive learning strategy was applied to align collaborative behavior representations and preference representations within a unified semantic space, which enhanced feature consistency and robustness. Experimental results showed that DPRec outperformed baseline methods on four public datasets, achieving an average improvement of approximately 4.22% in NDCG@20.



Key wordsmultimodal recommendation      graph neural networks      modality denoising      user preference modeling      contrastive learning     
Received: 16 August 2025      Published: 28 July 2026
CLC:  TP 391  
Fund:  国家自然科学基金资助项目(61972182).
Corresponding Authors: Hongbin XIA     E-mail: 6233115016@stu.jiangnan.edu.cn;hbxia@163.com
Cite this article:

Wenyu MA,Hongbin XIA,Xiaofeng WANG. Multi-modal recommendation model for denoising and user preference modeling. Journal of ZheJiang University (Engineering Science), 2026, 60(10): 2215-2226.

URL:

https://www.zjujournals.com/eng/10.3785/j.issn.1008-973X.2026.10.014     OR     https://www.zjujournals.com/eng/Y2026/V60/I10/2215


面向去噪与用户偏好建模的多模态推荐模型

多模态推荐通过融合图像与文本信息提升推荐性能,但仍面临模态噪声干扰与用户偏好刻画不足2大挑战. 针对上述问题,提出面向去噪与用户偏好建模的多模态推荐模型(DPRec). 模型构建多模态双重去噪机制,在频域对信号级噪声进行自适应滤除,同时引入基于ID引导的门控单元抑制语义级冗余信息,从而提升模态特征表达质量与稳定性. 在结构建模方面,引入物品-物品图与用户-物品图以刻画高阶关联关系,增强多模态语义传播能力. 在偏好建模方面,设计用户动态偏好推断机制,融合长期兴趣表示与上下文信息,对不同模态分配自适应权重,实现个性化表达更新. 同时结合对比学习策略约束协同行为表示与偏好表示在统一语义空间内对齐,增强特征一致性与表示鲁棒性. 实验结果表明,DPRec在4个公开数据集上优于基线方法,NDCG@20平均提升约4.22%.


关键词: 多模态推荐,  图神经网络,  模态去噪,  用户偏好建模,  对比学习 
Fig.1 Structure of DPRec model
Fig.2 Feature enhancement module
数据集$ |U| $$ |I| $$ |E| $spa/%
Baby19445705016079299.88
Sports355981835729633799.95
Clothing393872303327867799.91
MicroLens981291722870517499.97
Tab.1 Statistics of Amazon dataset
数据集BabySportsClothingMicroLens
R@10R@20N@10N@20R@10R@20N@10N@20R@10R@20N@10N@20R@10R@20N@10N@20
BPR0.03570.05750.01920.02490.04320.06530.02410.02980.01870.02790.01030.01260.06240.09590.03220.0408
LightGCN0.04790.07540.02570.03280.05690.08640.03130.03870.03400.05260.01880.02360.07200.10750.03760.0467
VBPR0.04230.06630.02230.02840.05580.08560.03070.03840.04230.06630.02230.02840.06770.10260.03510.0441
MMGCN0.03780.06150.02000.02610.03700.06050.01930.02540.03780.06150.02000.02610.04210.07010.02070.0279
BM30.05640.08830.03010.03830.06560.09800.03550.04380.04220.06210.02310.02810.06060.09810.03040.0400
MGCN0.06200.09640.03390.04270.07290.11060.03970.04960.06410.09450.03470.04280.07560.11340.03870.0484
DiffMM0.06250.09750.03230.04110.06810.10170.03700.04580.05230.07770.02760.03440.07300.11410.03680.0472
LGMRec0.06440.10020.03490.04400.07200.10680.03900.04800.05550.08280.03020.03710.07480.11320.03900.0489
DA-MRS0.06500.09940.03460.04350.07510.11250.04020.04980.06470.09630.03530.04330.07580.11660.03880.0492
TMLP0.06710.10160.03600.04490.07690.11520.04160.05150.06510.09630.03480.04310.07700.11790.03940.0498
DPRec0.06810.10610.03660.04680.07890.11640.04330.05310.06930.10110.03710.04520.08160.12230.04200.0523
Tab.2 Performance comparison of different methods on four datasets
Fig.3 Module ablation study of DPRec and its five variants
Fig.4 Modal ablation study of DPRec and its two variants
数据集BabySportsClothingMicrolens
Memory/GBTime/(s·epoch?1)Memory/GBTime/(s·epoch?1)Memory/GBTime/(s·epoch?1)Memory/GBTime/(s·epoch?1)
BPR0.860.171.510.351.740.421.521.06
LightGCN0.870.241.520.531.760.581.531.92
BM31.240.352.461.283.021.382.152.27
DiffMM1.982.813.719.604.4210.453.9416.24
DPRec1.861.213.343.573.823.513.3711.87
Tab.3 Efficiency comparison between DPRec and baseline models
LBabySportsClothing
R@20N@20R@20N@20R@20N@20
10.09800.04310.10880.04910.09680.0437
20.10610.04680.11640.05310.10110.0452
30.10330.04580.11090.04100.10040.0455
40.10290.04530.11010.04990.09730.0426
Tab.4 Impact of user-item graph layer depth on model performance
kBabySportsClothing
R@20N@20R@20N@20R@20N@20
100.10410.04610.11610.05230.10080.0445
200.10300.04590.11250.05050.09880.0442
300.10320.04550.10940.04880.09310.0415
400.10280.04530.10800.04830.09190.0409
Tab.5 Impact of Top-k parameter in item-item graph on model performance
Fig.5 Performance comparison of different contrastive learning weights on Baby and Clothing datasets
Fig.6 User and item representations learned by LightGCN
Fig.7 User and item representations learned by DPRec
[1]   ADOMAVICIUS G, TUZHILIN A Toward the next generation of recommender systems: a survey of the state-of-the-art and possible extensions[J]. IEEE Transactions on Knowledge and Data Engineering, 2005, 17 (6): 734- 749
doi: 10.1109/TKDE.2005.99
[2]   HUANG L W, JIANG B T, LV S Y, et al Survey on deep learning based recommender systems[J]. Chinese Journal of Computers, 2018, 41 (7): 1619- 1647
[3]   DING Y, LAI Z, MOK P Y, et al Computational technologies for fashion recommendation: a survey[J]. ACM Computing Surveys, 2024, 56 (5): 1- 45
[4]   XU B B, CEN K T, HUANG J J, et al A survey on graph convolutional neural network[J]. Chinese Journal of Computers, 2020, 43 (5): 755- 780
doi: 10.31219/osf.io/9wk3y
[5]   HE R, MCAULEY J. VBPR: visual Bayesian personalized ranking from implicit feedback [C]// Proceedings of the AAAI Conference on Artificial Intelligence. Phoenix: AAAI Press, 2016: 1444–1449.
[6]   HE X, DENG K, WANG X, et al. LightGCN: simplifying and powering graph convolution network for recommendation [C]// Proceedings of the 43rd International ACM SIGIR Conference on Research and Development in Information Retrieval. [S.l.]: ACM, 2020: 639–648.
[7]   WEI Y, WANG X, NIE L, et al. MMGCN: multi-modal graph convolution network for personalized recommendation of micro-video [C]// Proceedings of the 27th ACM International Conference on Multimedia. Nice: ACM, 2019: 1437–1445.
[8]   WANG Q, WEI Y, YIN J, et al DualGNN: dual graph neural network for multimedia recommendation[J]. IEEE Transactions on Multimedia, 2023, 25: 1074- 1084
doi: 10.1109/TMM.2021.3138298
[9]   ZHANG J, ZHU Y, LIU Q, et al. Mining latent structures for multimedia recommendation [C]// Proceedings of the 29th ACM International Conference on Multimedia. [S.l.]: ACM, 2021: 3872–3880.
[10]   LIU F, CHEN H, CHENG Z, et al. Semantic-guided feature distillation for multimodal recommendation [C]// Proceedings of the 31st ACM International Conference on Multimedia. Ottawa: ACM, 2023: 6567–6575.
[11]   XV G, LI X, XIE R, et al. Improving multi-modal recommender systems by denoising and aligning multi-modal content and user feedback [C]// Proceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining. Barcelona: ACM, 2024: 3645–3656.
[12]   JIANG Y, XIA L, WEI W, et al. DiffMM: multi-modal diffusion model for recommendation [C]// Proceedings of the 32nd ACM International Conference on Multimedia. Melbourne: ACM, 2024: 7591–7599.
[13]   YU P, TAN Z, LU G, et al. Mind individual information! principal graph learning for multimedia recommendation [C]// Proceedings of the AAAI Conference on Artificial Intelligence. Philadelphia: AAAI Press, 2025: 13096–13105.
[14]   YU P, TAN Z, LU G, et al. Multi-view graph convolutional network for multimedia recommendation [C]// Proceedings of the 31st ACM International Conference on Multimedia. Ottawa: ACM, 2023: 6576–6585.
[15]   BAI H, WU L, HOU M, et al. Multimodality invariant learning for multimedia-based new item recommendation [C]// Proceedings of the 47th International ACM SIGIR Conference on Research and Development in Information Retrieval. Washington DC: ACM, 2024: 677–686.
[16]   WU J, WANG X, FENG F, et al. Self-supervised graph learning for recommendation [C]// Proceedings of the 44th International ACM SIGIR Conference on Research and Development in Information Retrieval. [S.l.]: ACM, 2021: 726–735.
[17]   YU J, YIN H, XIA X, et al. Are graph augmentations necessary? simple graph contrastive learning for recommendation [C]// Proceedings of the 45th International ACM SIGIR Conference on Research and Development in Information Retrieval. New York: ACM, 2022: 1294–1303.
[18]   TAO Z, LIU X, XIA Y, et al Self-supervised learning for multimedia recommendation[J]. IEEE Transactions on Multimedia, 2023, 25: 5107- 5116
doi: 10.1109/TMM.2022.3187556
[19]   ZHOU X, ZHOU H, LIU Y, et al. Bootstrap latent representations for multi-modal recommendation [C]// Proceedings of the ACM Web Conference 2023. Austin: ACM, 2023: 845–854.
[20]   XU J, CHEN Z, YANG S, et al MENTOR: multi-level self-supervised learning for multimodal recommendation[J]. Proceedings of the AAAI Conference on Artificial Intelligence, 2025, 39 (12): 12908- 12917
doi: 10.1609/aaai.v39i12.33408
[21]   LIN X, LIU R, CAO Y, et al Contrastive modality-disentangled learning for multimodal recommendation[J]. ACM Transactions on Information Systems, 2025, 43 (3): 1- 31
[22]   VAN DEN OORD A, LI Y, VINYALS O. Representation learning with contrastive predictive coding [EB/OL]. (2018–07–10) [2026–04–11]. https://arxiv.org/abs/1807.03748.
[23]   RENDLE S, FREUDENTHALER C, GANTNER Z, et al. BPR: Bayesian personalized ranking from implicit feedback [EB/OL]. (2012–05–11) [2026–04–11]. https://arxiv.org/abs/1205.2618.
[24]   HE R, MCAULEY J. Ups and downs: modeling the visual evolution of fashion trends with one-class collaborative filtering [C]// Proceedings of the 25th International Conference on World Wide Web. New York: ACM 2016: 507–517.
[25]   NI Y, CHENG Y, LIU X, et al. A content-driven micro-video recommendation dataset at scale [EB/OL]. (2023–09–27) [2026–04–11]. https://arxiv.org/abs/2309.15379.
[26]   ZHOU X. MMRec: simplifying multimodal recommendation [C]// Proceedings of the 5th ACM International Conference on Multimedia in Asia Workshops. Tainan: ACM, 2023: 1–2.
[27]   GUO Z, LI J, LI G, et al LGMRec: local and global graph learning for multimodal recommendation[J]. Proceedings of the AAAI Conference on Artificial Intelligence, 2024, 38 (8): 8454- 8462
[1] Xuemei ZHANG,Ying SUN,Xueying ZHANG. Speech emotion recognition with unsupervised graph contrastive learning[J]. Journal of ZheJiang University (Engineering Science), 2026, 60(4): 782-790.
[2] Yanle WANG,Ruifeng ZHANG,Qiang LI. Graph neural network recommendation model integrating global information and contrastive learning[J]. Journal of ZheJiang University (Engineering Science), 2026, 60(2): 351-359.
[3] Tao XIE,Huili GE,Ning CHEN,Xiaofeng WANG,Yansong LI,Xiaofeng HUANG. Knowledge embedding-enhanced contrastive recommendation model[J]. Journal of ZheJiang University (Engineering Science), 2026, 60(1): 90-98.
[4] Wenhu HUANG,Xing ZHAO,Liang XIE,Haoran LIANG,Ronghua LIANG. Contrastive learning-based sound source localization-guided audio-visual segmentation model[J]. Journal of ZheJiang University (Engineering Science), 2025, 59(9): 1803-1813.
[5] Yue HOU,Qianhui LI,Peng YUAN,Xin ZHANG,Tiantian WANG,Ziwei HAO. Scalable traffic image auto-annotation method based on contrastive learning[J]. Journal of ZheJiang University (Engineering Science), 2025, 59(8): 1634-1643.
[6] Lizhou FENG,Zhichun BAI,Youwei WANG. Dual-channel E-commerce fraud detection method integrating user behavior and review relationships[J]. Journal of ZheJiang University (Engineering Science), 2025, 59(10): 2164-2174.
[7] Ke CHEN,Wenhao ZHANG. Zero-shot object rumor detection based on contrastive learning[J]. Journal of ZheJiang University (Engineering Science), 2024, 58(9): 1790-1800.
[8] Tian-qi ZHOU,Yan YANG,Ji-jie ZHANG,Shao-wei YIN,Zeng-qiang GUO. Graph contrastive learning based on negative-sample-free loss and adaptive augmentation[J]. Journal of ZheJiang University (Engineering Science), 2023, 57(2): 259-266.