|
|
|
| Multi-modal recommendation model for denoising and user preference modeling |
Wenyu MA1( ),Hongbin XIA1,2,*( ),Xiaofeng WANG1,3 |
1. School of Artificial Intelligence and Computer Science, Jiangnan University, Wuxi 214122, China 2. Jiangsu Key University Laboratory of Software and Media Technology under Human-Computer Cooperation, Jiangnan University, Wuxi 214122, China 3. Pengcheng Laboratory, Shenzhen 518055, China |
|
|
|
Abstract Multimodal recommendation improves performance by integrating visual and textual information, but it still faces two major challenges: modality noise interference and insufficient user preference modeling. To address these issues, a denoising and preference-aware multimodal recommendation model, DPRec, was proposed. A dual denoising mechanism was constructed to enhance feature quality. In the frequency domain, signal-level noise was adaptively filtered, while an ID-guided gating unit was introduced to suppress semantic-level redundancy, thereby improving the stability and expressiveness of multimodal representations. For structural modeling, both item-item and user-item graphs were incorporated to capture higher-order relationships and strengthen multimodal semantic propagation. For preference modeling, a dynamic user preference inference mechanism was designed. Long-term interest representations were combined with contextual information, and adaptive weights were assigned to different modalities to achieve personalized representation updates. In addition, a contrastive learning strategy was applied to align collaborative behavior representations and preference representations within a unified semantic space, which enhanced feature consistency and robustness. Experimental results showed that DPRec outperformed baseline methods on four public datasets, achieving an average improvement of approximately 4.22% in NDCG@20.
|
|
Received: 16 August 2025
Published: 28 July 2026
|
|
|
| Fund: 国家自然科学基金资助项目(61972182). |
|
Corresponding Authors:
Hongbin XIA
E-mail: 6233115016@stu.jiangnan.edu.cn;hbxia@163.com
|
面向去噪与用户偏好建模的多模态推荐模型
多模态推荐通过融合图像与文本信息提升推荐性能,但仍面临模态噪声干扰与用户偏好刻画不足2大挑战. 针对上述问题,提出面向去噪与用户偏好建模的多模态推荐模型(DPRec). 模型构建多模态双重去噪机制,在频域对信号级噪声进行自适应滤除,同时引入基于ID引导的门控单元抑制语义级冗余信息,从而提升模态特征表达质量与稳定性. 在结构建模方面,引入物品-物品图与用户-物品图以刻画高阶关联关系,增强多模态语义传播能力. 在偏好建模方面,设计用户动态偏好推断机制,融合长期兴趣表示与上下文信息,对不同模态分配自适应权重,实现个性化表达更新. 同时结合对比学习策略约束协同行为表示与偏好表示在统一语义空间内对齐,增强特征一致性与表示鲁棒性. 实验结果表明,DPRec在4个公开数据集上优于基线方法,NDCG@20平均提升约4.22%.
关键词:
多模态推荐,
图神经网络,
模态去噪,
用户偏好建模,
对比学习
|
|
| [1] |
ADOMAVICIUS G, TUZHILIN A Toward the next generation of recommender systems: a survey of the state-of-the-art and possible extensions[J]. IEEE Transactions on Knowledge and Data Engineering, 2005, 17 (6): 734- 749
doi: 10.1109/TKDE.2005.99
|
|
|
| [2] |
HUANG L W, JIANG B T, LV S Y, et al Survey on deep learning based recommender systems[J]. Chinese Journal of Computers, 2018, 41 (7): 1619- 1647
|
|
|
| [3] |
DING Y, LAI Z, MOK P Y, et al Computational technologies for fashion recommendation: a survey[J]. ACM Computing Surveys, 2024, 56 (5): 1- 45
|
|
|
| [4] |
XU B B, CEN K T, HUANG J J, et al A survey on graph convolutional neural network[J]. Chinese Journal of Computers, 2020, 43 (5): 755- 780
doi: 10.31219/osf.io/9wk3y
|
|
|
| [5] |
HE R, MCAULEY J. VBPR: visual Bayesian personalized ranking from implicit feedback [C]// Proceedings of the AAAI Conference on Artificial Intelligence. Phoenix: AAAI Press, 2016: 1444–1449.
|
|
|
| [6] |
HE X, DENG K, WANG X, et al. LightGCN: simplifying and powering graph convolution network for recommendation [C]// Proceedings of the 43rd International ACM SIGIR Conference on Research and Development in Information Retrieval. [S.l.]: ACM, 2020: 639–648.
|
|
|
| [7] |
WEI Y, WANG X, NIE L, et al. MMGCN: multi-modal graph convolution network for personalized recommendation of micro-video [C]// Proceedings of the 27th ACM International Conference on Multimedia. Nice: ACM, 2019: 1437–1445.
|
|
|
| [8] |
WANG Q, WEI Y, YIN J, et al DualGNN: dual graph neural network for multimedia recommendation[J]. IEEE Transactions on Multimedia, 2023, 25: 1074- 1084
doi: 10.1109/TMM.2021.3138298
|
|
|
| [9] |
ZHANG J, ZHU Y, LIU Q, et al. Mining latent structures for multimedia recommendation [C]// Proceedings of the 29th ACM International Conference on Multimedia. [S.l.]: ACM, 2021: 3872–3880.
|
|
|
| [10] |
LIU F, CHEN H, CHENG Z, et al. Semantic-guided feature distillation for multimodal recommendation [C]// Proceedings of the 31st ACM International Conference on Multimedia. Ottawa: ACM, 2023: 6567–6575.
|
|
|
| [11] |
XV G, LI X, XIE R, et al. Improving multi-modal recommender systems by denoising and aligning multi-modal content and user feedback [C]// Proceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining. Barcelona: ACM, 2024: 3645–3656.
|
|
|
| [12] |
JIANG Y, XIA L, WEI W, et al. DiffMM: multi-modal diffusion model for recommendation [C]// Proceedings of the 32nd ACM International Conference on Multimedia. Melbourne: ACM, 2024: 7591–7599.
|
|
|
| [13] |
YU P, TAN Z, LU G, et al. Mind individual information! principal graph learning for multimedia recommendation [C]// Proceedings of the AAAI Conference on Artificial Intelligence. Philadelphia: AAAI Press, 2025: 13096–13105.
|
|
|
| [14] |
YU P, TAN Z, LU G, et al. Multi-view graph convolutional network for multimedia recommendation [C]// Proceedings of the 31st ACM International Conference on Multimedia. Ottawa: ACM, 2023: 6576–6585.
|
|
|
| [15] |
BAI H, WU L, HOU M, et al. Multimodality invariant learning for multimedia-based new item recommendation [C]// Proceedings of the 47th International ACM SIGIR Conference on Research and Development in Information Retrieval. Washington DC: ACM, 2024: 677–686.
|
|
|
| [16] |
WU J, WANG X, FENG F, et al. Self-supervised graph learning for recommendation [C]// Proceedings of the 44th International ACM SIGIR Conference on Research and Development in Information Retrieval. [S.l.]: ACM, 2021: 726–735.
|
|
|
| [17] |
YU J, YIN H, XIA X, et al. Are graph augmentations necessary? simple graph contrastive learning for recommendation [C]// Proceedings of the 45th International ACM SIGIR Conference on Research and Development in Information Retrieval. New York: ACM, 2022: 1294–1303.
|
|
|
| [18] |
TAO Z, LIU X, XIA Y, et al Self-supervised learning for multimedia recommendation[J]. IEEE Transactions on Multimedia, 2023, 25: 5107- 5116
doi: 10.1109/TMM.2022.3187556
|
|
|
| [19] |
ZHOU X, ZHOU H, LIU Y, et al. Bootstrap latent representations for multi-modal recommendation [C]// Proceedings of the ACM Web Conference 2023. Austin: ACM, 2023: 845–854.
|
|
|
| [20] |
XU J, CHEN Z, YANG S, et al MENTOR: multi-level self-supervised learning for multimodal recommendation[J]. Proceedings of the AAAI Conference on Artificial Intelligence, 2025, 39 (12): 12908- 12917
doi: 10.1609/aaai.v39i12.33408
|
|
|
| [21] |
LIN X, LIU R, CAO Y, et al Contrastive modality-disentangled learning for multimodal recommendation[J]. ACM Transactions on Information Systems, 2025, 43 (3): 1- 31
|
|
|
| [22] |
VAN DEN OORD A, LI Y, VINYALS O. Representation learning with contrastive predictive coding [EB/OL]. (2018–07–10) [2026–04–11]. https://arxiv.org/abs/1807.03748.
|
|
|
| [23] |
RENDLE S, FREUDENTHALER C, GANTNER Z, et al. BPR: Bayesian personalized ranking from implicit feedback [EB/OL]. (2012–05–11) [2026–04–11]. https://arxiv.org/abs/1205.2618.
|
|
|
| [24] |
HE R, MCAULEY J. Ups and downs: modeling the visual evolution of fashion trends with one-class collaborative filtering [C]// Proceedings of the 25th International Conference on World Wide Web. New York: ACM 2016: 507–517.
|
|
|
| [25] |
NI Y, CHENG Y, LIU X, et al. A content-driven micro-video recommendation dataset at scale [EB/OL]. (2023–09–27) [2026–04–11]. https://arxiv.org/abs/2309.15379.
|
|
|
| [26] |
ZHOU X. MMRec: simplifying multimodal recommendation [C]// Proceedings of the 5th ACM International Conference on Multimedia in Asia Workshops. Tainan: ACM, 2023: 1–2.
|
|
|
| [27] |
GUO Z, LI J, LI G, et al LGMRec: local and global graph learning for multimodal recommendation[J]. Proceedings of the AAAI Conference on Artificial Intelligence, 2024, 38 (8): 8454- 8462
|
|
|
|
Viewed |
|
|
|
Full text
|
|
|
|
|
Abstract
|
|
|
|
|
Cited |
|
|
|
|
| |
Shared |
|
|
|
|
| |
Discussed |
|
|
|
|