Denoising-Guided Deep Reinforcement Learning for Social Recommendation
This research introduces a Denoising-Guided Deep Reinforcement Learning framework, DRL4So, for enhancing social recommendation systems. By automatically masking noise from social friends to improve recommendation performance, this framework focuses on maximizing the positive utility of social denoising. The study tackles the challenge of denoising social friends without known relevance annotations between friends and target products. Through a carefully designed reward function, the proposed framework aims to optimize social recommendation efficiency and accuracy.
Download Presentation
Please find below an Image/Link to download the presentation.
The content on the website is provided AS IS for your information and personal use only. It may not be sold, licensed, or shared on other websites without obtaining consent from the author. Download presentation by click this link. If you encounter any issues during the download, it is possible that the publisher has removed the file from their server.
E N D
Presentation Transcript
Denoising-Guided Deep Reinforcement Learning For Social Recommendation Qihan Du, Li Yu*, Huiyuan Li, Youfang Leng, Ningrui Ou and Junyao Xiang Renmin University of China, School of Information duqihan@ruc.edu.cn
Outline Introduction 1 Method: DRL4So 2 Experimental Results 3 Conclusion 4 Denoising-Guided Deep Reinforcement Learning For Social Recommendation 1/13 Qihan Du May 11, 2022 ICASSP 2022
Introduction What is social recommendation ? Item 1 Item 2 Item 3 Item 4 Item 5 Social network getUserInfo Items Recommender System Users-Items Interaction Social media Example: How to recommend Item 5 to User 1 ? Users User 4 is a social friend of User 1. User 1 User 3 User 4 User 2 User 4 has interacted with Item 5, while User 1 has not. Users-Users Social relation Recommend Item 5 to User 1. Think: Is User 4 always reliable ? Denoising-Guided Deep Reinforcement Learning For Social Recommendation 2/13 Qihan Du May 11, 2022 ICASSP 2022
Introduction Main problem. Social friends are NOT always reliable, e.g., For the target product, consider 3 social friends rather than all. Our idea. Intuitively, it is necessary to automatically mask friends (i.e., noise) who are irrelevant to the target product before modeling the social relationships. Denoising-Guided Deep Reinforcement Learning For Social Recommendation 3/13 Qihan Du May 11, 2022 ICASSP 2022
Introduction New challenge of social recommendation. How to denoise social friends without the relevance annotation between social friends & target products ? Social friends Target product I know! How? Denoising-Guided Deep Reinforcement Learning For Social Recommendation 4/13 Qihan Du May 11, 2022 ICASSP 2022
Introduction Main contribution. We propose a Denoising-guided deep Reinforcement Learning framework for Social recommendation (called DRL4So). DRL4So can automatically mask noise social friends to improve the performance of social recommendation. Our optimization object is: the maximum positive utility of social denoising, and we carefully design the reward function to satisfy this object. Denoising-Guided Deep Reinforcement Learning For Social Recommendation 5/13 Qihan Du May 11, 2022 ICASSP 2022
Method: DRL4So Problem Formulation. User set ? = {?1, ,??}, Item set ? = {?1, ,??}. Interaction matrix ? ?? ?. Social relation matrix ? ?? |?|. Sequential recommendation task: Input: A sequence of items ?1,, ,?? that the target user has interacted before time ?. Output: the recommended item ??+1at time ? + 1. Denoising-Guided Deep Reinforcement Learning For Social Recommendation 6/13 Qihan Du May 11, 2022 ICASSP 2022
Method: DRL4So Model Summary. The social denoiser (Agent) observes the target user's preferences on the target product (state) and decides to mask or retain a social friend (action); the recommender (Environment) generates products and calculates the recommendation probability (reward) of the target product before and after denoising. Denoising-Guided Deep Reinforcement Learning For Social Recommendation 7/13 Qihan Du May 11, 2022 ICASSP 2022
Method: DRL4So Agent (Social Denoiser). ?? ? State encoder. Get the state vector from the current preference and the target product. ?= ??? ?1,, ,?? ??= ????? ???????( ?,??) ?= 0) or retain (?? ?= 1) social friends ?? ?1,, ,?? for the target user. Actor. Mask (?? ??,??,?? ?= ????? ? ?= {0,1} ?? ?,?,? ,?? ?? ??= ????? ?? ?? Critic. Estimate the expected return of the state-action pair. ? ??,?? = ?????? ???????(??,??) Denoising-Guided Deep Reinforcement Learning For Social Recommendation 8/13 Qihan Du May 11, 2022 ICASSP 2022
Method: DRL4So Environment (Recommender). Recommendation module. Generate the product with highest score as the target product ??. ??????= ?( ? ??) Reward function. The positive utility on recommended probability before and after denoising. ? ??,?? = ? ??????? ? ???????? = ? ?? ?? ?(?? ??) where ??= ? ??+ 1 ? ?means the target user s fusion preference. and ??= ????????? ???????? ?????? ??????? ; ??= ?????????(?????? ???????) Denoising-Guided Deep Reinforcement Learning For Social Recommendation 9/13 Qihan Du May 11, 2022 ICASSP 2022
Experimental Results Dataset. LastFM, Ciao and Epinions. Evaluation Metrics. HR@K , NDCG@K. Experimental Design. Overall Comparison RQ1: Does social denoiser really helpful ? RQ2: What is the effect of the size of candidate set ?(?) ? RQ3: What is the effect of hyper-parameter ? ? Denoising-Guided Deep Reinforcement Learning For Social Recommendation 10/13 Qihan Du May 11, 2022 ICASSP 2022
Experimental Results Overall Comparison. Baselines. Regularization Matrix Factorization & Attention mechanism & Graph Neural Network & RL-based method From Table 2, DRL4So outperforms the baselines on all datasets and metrics. RQ1: Does social denoiser really helpful ? A variant without social denoiser: DRL4So- From Table 2, we find the performance of DRL4So- decrease sharply. The ablation study supports the effectiveness of social denoiser. Denoising-Guided Deep Reinforcement Learning For Social Recommendation 11/13 Qihan Du May 11, 2022 ICASSP 2022
Experimental Results RQ2: What is the effect of the size of candidate set ?(?) ? Target products ?1, ,??are picked up from the ?(?) . We test ? = {100,200,500,1000}. The optimal size of the ?(?) increases as the larger datasets. RQ3: What is the effect of hyper-parameter ? ? The lower -> the user s personalized preferences. the higher -> the influence of social networks. ??= ? ??+ 1 ? ? the range of ? [0.2,0.6] tends to do relatively well . Denoising-Guided Deep Reinforcement Learning For Social Recommendation 12/13 Qihan Du May 11, 2022 ICASSP 2022
Conclusions A Challenge: A challenge about social denoising for social recommendation systems. An Object: Maximize the incremental recommendation probability before and after social denoising. A Framework: A reinforcement learning-based social denoising framework for better recommendation. Denoising-Guided Deep Reinforcement Learning For Social Recommendation 13/13 Qihan Du May 11, 2022 ICASSP 2022
THANKS 2022.05.11
