Denoising-Guided Deep Reinforcement Learning for Social Recommendation
D
e
n
o
i
s
i
n
g
-
G
u
i
d
e
d
D
e
e
p
R
e
i
n
f
o
r
c
e
m
e
n
t
L
e
a
r
n
i
n
g
F
o
r
S
o
c
i
a
l
R
e
c
o
m
m
e
n
d
a
t
i
o
n
Q
i
h
a
n
D
u
,
L
i
Y
u
*
,
H
u
i
y
u
a
n
L
i
,
Y
o
u
f
a
n
g
L
e
n
g
,
N
i
n
g
r
u
i
O
u
a
n
d
J
u
n
y
a
o
X
i
a
n
g
R
e
n
m
i
n
U
n
i
v
e
r
s
i
t
y
o
f
C
h
i
n
a
,
S
c
h
o
o
l
o
f
I
n
f
o
r
m
a
t
i
o
n@ruc.edu.cnduqihan
Outline
Introduction
Method: DRL4So
Experimental Results
Conclusion
Introduction
What is social recommendation ?
Example: How to recommend Item 5 to User 1 ?
①
User 4
is a social friend of
User 1
.
②
User 4
has interacted with
Item 5
, while
User 1
has not.
③
Recommend
Item 5
to
User 1
.
Think: Is User 4 always reliable ?
Introduction
Main problem.
Social friends are
NOT
always reliable, e.g., For
the
target product
, consider
3 social friends
rather than all.
Our idea.
Intuitively, it is necessary to automatically
mask
friends
(i.e., noise) who are
irrelevant
to the target product before
modeling the social relationships.
Introduction
New challenge of social recommendation.
How to denoise social friends
without
the relevance annotation between
social friends
&
target products
?
Introduction
Main contribution.
We propose a Denoising-guided deep Reinforcement Learning framework for Social
recommendation (called DRL4So).
DRL4So can automatically
mask
noise
social friends
to improve the performance of social
recommendation.
Our optimization object is:
the maximum positive utility of social denoising
, and we carefully
design
the reward function
to satisfy this object.
Method: DRL4So
Model Summary.
The
social denoiser
(Agent) observes the target user's
preferences on the target product
(state) and
decides to
mask or retain
a social friend (action);
the recommender
(Environment) generates products
and calculates the
recommendation probability
(reward) of the target product before and after denoising.
Method: DRL4So
Method: DRL4So
Method: DRL4So
Experimental Results
Experimental Results
Overall Comparison.
Baselines. Regularization Matrix Factorization & Attention mechanism
& Graph Neural Network & RL-based method
From Table 2, DRL4So outperforms the baselines on all datasets and
metrics.
RQ1: Does social denoiser really helpful ?
A variant without social denoiser: DRL4So-
From Table 2, we find the performance of DRL4So- decrease sharply.
The ablation study supports the effectiveness of social denoiser.
Experimental Results
Conclusions
A Challenge:
A challenge about social denoising for social recommendation systems.
An Object:
Maximize the incremental recommendation probability before and after social denoising.
A Framework:
A reinforcement learning-based social denoising framework for better recommendation.
T
H
A
N
K
S
2022.05.11
This research introduces a Denoising-Guided Deep Reinforcement Learning framework, DRL4So, for enhancing social recommendation systems. By automatically masking noise from social friends to improve recommendation performance, this framework focuses on maximizing the positive utility of social denoising. The study tackles the challenge of denoising social friends without known relevance annotations between friends and target products. Through a carefully designed reward function, the proposed framework aims to optimize social recommendation efficiency and accuracy.
Download Presentation
Please find below an Image/Link to download the presentation.
The content on the website is provided AS IS for your information and personal use only. It may not be sold, licensed, or shared on other websites without obtaining consent from the author. Download presentation by click this link. If you encounter any issues during the download, it is possible that the publisher has removed the file from their server.
E N D
Presentation Transcript
Denoising-Guided Deep Reinforcement Learning For Social Recommendation Qihan Du, Li Yu*, Huiyuan Li, Youfang Leng, Ningrui Ou and Junyao Xiang Renmin University of China, School of Information duqihan@ruc.edu.cn
Outline Introduction 1 Method: DRL4So 2 Experimental Results 3 Conclusion 4 Denoising-Guided Deep Reinforcement Learning For Social Recommendation 1/13 Qihan Du May 11, 2022 ICASSP 2022
Introduction What is social recommendation ? Item 1 Item 2 Item 3 Item 4 Item 5 Social network getUserInfo Items Recommender System Users-Items Interaction Social media Example: How to recommend Item 5 to User 1 ? Users User 4 is a social friend of User 1. User 1 User 3 User 4 User 2 User 4 has interacted with Item 5, while User 1 has not. Users-Users Social relation Recommend Item 5 to User 1. Think: Is User 4 always reliable ? Denoising-Guided Deep Reinforcement Learning For Social Recommendation 2/13 Qihan Du May 11, 2022 ICASSP 2022
Introduction Main problem. Social friends are NOT always reliable, e.g., For the target product, consider 3 social friends rather than all. Our idea. Intuitively, it is necessary to automatically mask friends (i.e., noise) who are irrelevant to the target product before modeling the social relationships. Denoising-Guided Deep Reinforcement Learning For Social Recommendation 3/13 Qihan Du May 11, 2022 ICASSP 2022
Introduction New challenge of social recommendation. How to denoise social friends without the relevance annotation between social friends & target products ? Social friends Target product I know! How? Denoising-Guided Deep Reinforcement Learning For Social Recommendation 4/13 Qihan Du May 11, 2022 ICASSP 2022
Introduction Main contribution. We propose a Denoising-guided deep Reinforcement Learning framework for Social recommendation (called DRL4So). DRL4So can automatically mask noise social friends to improve the performance of social recommendation. Our optimization object is: the maximum positive utility of social denoising, and we carefully design the reward function to satisfy this object. Denoising-Guided Deep Reinforcement Learning For Social Recommendation 5/13 Qihan Du May 11, 2022 ICASSP 2022
Method: DRL4So Problem Formulation. User set ? = {?1, ,??}, Item set ? = {?1, ,??}. Interaction matrix ? ?? ?. Social relation matrix ? ?? |?|. Sequential recommendation task: Input: A sequence of items ?1,, ,?? that the target user has interacted before time ?. Output: the recommended item ??+1at time ? + 1. Denoising-Guided Deep Reinforcement Learning For Social Recommendation 6/13 Qihan Du May 11, 2022 ICASSP 2022
Method: DRL4So Model Summary. The social denoiser (Agent) observes the target user's preferences on the target product (state) and decides to mask or retain a social friend (action); the recommender (Environment) generates products and calculates the recommendation probability (reward) of the target product before and after denoising. Denoising-Guided Deep Reinforcement Learning For Social Recommendation 7/13 Qihan Du May 11, 2022 ICASSP 2022
Method: DRL4So Agent (Social Denoiser). ?? ? State encoder. Get the state vector from the current preference and the target product. ?= ??? ?1,, ,?? ??= ????? ???????( ?,??) ?= 0) or retain (?? ?= 1) social friends ?? ?1,, ,?? for the target user. Actor. Mask (?? ??,??,?? ?= ????? ? ?= {0,1} ?? ?,?,? ,?? ?? ??= ????? ?? ?? Critic. Estimate the expected return of the state-action pair. ? ??,?? = ?????? ???????(??,??) Denoising-Guided Deep Reinforcement Learning For Social Recommendation 8/13 Qihan Du May 11, 2022 ICASSP 2022
Method: DRL4So Environment (Recommender). Recommendation module. Generate the product with highest score as the target product ??. ??????= ?( ? ??) Reward function. The positive utility on recommended probability before and after denoising. ? ??,?? = ? ??????? ? ???????? = ? ?? ?? ?(?? ??) where ??= ? ??+ 1 ? ?means the target user s fusion preference. and ??= ????????? ???????? ?????? ??????? ; ??= ?????????(?????? ???????) Denoising-Guided Deep Reinforcement Learning For Social Recommendation 9/13 Qihan Du May 11, 2022 ICASSP 2022
Experimental Results Dataset. LastFM, Ciao and Epinions. Evaluation Metrics. HR@K , NDCG@K. Experimental Design. Overall Comparison RQ1: Does social denoiser really helpful ? RQ2: What is the effect of the size of candidate set ?(?) ? RQ3: What is the effect of hyper-parameter ? ? Denoising-Guided Deep Reinforcement Learning For Social Recommendation 10/13 Qihan Du May 11, 2022 ICASSP 2022
Experimental Results Overall Comparison. Baselines. Regularization Matrix Factorization & Attention mechanism & Graph Neural Network & RL-based method From Table 2, DRL4So outperforms the baselines on all datasets and metrics. RQ1: Does social denoiser really helpful ? A variant without social denoiser: DRL4So- From Table 2, we find the performance of DRL4So- decrease sharply. The ablation study supports the effectiveness of social denoiser. Denoising-Guided Deep Reinforcement Learning For Social Recommendation 11/13 Qihan Du May 11, 2022 ICASSP 2022
Experimental Results RQ2: What is the effect of the size of candidate set ?(?) ? Target products ?1, ,??are picked up from the ?(?) . We test ? = {100,200,500,1000}. The optimal size of the ?(?) increases as the larger datasets. RQ3: What is the effect of hyper-parameter ? ? The lower -> the user s personalized preferences. the higher -> the influence of social networks. ??= ? ??+ 1 ? ? the range of ? [0.2,0.6] tends to do relatively well . Denoising-Guided Deep Reinforcement Learning For Social Recommendation 12/13 Qihan Du May 11, 2022 ICASSP 2022
Conclusions A Challenge: A challenge about social denoising for social recommendation systems. An Object: Maximize the incremental recommendation probability before and after social denoising. A Framework: A reinforcement learning-based social denoising framework for better recommendation. Denoising-Guided Deep Reinforcement Learning For Social Recommendation 13/13 Qihan Du May 11, 2022 ICASSP 2022
THANKS 2022.05.11
