Abstract

Modeling users' dynamic preferences from their historical behaviors is challenging and crucial for recommendation systems. Previous methods employ sequential neural networks to encode users' historical interactions from left to right into hidden representations for making recommendations. Despite their effectiveness, we argue that such left-to-right unidirectional models are sub-optimal due to the limitations including: \begin enumerate* [label=series\itshape\alph*\upshape)] \item unidirectional architectures restrict the power of hidden representation in users' behavior sequences; \item they often assume a rigidly ordered sequence which is not always practical. \end enumerate* To address these limitations, we proposed a sequential recommendation model called BERT4Rec, which employs the deep bidirectional self-attention to model user behavior sequences. To avoid the information leakage and efficiently train the bidirectional model, we adopt the Cloze objective to sequential recommendation, predicting the random masked items in the sequence by jointly conditioning on their left and right context. In this way, we learn a bidirectional representation model to make recommendations by allowing each item in user historical behaviors to fuse information from both left and right sides. Extensive experiments on four benchmark datasets show that our model outperforms various state-of-the-art sequential models consistently.

Keywords

Computer scienceENCODESequence (biology)Benchmark (surveying)Representation (politics)Artificial intelligenceContext (archaeology)Expressive powerMachine learningTheoretical computer science

Affiliated Institutions

Related Publications

Like like alike

Targeting interest to match a user with services (e.g. news, products, games, advertisements) and predicting friendship to build connections among users are two fundamental task...

2011 366 citations

Finding Structure in Time

Time underlies many interesting human behaviors. Thus, the question of how to represent time in connectionist models is very important. One approach is to represent time implici...

1990 Cognitive Science 10427 citations

Publication Info

Year
2019
Type
article
Pages
1441-1450
Citations
1977
Access
Closed

Social Impact

Altmetric
PlumX Metrics

Social media, news, blog, policy document mentions

Citation Metrics

1977
OpenAlex
353
Influential

Cite This

Fei Sun, Jun Liu, Jian Wu et al. (2019). BERT4Rec. Proceedings of the 28th ACM International Conference on Information and Knowledge Management , 1441-1450. https://doi.org/10.1145/3357384.3357895

Identifiers

DOI
10.1145/3357384.3357895
arXiv
1904.06690

Data Quality

Data completeness: 79%