References & Citations
Computer Science > Information Retrieval
Title: Positional encoding is not the same as context: A study on positional encoding for Sequential recommendation
(Submitted on 16 May 2024)
Abstract: The expansion of streaming media and e-commerce has led to a boom in recommendation systems, including Sequential recommendation systems, which consider the user's previous interactions with items. In recent years, research has focused on architectural improvements such as transformer blocks and feature extraction that can augment model information. Among these features are context and attributes. Of particular importance is the temporal footprint, which is often considered part of the context and seen in previous publications as interchangeable with positional information. Other publications use positional encodings with little attention to them. In this paper, we analyse positional encodings, showing that they provide relative information between items that are not inferable from the temporal footprint. Furthermore, we evaluate different encodings and how they affect metrics and stability using Amazon datasets. We added some new encodings to help with these problems along the way. We found that we can reach new state-of-the-art results by finding the correct positional encoding, but more importantly, certain encodings stabilise the training.
Submission history
From: Alejo Lopez-Avila [view email][v1] Thu, 16 May 2024 20:42:54 GMT (612kb,D)
Link back to: arXiv, form interface, contact.