Please use this identifier to cite or link to this item:
Title: Initialization matters : regularizing manifold-informed initialization for neural recommendation systems
Authors: Zhang, Yinan
Li, Boyang
Liu, Yong
Wang, Hao
Miao, Chunyan
Keywords: Engineering::Computer science and engineering
Issue Date: 2021
Source: Zhang, Y., Li, B., Liu, Y., Wang, H. & Miao, C. (2021). Initialization matters : regularizing manifold-informed initialization for neural recommendation systems. 27th ACM SIGKDD Conference on Knowledge Discovery & Data Mining (KDD 2021), 2263-2273.
Project: AISG-GC-2019-003
NRF-NRFI05- 2019-0002
Abstract: Proper initialization is crucial to the optimization and the generalization of neural networks. However, most existing neural recommendation systems initialize the user and item embeddings randomly. In this work, we propose a new initialization scheme for user and item embeddings called Laplacian Eigenmaps with Popularity-based Regularization for Isolated Data (LEPORID). LEPORID endows the embeddings with information regarding multi-scale neighborhood structures on the data manifold and performs adaptive regularization to compensate for high embedding variance on the tail of the data distribution. Exploiting matrix sparsity, LEPORID embeddings can be computed efficiently. We evaluate LEPORID in a wide range of neural recommendation models. In contrast to the recent surprising finding that the simple K-nearest-neighbor (KNN) method often outperforms neural recommendation systems, we show that existing neural systems initialized with LEPORID often perform on par or better than KNN. To maximize the effects of the initialization, we propose the Dual-Loss Residual Recommendation (DLR2) network, which, when initialized with LEPORID, substantially outperforms both traditional and state-of-the-art neural recommender systems.
ISBN: 9781450383325
DOI: 10.1145/3447548.3467338
Rights: © 2021 The Owner/Author(s). Publication rights licensed to ACM. All rights reserved. This paper was published in Proceedings of the 27th ACM SIGKDD Conference on Knowledge Discovery & Data Mining (KDD 2021) and is made available with permission of The Owner/Author(s).
Fulltext Permission: open
Fulltext Availability: With Fulltext
Appears in Collections:SCSE Conference Papers

Files in This Item:
File Description SizeFormat 
KDD_2021_arXiv (1).pdf896.75 kBAdobe PDFView/Open

Page view(s)

Updated on May 20, 2022

Download(s) 50

Updated on May 20, 2022

Google ScholarTM




Items in DR-NTU are protected by copyright, with all rights reserved, unless otherwise indicated.