Please use this identifier to cite or link to this item:
|Title:||Initialization matters : regularizing manifold-informed initialization for neural recommendation systems||Authors:||Zhang, Yinan
|Keywords:||Engineering::Computer science and engineering||Issue Date:||2021||Source:||Zhang, Y., Li, B., Liu, Y., Wang, H. & Miao, C. (2021). Initialization matters : regularizing manifold-informed initialization for neural recommendation systems. 27th ACM SIGKDD Conference on Knowledge Discovery & Data Mining (KDD 2021), 2263-2273. https://dx.doi.org/10.1145/3447548.3467338||Project:||AISG-GC-2019-003
|Abstract:||Proper initialization is crucial to the optimization and the generalization of neural networks. However, most existing neural recommendation systems initialize the user and item embeddings randomly. In this work, we propose a new initialization scheme for user and item embeddings called Laplacian Eigenmaps with Popularity-based Regularization for Isolated Data (LEPORID). LEPORID endows the embeddings with information regarding multi-scale neighborhood structures on the data manifold and performs adaptive regularization to compensate for high embedding variance on the tail of the data distribution. Exploiting matrix sparsity, LEPORID embeddings can be computed efficiently. We evaluate LEPORID in a wide range of neural recommendation models. In contrast to the recent surprising finding that the simple K-nearest-neighbor (KNN) method often outperforms neural recommendation systems, we show that existing neural systems initialized with LEPORID often perform on par or better than KNN. To maximize the effects of the initialization, we propose the Dual-Loss Residual Recommendation (DLR2) network, which, when initialized with LEPORID, substantially outperforms both traditional and state-of-the-art neural recommender systems.||URI:||https://hdl.handle.net/10356/153528||ISBN:||9781450383325||DOI:||10.1145/3447548.3467338||Rights:||© 2021 The Owner/Author(s). Publication rights licensed to ACM. All rights reserved. This paper was published in Proceedings of the 27th ACM SIGKDD Conference on Knowledge Discovery & Data Mining (KDD 2021) and is made available with permission of The Owner/Author(s).||Fulltext Permission:||open||Fulltext Availability:||With Fulltext|
|Appears in Collections:||SCSE Conference Papers|
Updated on May 20, 2022
Updated on May 20, 2022
Items in DR-NTU are protected by copyright, with all rights reserved, unless otherwise indicated.