Unsupervised graph-level representation learning

Abstract

Graph-level representation learning has been crucial for its ability to interpret unstructured graph data, such as social network and biological molecules. Unsupervised clustering in particular, has gained increasing attention for the ability to produce useful representations for downstream task without manually labeled data. In this report, I am presenting enhancements to the current state-of-the-art unsupervised graph clustering model: Dual Contrastive Graph-Level Clustering model(DCGLC), aimed at producing more insightful graph representations, ultimately enhancing the clustering performance. I replaced the original augmentation method that generates a random augmentation per graph, to generate two augmentations, diversifying the pseudo positive pair used in contrastive learning. I added a knowledge distillation function, in which a momentum updated ’teacher’ layer provides a stable, slowly changing target. As the model receives information from this stable teacher, it potentially reduces training noise and prevents the exploding gradient problem. I also included a local-global mutual information objective, aimed at aligning node-level(local) and graph-level(global) representations, ensuring the learned representation from the same graph are consistent, thereby improving the embeddings. Preliminary experiments on benchmark datasets have showed improvements in clustering accuracy (ACC), normalized mutual information (NMI), and adjusted rand index (ARI).