Generalized few-shot 3D point cloud segmentation

Abstract

Few-Shot 3D Point Cloud Semantic Segmentation (3D-FS) mitigates the issues of insufficient data annotation and emerging novel classes in real-world scenarios, but it totally ignores the performance on base classes.

In this paper, we address a more practical task, Generalized Few-Shot 3D Point Cloud Semantic Segmentation (3D-GFS), which aims to perform segmentation simultaneously on base classes with adequate samples and novel classes with few samples. Based on the prototypical Base Model, we propose Adaptive Support Enrichment module and Query Aware Representation module to utilize the contextual information of semantic segmentation. The former exploits the co-relationship between base and novel classes in support samples while the latter mines semantic information from query samples. Besides, considering the different embedding spaces, we propose a new training strategy to get a better representation of prototypes. Experiments on S3DIS and ScanNet show that our proposed method outperforms our Base Model and the conventional 3D-FS methods.