Unsupervised object segmentation with slot attention: validation and challenges on real-world data

Abstract

Traditional deep learning methods often have difficulty preserving object-level semantics because they usually encode the entire scene into a single holistic representation. Object-centric learning (OCL) addresses this limitation by decomposing the scene into individual object representations, making the model more interpretable and composable. Among OCL methods, slot attention shows promising results on synthetic datasets, but its generalization to real-world data remains underexplored. This study investigates the segmentation performance of slot attention in both synthetic and natural environments. We first validate its effectiveness on the CLEVR6 dataset, confirming its ability to learn object-centric representations in structured environments. Building on this, we train the model on the COCO dataset to evaluate its performance on complex real-world scenes. By comparing the results on different datasets, we reveal key challenges such as background clutter, object occlusion, and slot competition, and propose corresponding potential improvement directions. Our results provide empirical insights into the limitations of slot attention in natural image segmentation and lay the foundation for future enhancements of object-centric architectures in practical applications.