Graph-based model on limited dataset: land cover semantic segmentation in remote sensing image analysis

Abstract

Land cover semantic segmentation in remote sensing image analysis is essential for various applications. However, the success of deep learning models like convolutional neural networks (CNN) relies on large-scale datasets, which can be challenging to acquire. This project investigates the generalizability of graph convolutional networks (GCNs) when trained on limited datasets for land cover semantic segmentation tasks. To address the problem of limited data, a combination of a pre-trained CNN (ResNet50) and a GCN architecture is implemented, adopting a semi-supervised learning, where only a subset of nodes has ground truth labels. The feature extraction is performed by ResNet50, and the GCN architecture motivated by Thomas Kipf and Max Welling processes the extracted features. The graph construction is based on adjacency matrix and proximity-based relationships between nodes. During the implementation phase, several challenges were faced, including a persistent error during the training phase, which required a workaround involving interrupting the training process. This led to the evaluation of the model using the half-trained state. The results show a relationship between the training duration and the quality of the predicted masks and mean intersection over union (mIoU) scores, indicating that the semi-supervised method may have contributed to the observed outcome. In conclusion, this project demonstrates the potential of combining pre-trained CNNs and GCNs for land cover semantic segmentation with limited datasets. However, the implementation faced challenges due to memory constraints, highlighting the importance of good data structure knowledge and well-coded programs. Future work can explore transfer learning and experimenting with the subset of nodes chosen to improve generalizability.