Adaptive multi-scale graph and convolutional fusion networks for hyperspectral image classification

Abstract

In recent years, the application of deep learning networks represented by Convolutional Neural Networks (CNN) in hyperspectral image classification has made good progress. Meanwhile, Graph Convolutional Neural Networks (GCN) have also attracted considerable attention by using unlabeled data, broadly and explicitly exploiting correlations between adjacent parcels. However, the CNN limited by the fixed square convolution kernel is not flexible enough to deal with irregular patterns, while the GCN based on superpixel nodes will lose the pixel-level features, and the features extracted by the two networks are always partial. Therefore, it is of great significance to explore a hyperspectral image classification network that combines the advantages of both. In this dissertation, we propose a novel model termed Adaptive Multi-scale Graph and Convolutional Fusion Network (AMGCFN), which includes two sub-networks of multi-scale fully CNN and multi-hop GCN to extract spatial and spectral information respectively. Specifically, multi-scale fully CNN aims to comprehensively capture pixel-level features with different kernel sizes, and an adaptive weighting manner is used to fuse multi-scale features. Multi-hop GCN systematically aggregates contextual information by applying multi-hop graphs in different layers to transform the relationships between nodes, and the multi-head self-attention mechanism is adopted to fuse the features from different multi-hop graph layers. AMGCFN makes full use of multi-scale convolution and graph features, which is conducive to the learning of high-level contextual semantic features. Three benchmark HSI datasets are used to evaluate the performance of this method in our experiments, and the extensive data results show that AMGCFN has better performance and competitiveness than the state-of-the-art methods.