Content-dependency reduction with multi-task learning in blind stitched panoramic image quality assessment

Abstract

In this work, we investigate deep learning based solutions to blind quality assessment of stitched panoramic images (SPI). The main problem to tackle is that the ground truth data is usually insufficient. As a result, the learned model can easily overfit data with specific content. Because most distortions of SPIs lie within local regions, the problem cannot be alleviated by commonly-used patch-wise training, which assumes local quality equals global quality. We propose a multi-task learning strategy which encourages learned representation to be less dependent on image content. A siamese network with two weight-shared CNN branches is trained to simultaneously compare the quality of two images of the same scene and predict the quality score of each image. Since two images of the same scene are processed by the same CNN, the CNN tends to find their quality differences instead of content differences under the constraint of the quality ranking objective. Because two tasks share the same representations learned by the CNN, the regression task can be further benefited from the quality-sensitive representations. Extensive experiments demonstrate the effectiveness of the proposed model and its superiority over existing SPI quality assessment methods.