PA-Fuse : deep supervised approach for the fusion of photoacoustic images with distinct reconstruction characteristics

Abstract

The methods available for solving the inverse problem of photoacoustic tomography promote only one feature–either being smooth or sharp–in the resultant image. The fusion of photoacoustic images reconstructed from distinct methods improves the individually reconstructed images, with the guided filter based approach being state-of-the-art, which requires that implicit regularization parameters are chosen. In this work, a deep fusion method based on convolutional neural networks has been proposed as an alternative to the guided filter based approach. It has the combined benefit of using less data for training without the need for the careful choice of any parameters and is a fully data-driven approach. The proposed deep fusion approach outperformed the contemporary fusion method, which was proved using experimental, numerical phantoms and in-vivo studies. The improvement obtained in the reconstructed images was as high as 95.49% in root mean square error and 7.77 dB in signal to noise ratio (SNR) in comparison to the guided filter approach. Also, it was demonstrated that the proposed deep fuse approach, trained on only blood vessel type images at measurement data SNR being 40 dB, was able to provide a generalization that can work across various noise levels in the measurement data, experimental set-ups as well as imaging objects.