Fourier warm start for physics-informed neural networks

Abstract

Physics-informed neural networks (PINNs) have shown applicability in a wide range of engineering domains. However, there remain some challenges in their use, namely, PINNs are notoriously difficult to train and prone to failure when dealing with complex tasks with multi-frequency patterns or steep gradients in the outputs. In this work, we leverage the Neural Tangent Kernel (NTK) theory and introduce the Fourier Warm Start (FWS) algorithm to balance the convergence rate of neural networks at different frequencies, thereby mitigating spectral bias and improving overall model performance. We then propose the Fourier Analysis Boosted Physics-Informed Neural Network (Fab-PINN), a novel integrated architecture based on the FWS algorithm. Finally, we present a series of challenging numerical examples with multi-frequency or sparse observations to validate the effectiveness of the proposed method. Compared to standard PINN, Fab-PINN exhibits a reduction of relative L2 errors in solving the heat transfer equation, the Klein–Gordon equation, and the transient Navier–Stokes equations from 9.9×10−1 to 4.4×10−3, 5.4×10−1 to 2.6×10−3, and 6.5×10−1 to 9.6×10−4, respectively.