Feature division and fusion for multi-task dense prediction

Abstract

In MTL, how to simplify the high-level task network structure while improving the model’s performance across multiple tasks, while effectively utilizing the inter-task correlation to further enhance the model’s generalization ability, re mains a challenging problem. Most multi-task dense prediction works typically adopt the encoder-focused sharing model, where the lower-level backbone net work is shared across tasks, while each task has its own independent high-level prediction network. This approach requires the design of high-level networks that can effectively extract task-specific features, as well as facilitate the ex change and acquisition of cross-task information within the high-level networks. In this paper, we propose a Feature Division and Fusion layer, which gen crates preliminary task-specific representations for different visual tasks using the generic features extracted from the backbone network, while also effectively leveraging the generic features to obtain relevant information from other tasks, thereby enhancing the model’s generalization ability.