Abstract:

Transformer is a kind of mainstream method used for infrared and visible image fusion (IVIF).However,most of the Transformer-based methods for IVIF suffer from the problems such as large number of parameters and high computational complexity.To this end,a method for infrared and visible image fusion based on Mamba-empowered triple-branch generative adversarial network is proposed.Specifically,three independent and cooperative branches for feature extraction and fusion are designed in the generator network.The infrared and visible branches are utilized to extract the global features from the infrared and visible images,respectively,through the shallow feature extraction modules and Mamba blocks.Meanwhile,the global features extracted by the infrared and visible branches are hierarchically integrated into the convolutional neural network-based intermediate fusion branch,thereby achieving the full interaction and fusion between the local and global features.Furthermore,the generator is competitively trained against two discriminators (infrared discriminator and visible discriminator),forcing the generator to improve its ability to produce fusion image.The experiments on public datasets indicate that the proposed method outperforms other methods both in the qualitative visual effects and the quantitative objective metrics.

Key words: image fusion, infrared image, generative adversarial network, Mamba