基于非下采样轮廓波变换与引导滤波器的红外及可见光图像融合

doi:10.3969/j.issn.1000-1093.2021.09.012

摘要/Abstract

摘要： 为克服传统融合方法对灰度相关性较弱的红外与可见光图像融合存在的不足，提出一种基于非下采样轮廓波变换（NSCT）与引导滤波器的融合方法。利用NSCT对源图像进行多尺度多方向分解，分离出包含在不同频带内的特征信息，得到一个低频近似图像和多个高频方向细节图像；局域窗口加权平均能量和改进拉普拉斯能量分别作为低频近似图像的活性测度，构造显著特征图对近似图像进行加权平均，以解决能量保持和细节提取两个关键问题；在方向细节图像中，基于活性测度取大规则获得决策映射图，将源图像作为引导图、决策映射图作为输入图像进行引导滤波，得到权重分配图，对方向细节图像进行加权平均，降低噪声的敏感度。对融合后的近似图像和方向细节图像进行NSCT逆变换，得到最后的融合图像。采用多组红外与可见光图像进行融合实验，并对融合结果进行客观评价。实验结果表明，该融合方法在主观和客观评价上均优于已有文献的一些典型融合方法，如基于两尺度分解的引导滤波融合方法、NSCT域内稀疏表示融合方法、基于像素显著性的交叉双边滤波融合方法、基于深度学习的卷积神经网络融合方法、基于显著性检测的双尺度融合方法，可获得更好的融合效果。

关键词: 非下采样轮廓波, 引导滤波器, 图像融合, 显著特征图, 平移不变性

Abstract: An image fusion method based on non-subsampled contourlet transform (NSCT) and guided filter is proposed to overcome the shortcomings of traditional fusion methods for infrared and visible images with weak gray correlation. The source images are decomposed into multi-scale and multi-directional sub-bands using NSCT, which can separate the feature information contained in different frequency domains to obtain a low-frequency approximate image and the high-frequency directional sub-band images. The local window weighted average energy and sum-modified-Laplacian energy are regarded as the activity measures of low frequency approximate image, respectively, which are used to construct a salient feature map to solve the two key problems of energy preservation and detail extraction. In the directional sub-band image, the maximun activity measure rule is used to obtain the decision maps, the source images are taken as the guided images, and the decision maps are used as the input images for guided filtering. The weight distribution graph is obtained to weight and average the directional sub-band images to reduce the noise sensitivity. Finally, the fused approximate and directional sub-band images are reconstructed by non-subsampled contourlet inverse transform, and the final fusion image is obtained. Some fusion experiments on several sets of infrared and visible images were did, and the objective performance assessments were implemented to fusion results. The experimental results indicate that the proposed method performs better in subjective and objective assessments than a few existing typical fusion techniques，such as guided filter fusion method based on two-scale decomposition, sparse representation fusion method in NSCT domain, cross bilateral filter fusion method based on pixel saliency, convolution neural network fusion method based on deep learning, two-scale fusion method based on saliency detection, and obtains better fusion performance.

Key words: non-subsampledcontourlettransform, guidedfilter, infraredimage, salientfeaturemap, shift-invariance

中图分类号:

TP391.413

丁贵鹏，陶钢，李英超, 庞春桥，王小峰，段桂茹. 基于非下采样轮廓波变换与引导滤波器的红外及可见光图像融合[J]. 兵工学报, 2021, 42(9): 1911-1922.

DING Guipeng， TAO Gang, LI Yingchao, PANG Chunqiao, WANG Xiaofeng, DUAN Guiru. Infrared and Visible Images Fusion based on Non-subsampled Contourlet Transform and Guided Filter[J]. Acta Armamentarii, 2021, 42(9): 1911-1922.

参考文献

［1］高嘉明,佘冲冲,陈军,等.基于近红外光谱的梯恩梯凝固点温度检测方法研究［J］.兵工学报,2020,41(5):1034-1040.
GAO J M, SHE C C,CHEN J,et al.The detection of solidifying point temperature of TNT using Fourier transform near-infrared spectroscopy［J］.Acta Armamentraii,2020,41(5):1034-1040.（in Chinese）
［2］ LEE M W,KWAK K C.Performance comparison of infrared and visible image fusion approaches ［C］∥Proceedings of International Conference on Control,Artificial Intelligence,Robotics & Optimization.Prague,Czech Republic:IEEE,2017:274-277.
［3］ JIN X,JIANG Q,YAO S W,et al.A survey of infrared and visual image fusion methods［J］.Infrared Physics & Technology,2017,85:479-501.
［4］ LI S T,KANG X D,FANG L Y,et al.Pixel-level image fusion:a survey of the state of the art［J］.Information Fusion,2016,33:100-112.
［5］ LI S T,YANG B,HU J W.Performance comparison of different multi-resolution transforms for image fusion ［J］.Information Fusion,2011,12(2):74-84.
［6］陈广秋,梁小伟,段锦,等.多级方向引导滤波器及其在多传感器图像融合中的应用［J］.吉林大学学报（理学版）,2019,57(1): 129-138.
CHEN G Q,LIANG X W,DUAN J,et al.Multistage directional guided filter and its application in multi-sensor image fusion［J］.Journal of Jilin University(Science Edition),2019,57(1):129-138.（in Chinese）
［7］ MEHERA B,AGRAWALA S,PANDA R,et al.A survey on region based image fusion methods［J］.Information Fusion, 2019,48:119-132.
［8］李东瑾，杨瑞娟，董睿杰.基于联合投影字典学习的辐射源调制识别［J］.兵工学报,2020,41(7):1464-1472.
LI D J,YANG R J,DONG R J.Emitter signal modulation recognition based on joint projection dictionary learning ［J］.Acta Armamentarii,2020,41(7):1464-1472.（in Chinese）
［9］ JIN X,JIANG Q,YAO S W,et al.A survey of infrared and visual image fusion methods［J］.Infrared Physics & Technology,2017,85:478-501.
［10］ LIU Y,CHEN X,WANG Z F,et al.Deep learning for pixel-level image fusion:recent advances and future prospects［J］.Information Fusion,2018,42:158-173.
［11］ DA CUNHA A L,ZHOU J,DO M N.The nonsubsampled contourlet transform:theory,design, and applications［J］.IEEE Transactions on Image Processing,2006,15(10):3089-3101.
［12］ HE K M,SUN J,TANG X O.Guided image filtering［J］.IEEE Transactions on Pattern Analysis & Machine Intelligence,2013,35(6):1397-1409.
［13］ YIN M,LIU X N,LIU Y,et al.Medical image fusion with para-meter- adaptive pulse coupled neural network in nonsubsampled shearlet transform domain［J］.IEEE Transactions on Instrumentation and Measurement,2019,68(1):49-64.
［14］ LIU S Q,ZHAO J,SHI M Z.Medical image fusion based on improved Sum-modified-Laplacian［J］.International Journal of Imaging Systems & Technology,2015,25(3):206-212.
［15］ LIU Z,BLASCH E,XUE Z Y,et al.Objective assessment of multiresolution image fusion algorithms for context enhancement in night vision: a comparative study［J］. IEEE Transactions on Pattern Analysis and Machine Intelligence,2012,34(1):94-109.
［16］ YE F M,SU Y F,XIAO H,et al.Remote sensing image registration using convolutional neural network features［J］.IEEE Geoscience & Remote Sensing Letters, 2018,15(2):232-236.
［17］ DU J,LI W S,XIAO B.Anatomical-functional image fusion by information of interest in local Laplacian filtering domain［J］.IEEE Transactions on Image Processing,2017,26(12): 5855-5866.
［18］ LI S T,KANG X D,HU J W.Image fusion with guided filtering［J］.IEEE Transactions on Image Processing, 2013,22 (7):2864-2875.
［19］ LIU Y,LIU S P,WANG Z F.A general framework for image fusion based on multi-scale transform and sparse representation［J］.Information Fusion,2015, 24:147-164.
［20］ SHREYAMSHA KUMAR B K.Image fusion based on pixel significance using cross bilateral filter［J］.Signal Image & Video Processing,2015,9(5):1193-1204.
［21］ LIU Y, CHEN X, CHENG J, et al. Infrared and visible image fusion with convolutional neural networks［J］. International Journal of Wavelets, Multiresolution and Information Processing,2018,16(3):1850018.
［22］ BAVIRISETTI D P,DHULI R.Two-scale image fusion of visible and infrared images using saliency detection［J］.Infrared Physics & Technology,2016,76:52-64.

第42卷第9期2021 年9月
兵工学报ACTA ARMAMENTARII
Vol.42No.9Sep.2021

[1]	李良福，陈卫东，高强，许开銮，刘轩，何曦，钱钧. 基于深度学习的光电系统智能目标识别[J]. 兵工学报, 2022, 43(S1): 162-168.
[2]	唐伟，贾方秀，王晓鸣. 基于双边滤波的可见光与红外图像自适应融合[J]. 兵工学报, 2022, 43(11): 2836-2845.
[3]	武鹤龙，邱政，张维全. 侧扫声纳图像非下采样轮廓波变换域分区增强方法[J]. 兵工学报, 2021, 42(7): 1463-1470.
[4]	王珺, 彭进业, 何贵青, 冯晓毅, 阎昆. 基于非下采样Contourlet 变换和稀疏表示的红外与可见光图像融合方法[J]. 兵工学报, 2013, 34(7): 815-820.
[5]	张玉珍，孙佳嵩，陈钱，顾国华，冯世杰，. 基于相位一致性的结构相似度图像融合质量评价方法[J]. 兵工学报, 2013, 34(12): 1514-1520.