基于双边滤波的可见光与红外图像自适应融合

doi:10.12382/bgxb.2021.0626

摘要/Abstract

摘要： 针对传统融合方法不能充分提取可见光图像与红外图像各自特有的细节信息和难以确定融合权值的问题，提出一种基于双边滤波的可见光与红外图像自适应融合算法。该方法结合双边滤波与改进的双边滤波，将可见光与红外图像分解为多尺度局部细节、特有细节以及基础信息。在图像融合阶段，局部细节信息采用基于图像边缘能量的融合方法，最大程度保留了图像的细节；图像基础部分的融合，引入了基于局部能量值的正则化参数来自适应调整融合的权值，优化参数的选择；特有细节信息的融合采用了绝对值最大的融合规则以充分保留源图像特有的细节信息。仿真实验结果表明，通过主观判断所提方法融合后的图像视觉效果更好、对比度高，边缘细节的融合比其他算法更加好，在客观指标的评价中，所提融合方法的MI、EN、FMI、SD、Q^AB/F、Mean 6种指标相对于其他方法综合提升了22.6%、5.7%、0.7%、30.4%、14.2%、18.4%，在主观视觉上和客观评价指标中，所提的算法均具备更优的融合效果。

关键词: 可见光与红外图像, 图像融合, 双边滤波, 局部能量

Abstract: Traditional fusion methods cannot fully extract the details of visible and infrared images, and it is hard to determine the fusion weights. An adaptive image fusion method based on bilateral filters is thus proposed. First, an improved bilateral filter is employed to decompose the source images into local details, unique details, and base parts. Second, the local details are merged based on the energy of image edges to retain details to the maximum. The base parts are fused using an adaptive regularization parameter based on local energy. Finally, the fused image is obtained. Experimental results indicate that the fused image obtained by the proposed fusion method has a better visual effect, higher contrast, and clearer edge details. In comparison with other four methods, the Mutual Information(MI), Feature Mutual information(FMI), Standard Deviation(SD), Mount of edge Information(Q), and Mean of the new approach have increased by 22.6%, 5.7%, 0.7%, 30.4%, 14.2%, and 18.4%, respectively. Therefore, the proposed image fusion method has a better performance than other methods in terms of both subjective and objective evaluation.

Key words: visibleandinfraredimage, imagefusion, imageenergyofedge, localenergy

中图分类号:

TP391.4

唐伟，贾方秀，王晓鸣. 基于双边滤波的可见光与红外图像自适应融合[J]. 兵工学报, 2022, 43(11): 2836-2845.

TANG Wei, JIA Fangxiu, WANG Xiaoming. Visible and Infrared Image Adaptive Fusion Based on Bilateral Filters[J]. Acta Armamentarii, 2022, 43(11): 2836-2845.

参考文献

［1］丁贵鹏，陶钢，李英超，等.基于非下采样轮廓波变换与引导滤波器的红外与可见光图像融合［J］.兵工学报，2021，42（9）: 1911-1922.
DING G P，TAO G，LI Y C，et al. Infrared and visible images fusion based on non-subsampled contourlet transform and guided filter［J］. Acta Armamentarii, 2021，42（9）:1911-1922.(in Chinese)
［2］ AISHWARYA N，THANGAMMAL C B. Visible and infrared image fusion using DTCWT and adaptive combined clustered dictionary［J］. Infrared Physics & Technology，2018，93: 300-309.
［3］ ZHU B，XIN Y H. Effective and robust infrared small target detection with the fusion of polydirectional first order derivative images under facet model［J］. Infrared Physics & Technology，2015，69: 136-144.
［4］ KONG S G，HEO J，BOUGHORBEL F，et al. Multi-scale fusion of visible and thermal IR images for illumination-invariant face recognition［J］.International Journal of Computer Vision，2007，71(2): 215-233.
［5］ PIELLA G. A general framework for multiresolution image fusion: from pixels to regions［J］. Information Fusion， 2003， 4(4): 259- 280.
［6］ YANG B，LI S T. Pixel-level image fusion with simultaneous orthogonal matching pursuit［J］. Information Fusion，2012， 13(1): 10-19.
［7］ YIN M，DUAN P H，LIU W，et al. A novel infrared and visible image fusion algorithm based on shift-invariant dual-tree complex shearlet transform and sparse representation［J］. Neurocomputing，2017，226: 182-191.
［8］ ZHANG Q，MALDAGUE X. An adaptive fusion approach for infrared and visible images based on NSCT and compressed sensing［J］. Infrared Physics & Technology，2016，74: 11-20.
［9］ CHEN J，LI X J，LUO L B，et al. Infrared and visible image fusion based on target-enhanced multiscale transform decomposition［J］. Information Sciences，2020，508: 64-78.
［10］ TOMASI C，MANDUCHI R. Bilateral filtering for gray and color images［C］∥Proceedings of the 6th International Conference on Computer Vision (IEEE Cat. No. 98CH36271). Bombay，India: IEEE，1998: 839-846.
［11］ ZHOU Z，WANG B，LI S，et al. Perceptual fusion of infrared and visible images through a hybrid multi-scale decomposition with Gaussian and bilateral filters［J］. Information Fusion，2016，30: 15-26.
［12］ CHEN J，LI X J，LUO L B，et al. Infrared and visible image fusion based on target-enhanced multiscale transform decomposition［J］. Information Sciences，2020，508: 64-78.
［13］ WANG J J，LI Q，JIA Z H，et al. A novel multi-focus image fusion method using PCNN in nonsubsampled contourlet transform domain［J］. Optik，2015，126(20): 2508-2511.
［14］ BURT P J，ADELSON E H. The Laplacian pyramid as a compact image code［J］. Readings in Computer Vision， 1987， 31(4): 671-679.
［15］ TOET A. Image fusion by a ratio of low-pass pyramid［J］. Pattern Recognition Letters，1989，9(4): 245-253.
［16］ NENCINI F，GARZELLI A，BARONTI S，et al. Remote sensing image fusion using the curvelet transform［J］. Information Fusion，2007，8(2): 143-156.
［17］ NAIDU V P S. Image fusion technique using multi-resolution singular value decomposition［J］. Defence Science Journal, 2011, 61(5): 479-484.
［18］ MA J Y，CHEN C，LI C，et al. Infrared and visible image fusion via gradient transfer and total variation minimization［J］. Information Fusion，2016，31: 100-109.
［19］ TOET A.TNO Image Fusion Dataset［DB/OL］. (2014-04-26) ［2021-08-15］.https:∥doi.org/10.6084/m9.figshare.1008029.v1.
［20］ JIN X，JIANG Q，YAO S W，et al. Infrared and visual image fusion method based on discrete cosine transform and local spatial frequency in discrete stationary wavelet transform domain［J］.Infrared Physics & Technology，2018，88: 1-12.
［21］ QU G H，ZHANG D L，YAN P F. Information measure for performance of image fusion［J］.Electronics Letters，2002，38(7): 313-315.
［22］ XYDEAS C S，PETROVIC V. Objective image fusion performance measure［J］. Electronics Letters, 2000, 36(4): 308-309.
［23］ HAGHIGHAT M B A, AGHAGOLZADEH A, SEYEDARABI H. A non-reference image fusion metric based on mutual information of image features［J］. Computers & Electrical Engineering, 2011, 37(5): 744-756.
［24］ LIU Y，LIU S P，WANG Z F. A general framework for image fusion based on multi-scale transform and sparse representation［J］. Information Fusion，2015，24: 147-164.

[1]	李良福，陈卫东，高强，许开銮，刘轩，何曦，钱钧. 基于深度学习的光电系统智能目标识别[J]. 兵工学报, 2022, 43(S1): 162-168.
[2]	丁贵鹏，陶钢，李英超, 庞春桥，王小峰，段桂茹. 基于非下采样轮廓波变换与引导滤波器的红外及可见光图像融合[J]. 兵工学报, 2021, 42(9): 1911-1922.
[3]	王珺, 彭进业, 何贵青, 冯晓毅, 阎昆. 基于非下采样Contourlet 变换和稀疏表示的红外与可见光图像融合方法[J]. 兵工学报, 2013, 34(7): 815-820.
[4]	张玉珍，孙佳嵩，陈钱，顾国华，冯世杰，. 基于相位一致性的结构相似度图像融合质量评价方法[J]. 兵工学报, 2013, 34(12): 1514-1520.
[5]	申琳，杨进华，韩福利，姜会林，王春艳，谢殿广. 基于光斑图像的激光束散角测量方法研究[J]. 兵工学报, 2011, 32(7): 890-895.
[6]	李骜，李一兵，刘丹丹. 基于非下采样Contourlet的熵测度图像融合算法[J]. 兵工学报, 2011, 32(11): 1348-1352.
[7]	赵诚，付梦印，王立平. 基于粒子群优化的多分辨率图像融合[J]. 兵工学报, 2010, 31(2): 171-176.
[8]	苏娟，鲜勇，宋建社. SAR图像与可见光图像融合的建筑物提取算法[J]. 兵工学报, 2010, 31(11): 1448-1454.
[9]	史世明，王岭雪，金伟其，赵源萌. 基于YUV空间色彩传递的可见光/热成像双通道彩色成像系统[J]. 兵工学报, 2009, 30(1): 30-35.
[10]	石俊生，金伟其. 彩色夜视融合图像熵评价研究[J]. 兵工学报, 2006, 27(6): 1039-1042.
[11]	杨风暴，韩焱，阳方林. 绝热层粘接检测图像的多分辨率融合[J]. 兵工学报, 2004, 25(1): 50-52.