基于通道校正卷积的真彩色微光图像增强

何锦成; 韩永成; 张闻文; 何伟基; 陈钱

doi:10.12382/bgxb.2022.0204

您当前的位置：

首页 >

文章列表页 >

基于通道校正卷积的真彩色微光图像增强

更新时间：2025-08-18

- 基于通道校正卷积的真彩色微光图像增强
- True Color Low-Light Image Enhancement Based on Channel-Calibrated Convolution
- 兵工学报 2023年44卷第6期页码：1643-1654
- 作者机构：
  
  南京理工大学电子工程与光电技术学院, 江苏南京 210094
- 作者简介：
  
  *邮箱: zhangww@njust.edu.cn
- 基金信息：
  
  国家自然科学基金项目(61875088);北京理工大学“青年教师学术启动计划”项目(2020年)
- DOI：10.12382/bgxb.2022.0204
  中图分类号：
- 收稿：2022-03-29，
  
  网络出版：2023-07-19，
  
  纸质出版：2023-06-30
- 稿件说明：
移动端阅览
何锦成, 韩永成, 张闻文, 等. 基于通道校正卷积的真彩色微光图像增强[J]. 兵工学报, 2023,44(6):1643-1654.

Jincheng HE, Yongcheng HAN, Wenwen ZHANG, et al. True Color Low-Light Image Enhancement Based on Channel-Calibrated Convolution[J]. Acta Armamentarii, 2023, 44(6): 1643-1654.
何锦成, 韩永成, 张闻文, 等. 基于通道校正卷积的真彩色微光图像增强[J]. 兵工学报, 2023,44(6):1643-1654. DOI： 10.12382/bgxb.2022.0204.

Jincheng HE, Yongcheng HAN, Wenwen ZHANG, et al. True Color Low-Light Image Enhancement Based on Channel-Calibrated Convolution[J]. Acta Armamentarii, 2023, 44(6): 1643-1654. DOI： 10.12382/bgxb.2022.0204.

摘要

针对现有真彩色夜视相机所成图像亮度低、对比度低、噪声和色彩失真等问题

提出基于通道校正卷积的神经网络算法。通道校正卷积的上分支引入通道注意力块分析RGB通道之间的特征

用来代替U-Net网络中的传统卷积

实现颜色恢复并保留更多图像信息;在传统损失函数中增加Sobel损失函数和色彩损失函数

抑制噪声的同时并保护图像细节

减小色差、增强对比度。采集真实场景下的图像数据集

提升对实际数据的处理效果。实验结果表明:该算法能同时处理低照度图像的亮度、对比度、噪声和色差问题

增强效果优于目前主流算法;与传统卷积的U-Net网络相比

降低了模型复杂度

提高了运行速度

计算量减少了13.71%

参数减少了13.65%

PSNR值提升了29.20%

SSIM值提升了7.23%

色差减少了10.46%

兼顾了成像质量与成像速度。

Abstract

Aiming at the problems of low brightness

low contrast

noise and color distortion of images produced by existing true color night vision cameras

a neural network algorithm based on channel-calibrated convolution is proposed. The upper branch of the channel-calibrated convolution introduces a channel attention block to analyze the features between the RGB channels. This replaces the traditional convolution in the U-Net network

enabling color recovery and the retention of more image information. The Sobel loss function and color loss function are added to the traditional loss function to suppress noise

preserve image details

reduce chromatic aberration

and enhance contrast. An image dataset under real conditions is collected

which improves the processing effect of the actual data. The experimental results show that the algorithm in this paper can simultaneously deal with the brightness

contrast

noise and chromatic aberration of low-light images

and the enhancement effect is better than the existing mainstream algorithms. Compared with the traditional convolutional U-Net network

the novel method reduces the model complexity and improves operating speed

with a 13.71% reduction in computation

a 13.65% reduction in parameters

a 29.20% increase in PSNR

a 7.23% increase in SSIM

and a 10.46% decrease in chromatic aberration. The algorithm in this paper strikes a balance between imaging quality and speed.

关键词

Keywords

references

蒋云峰 , 武东生 , 黄富瑜 . 彩色夜视技术方法综述 [J ] . 激光技术 , 2020 , 44 ( 1 ): 108 - 112 .

JIANG Y F , WU D S , HUANG F Y . Summary of color night vision technology [J ] . Laser Technology , 2020 , 44 ( 1 ): 108 - 112 . (in Chinese) DOI: 10.1016/j.optlastec.2011.06.001 http://doi.org/10.1016/j.optlastec.2011.06.001 https://linkinghub.elsevier.com/retrieve/pii/S0030399211001617 https://linkinghub.elsevier.com/retrieve/pii/S0030399211001617

陈一超 , 胡文刚 , 武东生 , 等 . 三波段微光彩色夜视方法研究 [J ] . 应用光学 , 2015 , 36 ( 3 ): 430 - 434 . DOI: 10.5768/JAO201536.0304001 http://doi.org/10.5768/JAO201536.0304001 为克服夜视设备产生单色图像的缺陷，得到更符合人眼视觉的彩色夜视图像。根据RGB理论模型以及滤光片的分谱作用原理，基于第二代像增强器设计出一种彩色夜视方案，通过测量滤光片的光谱透过率，得出滤光片对像增强器信噪比的影响；提出利用全波图像的四帧融合解决像增强器信噪比低的问题，并进行实验验证。实验结果显示室内和室外情况下四帧融合彩色图像熵和方差分别达到6.8、35.39及7.0、45.07，均优于三帧融合时对应值，说明提出的四帧融合方法能够克服信噪比降低的缺陷，得到清晰且信息量丰富的夜视图像。

CHEN Y C , HU W G , WU D S , et al . Triple-band low light level color night vision technology [J ] . Journal of Applied Optics , 2015 , 36 ( 3 ): 430 - 434 . (in Chinese) DOI: 10.5768/JAO201536.0304001 http://doi.org/10.5768/JAO201536.0304001 http://www.yygx.net/CN/abstract/abstract10616.shtml http://www.yygx.net/CN/abstract/abstract10616.shtml

WAXMAN A M , AGUILAR M , FAY D A , et al . Solid-state color night vision: fusion of low-light visible and thermal infrared imagery [J ] . Lincoln Laboratory Journal , 1998 , 11 ( 1 ): 41 - 60 .

马爽 , 方建安 , 孙韶媛 , 等 . 基于伪彩色融合图像聚类的夜视图像上色算法 [J ] . 光学学报 , 2009 , 29 ( 6 ): 1502 - 1507 .

MA S , FANG J A , SUN S Y , et al . Colorizing algorithm of night-vision image based on clustering of false color fused image [J ] . Acta Optica Sinica , 2009 , 29 ( 6 ): 1502 - 1507 . (in Chinese) DOI: 10.3788/AOS http://doi.org/10.3788/AOS http://www.opticsjournal.net/gxxb.htm http://www.opticsjournal.net/gxxb.htm

赵源萌 , 王岭雪 , 金伟其 , 等 . 基于色彩传递的单波段热图像彩色化处理算法 [J ] . 光学学报 , 2009 , 29 ( 3 ): 654 - 658 .

ZHAO Y M , WANG L X , JIN W Q , et al . Single-band infrared image colorization algorithm based on color transfer [J ] . Acta Optica Sinica , 2009 , 29 ( 3 ): 654 - 658 . (in Chinese) DOI: 10.3788/AOS http://doi.org/10.3788/AOS http://www.opticsjournal.net/gxxb.htm http://www.opticsjournal.net/gxxb.htm

KOYAMA S , INABA Y , KASANO M , et al . A day and night vision MOS imager with robust photonic-crystal-based RGB-and-IR [J ] . IEEE Transactions on Electron Devices , 2008 , 55 ( 3 ): 754 - 759 . DOI: 10.1109/TED.2007.914828 http://doi.org/10.1109/TED.2007.914828 http://ieeexplore.ieee.org/document/4455782/ http://ieeexplore.ieee.org/document/4455782/

曾桂林 , 苏秉华 , 牛丽红 , 等 . 基于神经网络的微光图像增强与复原 [J ] . 兵工学报 , 2006 , 27 ( 4 ): 652 - 654 . 微光图像的退化主要取决于探測器的灵敏度、非线性和噪声。神经网络是ー个高度复杂的非线性动力学系统，对于微光图像，提出ー种基于多层前向传播（MLP)神经网络的图像增强与复原方法。为了加速训练学习过程，釆用加动量项的自适应BP学习方法对网络进行循环迭代训练。实验结果表明，MLP神经网络增强与复原方法是可行的，并且对微光图像的增强与复原具有很好的效果。

ZENG G L , SU B H , NIU L H , et al . Neural network-based low-light-level image enhancement and reconstruction [J ] . Acta Armamentarii , 2006 , 27 ( 4 ): 652 - 654 . (in Chinese)

GUO X J , LI Y , LING H B . LIME: low-light image enhancement via illumination map estimation [J ] . IEEE Transactions on Image Processing , 2016 , 26 ( 2 ): 982 - 993 . DOI: 10.1109/TIP.2016.2639450 http://doi.org/10.1109/TIP.2016.2639450 http://ieeexplore.ieee.org/document/7782813/ http://ieeexplore.ieee.org/document/7782813/

冯维 , 吴贵铭 , 赵大兴 , 等 . 多图像融合Retinex用于弱光图像增强 [J ] . 光学精密工程 , 2020 , 28 ( 3 ): 736 - 744 .

FENG W , WU G M , ZHAO D X , et al . Multi images fusion retinex for low light image enhancement [J ] . Optics and Precision Engineering , 2020 , 28 ( 3 ): 736 - 744 . (in Chinese) DOI: 10.3788/OPE. http://doi.org/10.3788/OPE. http://www.opticsjournal.net/Journals/gxjmgc.htm http://www.opticsjournal.net/Journals/gxjmgc.htm

黄慧 , 董林鹭 , 刘小芳 , 等 . 改进Retinex的低光照图像增强 [J ] . 光学精密工程 , 2020 , 28 ( 8 ): 1835 - 1849 .

HUANG H , DONG L L , LIU X F , et al . Improved retinex low light image enhancement method [J ] . Optics and Precision Engineering , 2020 , 28 ( 8 ): 1835 - 1849 . (in Chinese)

DONG X A , WANG G , PANG Y , et al . Fast efficient algorithm for enhancement of low lighting video [C ] //Proceedings of 2011 IEEE International Conference on Multimedia and Expo . New York,NY,US : IEEE , 2011 : 1 - 6 .

WANG W J , WEI C , YANG W H , et al . GLADNet: Low-light enhancement network with global awareness [C ] //Proceedings of the 2018 13rd IEEE International Conference on Automatic Face & Gesture Recognition . New York,NY,US : IEEE , 2018 : 751 - 755 .

WEI C , WANG W J , YANG W H , et al . Deep retinex decomposition for low-light enhancement: arXiv:1808.04560 [R/OL ] . Ithaca, NY, US : Cornell University (2018-08-01)[2021-04-01 ] . https://arxiv.org/abs/1808.04560. https://arxiv.org/abs/1808.04560 https://arxiv.org/abs/1808.04560

SONG X J , HUANG J J , CAO J Z , et al . Multi-scale joint network based on Retinex theory for low-light enhancement [J ] . Signal, Image and Video Processing , 2021 , 15 ( 6 ): 1257 - 1264 . DOI: 10.1007/s11760-021-01856-y http://doi.org/10.1007/s11760-021-01856-y

ZHANG Y H , ZHANG J W , GUO X J . Kindling the darkness: a practical low-light image enhancer [C ] //Proceedings of the 27th ACM International Conference on Multimedia. New York, NY,US:ACM , 2019 : 1632 - 1640 .

黄辉先 , 陈凡浩 . 基于注意力机制和Retinex的低照度图像增强方法 [J ] . 激光与光电子学进展 , 2020 , 57 ( 20 ): 201004 .

HUANG H X , CHEN F H . Low-illumination image enhancement method based on attention mechanism and retinex [J ] . Laser & Optoelectronics Progress , 2020 , 57 ( 20 ): 201004 . (in Chinese)

JIANG Y F , GONG X Y , LIU D , et al . Enlightengan: deep light enhancement without paired supervision [J ] . IEEE Transactions on Image Processing , 2021 , 30 : 2340 - 2349 . DOI: 10.1109/TIP.2021.3051462 http://doi.org/10.1109/TIP.2021.3051462 https://ieeexplore.ieee.org/document/9334429/ https://ieeexplore.ieee.org/document/9334429/

GUO C L , LI C Y , GUO J C , et al . Zero-reference deep curve estimation for low-light image enhancement [C ] //Proceedings of the 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition. New York,NY , US : IEEE , 2020 : 1777 - 1786 .

ZHANG Y , DI X G , ZHANG B , et al . Self-supervised image enhancement network: training with low light images only: arXiv:2002.11300 [EB/OL ] . Ithaca, NY, US :Cornell University(2020-02-26)[2021-04-01 ] . https://arxiv.org/abs/2002.11300. https://arxiv.org/abs/2002.11300 https://arxiv.org/abs/2002.11300

唐超影 , 浦世亮 , 叶鹏钊 , 等 . 基于卷积神经网络的低照度可见光与近红外图像融合 [J ] . 光学学报 , 2020 , 40 ( 16 ): 1610001 .

TANG C Y , PU S L , YE P Z , et al . Fusion of low-illuminance visible and near-infrared images based on convolutional neural networks [J ] . Acta Optica Sinica , 2020 , 40 ( 16 ): 1610001 . (in Chinese) DOI: 10.3788/AOS http://doi.org/10.3788/AOS http://www.opticsjournal.net/Journals/gxxb.htm http://www.opticsjournal.net/Journals/gxxb.htm

CHEN C , CHEN Q F , XU J , et al . Learning to see in the dark [C ] //Proceedings of 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.New York,NY , US : IEEE , 2018 : 3291 - 3300 .

RONNEBERGER O , FISCHER P , BROX T . U-net: Convolutional networks for biomedical image segmentation [C ] //Proceedings of the 2015 18th International Conference on Medical Image Computing and Computer-assisted Intervention . Switzerland : Springer , 2015 : 234 - 241 .

李俊峰 . 基于RGB色彩空间自然场景统计的无参考图像质量评价 [J ] . 自动化学报 , 2015 , 41 ( 9 ): 1601 - 1615 .

LI J F . Non-reference image quality evaluation based on natural scene statistics in RGB color space [J ] . Acta Automatica Sinica , 2015 , 41 ( 9 ): 1601 - 1615 . (in Chinese)

WOO S , PARK J , LEE J Y , et al . CBAM: Convolutional block attention module [C ] //Proceedings of the 2018 15th European Conference on Computer Vision . Switzerland : Springer , 2018 : 3 - 19 .

LIU J J , HOU Q , CHENG M M , et al . Improving convolutional networks with self-calibrated convolutions [C ] //Proceedings of 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition. New York,NY , US : IEEE , 2020 : 10096 - 10105 .

MITTAL A , SOUNDARARAJAN R , BOVIK A C . Making a "completely blind" image quality analyzer [J ] . IEEE Signal Processing Letters , 2012 , 20 ( 3 ): 209 - 212 . DOI: 10.1109/LSP.2012.2227726 http://doi.org/10.1109/LSP.2012.2227726 http://ieeexplore.ieee.org/document/6353522/ http://ieeexplore.ieee.org/document/6353522/

HILL B , ROGER T , VORHAGEN F W . Comparative analysis of the quantization of color spaces on the basis of the CIELAB color-difference formula [J ] . ACM Transactions on Graphics , 1997 , 16 ( 2 ): 109 - 154 . DOI: 10.1145/248210.248212 http://doi.org/10.1145/248210.248212 https://dl.acm.org/doi/10.1145/248210.248212 https://dl.acm.org/doi/10.1145/248210.248212 \n This article discusses the CIELAB color spave within the limits of optimal colors including the complete volume of object colors. A graphical representation of this color space is composed of planes of constant lightness\n L\n * with an net of lines parallel to the\n a\n * and\n b\n * axes. This uniform net is projected onto a number of other color spaces (CIE XYZ, tristimulus RGB, predistorted RGB, and YCC color space) to demonstrate and study the structure of color differences in these spaces on the basis of CIELAB color difference formulas. Two formulas are considered: the CIE 1976 formula *** and the newer CiE 1994 formula ***. The various color spaces considered are uniformly quantized and the grid of quantized points is transformed into CIELAB colordinates to study the distribution of color differences due to basic quantization steps and to spacify the areas of the colors with the highest sensitivity to color discrimination. From a threshold value for the maximum color difference among neighboring quantized points searched for in each color space, concepts for the quantization of the color spaces are drived. The results are compared to quantization concepts based on average values of quantization errors published in previous work. In addition to color spaces bounded by the optimal colors, the studies are also applied to device-dependent color spaces limited by the range of a positive RGB cube or by the gamut of colors of practical print processes (thermal dye sublimation, chromalin, and match print). For all the color spaces, estimation of the number of distinguishable colors are given on the basis of a threshold value for the color difference perception of *** = 1 and *** = 1.\n

浏览量

241

下载量

CNKI被引量

文章被引用时，请邮件提醒。

提交

工具集

关联资源

基于峰度值估计的无线信号频谱感知方法

混合架构的多尺度特征交互去雾算法

基于卷积神经网络与支持向量机的适配器落点预测方法

基于光照感知和特征增强的可见光-热红外图像语义分割

基于卷积神经网络和时空特征的烟幕视频检测和参数提取