True Color Low-Light Image Enhancement Based on Channel-Calibrated Convolution

doi:10.12382/bgxb.2022.0204

Abstract

Abstract:

Aiming at the problems of low brightness, low contrast, noise and color distortion of images produced by existing true color night vision cameras, a neural network algorithm based on channel-calibrated convolution is proposed. The upper branch of the channel-calibrated convolution introduces a channel attention block to analyze the features between the RGB channels. This replaces the traditional convolution in the U-Net network, enabling color recovery and the retention of more image information. The Sobel loss function and color loss function are added to the traditional loss function to suppress noise, preserve image details, reduce chromatic aberration, and enhance contrast. An image dataset under real conditions is collected, which improves the processing effect of the actual data. The experimental results show that the algorithm in this paper can simultaneously deal with the brightness, contrast, noise and chromatic aberration of low-light images, and the enhancement effect is better than the existing mainstream algorithms. Compared with the traditional convolutional U-Net network, the novel method reduces the model complexity and improves operating speed, with a 13.71% reduction in computation, a 13.65% reduction in parameters, a 29.20% increase in PSNR, a 7.23% increase in SSIM, and a 10.46% decrease in chromatic aberration. The algorithm in this paper strikes a balance between imaging quality and speed.

Key words: low-light-level image, image enhancement, color restoration, noise suppression, convolutional neural network

HE Jincheng, HAN Yongcheng, ZHANG Wenwen, HE Weiji, CHEN Qian. True Color Low-Light Image Enhancement Based on Channel-Calibrated Convolution[J]. Acta Armamentarii, 2023, 44(6): 1643-1654.

Figures/Tables 21

References 27

[1]	蒋云峰, 武东生, 黄富瑜. 彩色夜视技术方法综述[J]. 激光技术, 2020, 44(1):108-112.
	JIANG Y F, WU D S, HUANG F Y. Summary of color night vision technology[J]. Laser Technology, 2020, 44(1):108-112. (in Chinese) doi: 10.1016/j.optlastec.2011.06.001 URL
[2]	陈一超, 胡文刚, 武东生, 等. 三波段微光彩色夜视方法研究[J]. 应用光学, 2015, 36(3):430-434. doi: 10.5768/JAO201536.0304001
	CHEN Y C, HU W G, WU D S, et al. Triple-band low light level color night vision technology[J]. Journal of Applied Optics, 2015, 36(3):430-434. (in Chinese) doi: 10.5768/JAO201536.0304001 URL
[3]	WAXMAN A M, AGUILAR M, FAY D A, et al. Solid-state color night vision: fusion of low-light visible and thermal infrared imagery[J]. Lincoln Laboratory Journal, 1998, 11(1):41-60.
[4]	马爽, 方建安, 孙韶媛, 等. 基于伪彩色融合图像聚类的夜视图像上色算法[J]. 光学学报, 2009, 29(6):1502-1507.
	MA S, FANG J A, SUN S Y, et al. Colorizing algorithm of night-vision image based on clustering of false color fused image[J]. Acta Optica Sinica, 2009, 29(6): 1502-1507. (in Chinese) doi: 10.3788/AOS URL
[5]	赵源萌, 王岭雪, 金伟其, 等. 基于色彩传递的单波段热图像彩色化处理算法[J]. 光学学报, 2009, 29(3):654-658.
	ZHAO Y M, WANG L X, JIN W Q, et al. Single-band infrared image colorization algorithm based on color transfer[J]. Acta Optica Sinica, 2009, 29(3):654-658. (in Chinese) doi: 10.3788/AOS URL
[6]	KOYAMA S, INABA Y, KASANO M, et al. A day and night vision MOS imager with robust photonic-crystal-based RGB-and-IR[J]. IEEE Transactions on Electron Devices, 2008, 55(3):754-759. doi: 10.1109/TED.2007.914828 URL
[7]	曾桂林, 苏秉华, 牛丽红, 等. 基于神经网络的微光图像增强与复原[J]. 兵工学报, 2006, 27(4): 652-654.
	ZENG G L, SU B H, NIU L H, et al. Neural network-based low-light-level image enhancement and reconstruction[J]. Acta Armamentarii, 2006, 27(4): 652-654. (in Chinese)
[8]	GUO X J, LI Y, LING H B. LIME: low-light image enhancement via illumination map estimation[J]. IEEE Transactions on Image Processing, 2016, 26(2):982-993. doi: 10.1109/TIP.2016.2639450 URL
[9]	冯维, 吴贵铭, 赵大兴, 等. 多图像融合Retinex用于弱光图像增强[J]. 光学精密工程, 2020, 28(3):736-744.
	FENG W, WU G M, ZHAO D X, et al. Multi images fusion retinex for low light image enhancement[J]. Optics and Precision Engineering, 2020, 28(3):736-744. (in Chinese) doi: 10.3788/OPE. URL
[10]	黄慧, 董林鹭, 刘小芳, 等. 改进Retinex的低光照图像增强[J]. 光学精密工程, 2020, 28(8):1835-1849.
	HUANG H, DONG L L, LIU X F, et al. Improved retinex low light image enhancement method[J]. Optics and Precision Engineering, 2020, 28(8):1835-1849. (in Chinese)
[11]	DONG X A, WANG G, PANG Y, et al. Fast efficient algorithm for enhancement of low lighting video[C]//Proceedings of 2011 IEEE International Conference on Multimedia and Expo. New York,NY,US: IEEE, 2011:1-6.
[12]	WANG W J, WEI C, YANG W H, et al. GLADNet: Low-light enhancement network with global awareness[C]//Proceedings of the 2018 13rd IEEE International Conference on Automatic Face & Gesture Recognition. New York,NY,US: IEEE, 2018:751-755.
[13]	WEI C, WANG W J, YANG W H, et al. Deep retinex decomposition for low-light enhancement: arXiv:1808.04560[R/OL]. Ithaca, NY, US: Cornell University (2018-08-01)[2021-04-01]. https://arxiv.org/abs/1808.04560.
[14]	SONG X J, HUANG J J, CAO J Z, et al. Multi-scale joint network based on Retinex theory for low-light enhancement[J]. Signal, Image and Video Processing, 2021, 15(6):1257-1264. doi: 10.1007/s11760-021-01856-y
[15]	ZHANG Y H, ZHANG J W, GUO X J. Kindling the darkness: a practical low-light image enhancer[C]//Proceedings of the 27th ACM International Conference on Multimedia. New York, NY,US:ACM, 2019:1632-1640.
[16]	黄辉先, 陈凡浩. 基于注意力机制和Retinex的低照度图像增强方法[J]. 激光与光电子学进展, 2020, 57(20):201004.
	HUANG H X, CHEN F H. Low-illumination image enhancement method based on attention mechanism and retinex[J]. Laser & Optoelectronics Progress, 2020, 57(20):201004. (in Chinese)
[17]	JIANG Y F, GONG X Y, LIU D, et al. Enlightengan: deep light enhancement without paired supervision[J]. IEEE Transactions on Image Processing, 2021, 30:2340-2349. doi: 10.1109/TIP.2021.3051462 URL
[18]	GUO C L, LI C Y, GUO J C, et al. Zero-reference deep curve estimation for low-light image enhancement[C]//Proceedings of the 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition. New York,NY, US: IEEE, 2020:1777-1786.
[19]	ZHANG Y, DI X G, ZHANG B, et al. Self-supervised image enhancement network: training with low light images only: arXiv:2002.11300[EB/OL]. Ithaca, NY, US:Cornell University(2020-02-26)[2021-04-01]. https://arxiv.org/abs/2002.11300.
[20]	唐超影, 浦世亮, 叶鹏钊, 等. 基于卷积神经网络的低照度可见光与近红外图像融合[J]. 光学学报, 2020, 40(16):1610001.
	TANG C Y, PU S L, YE P Z, et al. Fusion of low-illuminance visible and near-infrared images based on convolutional neural networks[J]. Acta Optica Sinica, 2020, 40(16):1610001. (in Chinese) doi: 10.3788/AOS URL
[21]	CHEN C, CHEN Q F, XU J, et al. Learning to see in the dark[C]//Proceedings of 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.New York,NY, US: IEEE, 2018:3291-3300.
[22]	RONNEBERGER O, FISCHER P, BROX T. U-net: Convolutional networks for biomedical image segmentation[C]//Proceedings of the 2015 18th International Conference on Medical Image Computing and Computer-assisted Intervention. Switzerland: Springer, 2015:234-241.
[23]	李俊峰. 基于RGB色彩空间自然场景统计的无参考图像质量评价[J]. 自动化学报, 2015, 41(9):1601-1615.
	LI J F. Non-reference image quality evaluation based on natural scene statistics in RGB color space[J]. Acta Automatica Sinica, 2015, 41(9):1601-1615. (in Chinese)
[24]	WOO S, PARK J, LEE J Y, et al. CBAM: Convolutional block attention module[C]//Proceedings of the 2018 15th European Conference on Computer Vision. Switzerland: Springer, 2018:3-19.
[25]	LIU J J, HOU Q, CHENG M M, et al. Improving convolutional networks with self-calibrated convolutions[C]//Proceedings of 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition. New York,NY, US: IEEE, 2020:10096-10105.
[26]	MITTAL A, SOUNDARARAJAN R, BOVIK A C. Making a "completely blind" image quality analyzer[J]. IEEE Signal Processing Letters, 2012, 20(3):209-212. doi: 10.1109/LSP.2012.2227726 URL
[27]	HILL B, ROGER T, VORHAGEN F W. Comparative analysis of the quantization of color spaces on the basis of the CIELAB color-difference formula[J]. ACM Transactions on Graphics, 1997, 16(2):109-154. doi: 10.1145/248210.248212 URL

算法	FLOPs/10⁶	参数量/10⁶	PSNR	SSIM
U-Net算法	15.5135	7.7597	20.1727	0.7673
本文算法	13.3869	6.7002	26.0631	0.8228

算法	FLOPs/10⁶	参数量/10⁶	PSNR	SSIM
U-Net算法	15.5135	7.7597	20.1727	0.7673
本文算法	13.3869	6.7002	26.0631	0.8228

算法	PSNR	SSIM	NIQE
Dong算法	13.0878	0.4033	10.5240
LIME算法	17.7791	0.4068	10.4590
Retinex Net算法	17.1443	0.4191	10.5295
Zero-DCE算法	10.5861	0.4143	9.1808
Enlighten GAN算法	13.3488	0.4747	5.4888
KinD算法	20.1770	0.7765	4.6527
ICE-Net算法	18.9233	0.5968	3.7195
本文算法	24.0554	0.7413	3.3008

算法	PSNR	SSIM	NIQE
Dong算法	13.0878	0.4033	10.5240
LIME算法	17.7791	0.4068	10.4590
Retinex Net算法	17.1443	0.4191	10.5295
Zero-DCE算法	10.5861	0.4143	9.1808
Enlighten GAN算法	13.3488	0.4747	5.4888
KinD算法	20.1770	0.7765	4.6527
ICE-Net算法	18.9233	0.5968	3.7195
本文算法	24.0554	0.7413	3.3008

参数	数值
分辨率	1984×1124
像素尺寸/μm	5.5 ×5.5
最大帧率/(帧·s^-1)	30
动态范围/dB	68
读噪声	9 e^-rms
有效图像尺寸/mm	10.56×5.94 × 12.1