基于生成对抗网络的弹载图像盲去模糊算法

doi:10.12382/bgxb.2022.0639

摘要/Abstract

摘要：

低速旋转弹导引头具有抖动和旋转等运动特性,使得其采集图像存在严重的模糊特点,直接影响了后续图像算法对目标识别的准确性,进而影响制导精度。针对以上问题,提出一种基于生成对抗神经网络的盲去模糊算法。利用复合运动模糊仿真系统,实现了对弹载图像抖动和旋转等运动模糊的模拟,并制作弹载图像模糊数据集。使用卷积神经网络作为生成器和判别器的基本架构,并设计多个损失函数共同优化网络,以减小图像在修复复原过程中的噪声并保持图像平滑。实现了弹载图像的去模糊,并获得更加稳定、清晰的图像序列。实验结果表明,该算法的去模糊效果在峰值信噪比、结构相似性指标上比现阶段其他算法更好,更符合人类视觉的主观感受,具有工程应用价值。

关键词: 低速旋转弹, 弹载图像, 生成对抗网络, 盲去模糊

Abstract:

The low-speed roll missile seeker has a serious blurring feature in image collection due to its motion characteristics such as dithering and rotating,which directly affects the accuracy of subsequent image algorithms for target recognition,thus affecting the guidance accuracy.To solve the above problems,a blind deblurring algorithm based on a generative adversarial network is proposed. The motion blurring simulation system is used to simulate the motion blurring such as jitter and rotation of missile-borne image,and a fuzzy dataset of missile-borne image is made.The convolution neural network is used as the basic architecture of generator and discriminator,and several loss functions are designed to optimize the network together to reduce the noise and keep the image smooth during image restoration.The de-blurring of missile-borne image is achieved,and a more stable and clear image sequence is obtained.The experimental results show that the proposed algorithm performs better in peak signal-to-noise ratio and structural similarity than other algorithms and achieves state-of-the-art performance,and accords with the subjective perception of human vision.It has practical application value.

Key words: low-speed rotational missile, missile-borne image, generative adversarial network, blind image-debluring

中图分类号:

TJ765

苏迪, 王少博, 张成, 陈志升, 刘超越. 基于生成对抗网络的弹载图像盲去模糊算法[J]. 兵工学报, 2024, 45(3): 855-863.

SU Di, WANG Shaobo, ZHANG Cheng, CHEN Zhisheng, LIU Chaoyue. Projectile-borne Image Deblurring Algorithm Based on Generative Adversarial Networks[J]. Acta Armamentarii, 2024, 45(3): 855-863.

图/表 13

图1 去模糊框架图

Fig.1 Framework of deblurring

图2 生成器网络结构图

Fig.2 Generator network structure diagram

图3 判别器网络结构图

Fig.3 Discriminator network structure diagram

图4 模糊-清晰图像对(3行图像为数据集中任选的3张图像,每行构成一对模糊-清晰图像对)

Fig.4 Blur-clear image pairs(3 rows of images are 3 images selected in the data set, and each row forms a blurring-clear image pair)

图5 运动向量模糊核生成伪代码

Fig.5 Pseudocode generated by motion vector blur kernel

图6 软件模拟平台

Fig.6 Software simulation platform

图7 硬件模拟平台

Fig.7 Hardware simulation platform

表1 软硬件配置

Table 1 Hardware and software configuration

实验平台	环境配置
操作系统	Ubuntu 18.04
内存	DDR4 2400MHz RegECC 128GB
CPU	Intel(R) Xeon(R) E5
GPU	3×NVIDIA RTX 3090Ti
编程语言	Python 3.7
深度学习框架	Pytorch1.10

图8 模糊图像复原效果图(图中每行为数据集中任取的一个数据)

Fig.8 Restoration result of blurred image(any data taken from each behavior data set in the graph)

表2 消融实验

Table 2 Ablation experiments

方法	PSNR	SSIM
基线	32.7	0.76
基线+梯度损失	36.4	0.85
基线+总变差损失	33.6	0.79
梯度损失+总变差损失	37.1	0.89

表3 弹载图像数据集去模糊效果

Table 3 Deblurred results on missile-borne image dataset

参数	算法
参数	DeepDeblur	DeblurGAN	DeblurGAN-v2	SRN	本文算法
PSNR	31.5	33	36.30	35.80	37.10
SSIM	0.75	0.77	0.83	0.85	0.89
时间/s	4.24	0.82	0.34	1.57	0.33

表4 图像去模糊效果对比

Table 4 Comparison of image blur effects

表5 GoPro数据集去模糊效果

Table 5 Deblurred results on GoPro dataset

参数	算法
参数	DeepDeblur	DeblurGAN	DeblurGAN-v2	SRN	本文算法
PSNR	29.08	28.70	29.55	30.10	29.80
SSIM	0.914	0.927	0.934	0.932	0.921
时间/s	4.33	0.85	0.35	1.60	0.35

参考文献 30

[1]	裴慧坤, 颜源, 林国安, 等. 基于生成对抗网络的无人机图像去模糊方法[J]. 地理空间信息, 2019, 17(12):4-9.
	PEI H K, YAN Y, LIN G A, et al. UAV image deblurring method based on generative adversarial network[J]. Geospatial Information, 2019, 17(12):4-9. (in Chinese)
[2]	周玉, 彭召意. 运动模糊图像的维纳滤波复原研究[J]. 计算机工程与应用, 2009, 45(19):181-183.
	ZHOU Y, PENG Z Y. Study on wiener filtering for restoration of motion blurried image[J]. Computer Engineering and Applications, 2009, 45(19):181-183. (in Chinese) doi: 10.3778/j.issn.1002-8331.2009.19.056
[3]	郭永彩, 高潮, 王婀娜. 基于小波变换和非负支撑域递归逆滤波算法的盲目图像复原[J]. 光学学报, 2009, 29(11): 3000-3003.
	GUO Y C, GAO C, WANG E N. Blind image restoration algorithm based on wavelet transform and NAS-RIF algorithm[J]. Acta Optica Sinica, 2009, 29(11):3000-3003. (in Chinese) doi: 10.3788/AOS URL
[4]	沈瑛, 吴建华, 吴禄慎. 由约束最小二乘方法改进的图像恢复方法[J]. 数据采集与处理, 2002, 17(3):325-327.
	SHEN Y, WU J H, WU L S. Improvement of image restorationby constrained least square methods[J]. Journal of Data Acquisition and Processing, 2002, 17(3):325-327. (in Chinese)
[5]	RUDIN L, OSHER S, FATEMI E. Nonlinear total variation based noise removal algorithms[J]. Physica D:Nonlinear Phenomena, 1992, 60:259-268. doi: 10.1016/0167-2789(92)90242-F URL
[6]	RICHARDSON W H. Bayesian-based iterative method of image restoration[J]. Journal of the Optical Society of America(1917-1983), 1972, 62(1):55-59.
[7]	CHAKRABARTI A. A neural approach to blind motion deblurring[C]//Proceedings of European Conference on Computer Vision. Amsterdam,the Netherlands: Springer, 2016:221-235.
[8]	NOROOZI M, CHANDRAMOULI P, FAVARO P. Motion deblurring in the wild[C]//Proceedings of German Conference on Pattern Recognition. Basel,Switzerland: Springer, 2017:65-77.
[9]	NAH S, HYUN K T, MU L K. Deep multi-scale convolutional neural network for dynamic scene deblurring[C]// Proceedings of IEEE Conference on Computer Vision and Pattern Recognition. Honolulu, HI,US:IEEE, 2017:3883-3891.
[10]	TAO X, GAO H Y, SHEN X, et al. Scale-recurrent network for deep image deblurring[C]// Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Salt Lake City,UT,US:IEEE 2018:8174-8182.
[11]	ZHANG K, ZUO W M, CHEN Y J, et al. Beyond a gaussiandenoiser:residual learning of deep cnn for image denoising[J]. IEEE Transactions on Image Processing, 2017, 26(7):3142-3155. doi: 10.1109/TIP.83 URL
[12]	ZHANG K, ZUO W M, GU S H, et al. Learning deep CNN denoiser prior for image restoration[C]// Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.Washington,D.C.,US:IEEE, 2017:3929-3938.
[13]	DAI T, CAI J R, ZHANG Y B, et al. Second-order attention network for single image super-resolution[C]// Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition.Los Angle,CA,US:IEEE, 2019:11065-11074.
[14]	ZAMIR S W, ARORA A, KHAN S, et al. Cycleisp:real image restoration via improved data synthesis[C]// Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. Washington,D.C.,US:IEEE, 2020:2696-2705.
[15]	PAN X G, ZHAN X H, DAI B, et al. Exploiting deep generative prior for versatile image restoration and manipulation[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2021.DOI: 10.1109/TPAMI.2021.3115428.
[16]	GOODFELLOW I J, POUGET-ABADIE J, MIRZA M, et al. Generative adversarial nets[C]//Proceedings of International Conference on Neural Information Processing Systems. Kuching,Malaysia: MIT Press, 2014:2672-2680.
[17]	RADFORD A, METZ L, CHINTALA S. Unsupervised representation learning with deep convolutional generative adversarial networks:arXiv:1511.06434[R]. Ithaca,NY,US: Cornell University, 2015:1511.06434,DOI:10.48550/arXiv.1511.06434.
[18]	LEDIG C, THEIS L, HUSZAR F, et al. Photo-realistic single image super-resolution using a generative adversarial network[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Honolulu, HI,US: IEEE, 2017:4681-4690.
[19]	MIRZA M, OSINDERO S. Conditional generative adversarial nets[J]. Computer Science, 2014:2672-2680.
[20]	KUPYN O, BUDZAN V, MYKHAILYCH M, et al. DeblurGAN:blind motion deblurring using conditional adversarial networks[C]// Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. Salt Lake City,UT,US:IEEE, 2018:8183-8192.
[21]	KUPYN O, MARTYNIUK T, WU J, et al. Deblurgan-v2: eblurring (orders-of-magnitude) faster and better[C] //Proceedings of the IEEE/CVF International Conference on Computer Vision.Seoul, Korea:IEEE, 2019:8878-8887.
[22]	洪汉玉. 成像探测系统图像复原算法研究[D]. 武汉: 华中科技大学, 2004.
	HONG H Y. Research on image restoration algorithm of imaging detection system[D]. Wuhan: Huazhong University of Science and Technology, 2004. (inChinese)
[23]	梁志勇, 肖衡, 杨琳. 基于DeblurGAN对运动图像的去模糊化研究[J]. 现代计算机, 2019(31):25-27.
	LIANG Z Y, XIAO H, YANG L. Deblurring of moving images based on DeblurGAN[J]. Modern Computer, 2019(31):25-27. (in Chinese)
[24]	李从利, 张思雨, 韦哲, 等. 基于深度卷积生成对抗网络的航拍图像去厚云方法[J]. 兵工学报, 2019, 40(7):1434-1442. doi: 10.3969/j.issn.1000-1093.2019.07.013
	LI C L, ZHANG S Y, WEI Z, et al. Thickcloud removal for aerial images based on deep convolutional generative adversarial networks[J]. Acta Armamentarii, 2019, 40(7):1434-1442. (in Chinese)
[25]	YANG C, LU X, LIN Z, et al. High-resolution image inpainting using multi-scale neural patch synthesis[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Honolulu,HI,US: IEEE, 2017, 1(2):6721-6729.
[26]	MUELLER M, SMITH N, GHANEM B. A benchmark and simulator for uav tracking[C]//Proceedings of European Conference on Computer Vision. Amsterdam,the Netherlands: Springer, 2016: 445-461.
[27]	BORACCHI G, FOI A. Modeling the performance of image restoration from motion blur[J]. IEEE Transactions on Image Processing, 2012, 21(8): 3502-3517. doi: 10.1109/TIP.2012.2192126 pmid: 22481817
[28]	北京理工大学. 一种基于对抗生成网络的弹载图像去模糊方法:中国,CN202111246243.X[P].2022-01-18.
	Beijing Institute of Technology. A deblurring method for missile borne images based on generative adversarial networks:China, CN202111246243.X[P].2022-01-18. (in Chinese)
[29]	HOREA, ZIOU D. Image quality metrics:PSNRvs.SSIM[C]// Proceedings of the 20th International Conference on Pattern Recognition. Istanbul, Turkey: IEEE, 2010:2366-2369.
[30]	KINGMA D, BA J. Adam:a method for stochastic optimization:arXiv.1412.6980[R]. Ithaca,NY,US: Cornell University, 2014:1412.6980.