Projectile-borne Image Deblurring Algorithm Based on Generative Adversarial Networks

doi:10.12382/bgxb.2022.0639

Abstract

Abstract:

The low-speed roll missile seeker has a serious blurring feature in image collection due to its motion characteristics such as dithering and rotating,which directly affects the accuracy of subsequent image algorithms for target recognition,thus affecting the guidance accuracy.To solve the above problems,a blind deblurring algorithm based on a generative adversarial network is proposed. The motion blurring simulation system is used to simulate the motion blurring such as jitter and rotation of missile-borne image,and a fuzzy dataset of missile-borne image is made.The convolution neural network is used as the basic architecture of generator and discriminator,and several loss functions are designed to optimize the network together to reduce the noise and keep the image smooth during image restoration.The de-blurring of missile-borne image is achieved,and a more stable and clear image sequence is obtained.The experimental results show that the proposed algorithm performs better in peak signal-to-noise ratio and structural similarity than other algorithms and achieves state-of-the-art performance,and accords with the subjective perception of human vision.It has practical application value.

Key words: low-speed rotational missile, missile-borne image, generative adversarial network, blind image-debluring

CLC Number:

TJ765

SU Di, WANG Shaobo, ZHANG Cheng, CHEN Zhisheng, LIU Chaoyue. Projectile-borne Image Deblurring Algorithm Based on Generative Adversarial Networks[J]. Acta Armamentarii, 2024, 45(3): 855-863.

Figures/Tables 13

Fig.1 Framework of deblurring

Fig.2 Generator network structure diagram

Fig.3 Discriminator network structure diagram

Fig.4 Blur-clear image pairs(3 rows of images are 3 images selected in the data set, and each row forms a blurring-clear image pair)

Fig.5 Pseudocode generated by motion vector blur kernel

Fig.6 Software simulation platform

Fig.7 Hardware simulation platform

Table 1 Hardware and software configuration

实验平台	环境配置
操作系统	Ubuntu 18.04
内存	DDR4 2400MHz RegECC 128GB
CPU	Intel(R) Xeon(R) E5
GPU	3×NVIDIA RTX 3090Ti
编程语言	Python 3.7
深度学习框架	Pytorch1.10

Fig.8 Restoration result of blurred image(any data taken from each behavior data set in the graph)

Table 2 Ablation experiments

方法	PSNR	SSIM
基线	32.7	0.76
基线+梯度损失	36.4	0.85
基线+总变差损失	33.6	0.79
梯度损失+总变差损失	37.1	0.89

Table 3 Deblurred results on missile-borne image dataset

参数	算法
参数	DeepDeblur	DeblurGAN	DeblurGAN-v2	SRN	本文算法
PSNR	31.5	33	36.30	35.80	37.10
SSIM	0.75	0.77	0.83	0.85	0.89
时间/s	4.24	0.82	0.34	1.57	0.33

Table 4 Comparison of image blur effects

Table 5 Deblurred results on GoPro dataset

参数	算法
参数	DeepDeblur	DeblurGAN	DeblurGAN-v2	SRN	本文算法
PSNR	29.08	28.70	29.55	30.10	29.80
SSIM	0.914	0.927	0.934	0.932	0.921
时间/s	4.33	0.85	0.35	1.60	0.35

References 30

[1]	裴慧坤, 颜源, 林国安, 等. 基于生成对抗网络的无人机图像去模糊方法[J]. 地理空间信息, 2019, 17(12):4-9.
	PEI H K, YAN Y, LIN G A, et al. UAV image deblurring method based on generative adversarial network[J]. Geospatial Information, 2019, 17(12):4-9. (in Chinese)
[2]	周玉, 彭召意. 运动模糊图像的维纳滤波复原研究[J]. 计算机工程与应用, 2009, 45(19):181-183.
	ZHOU Y, PENG Z Y. Study on wiener filtering for restoration of motion blurried image[J]. Computer Engineering and Applications, 2009, 45(19):181-183. (in Chinese) doi: 10.3778/j.issn.1002-8331.2009.19.056
[3]	郭永彩, 高潮, 王婀娜. 基于小波变换和非负支撑域递归逆滤波算法的盲目图像复原[J]. 光学学报, 2009, 29(11): 3000-3003.
	GUO Y C, GAO C, WANG E N. Blind image restoration algorithm based on wavelet transform and NAS-RIF algorithm[J]. Acta Optica Sinica, 2009, 29(11):3000-3003. (in Chinese) doi: 10.3788/AOS URL
[4]	沈瑛, 吴建华, 吴禄慎. 由约束最小二乘方法改进的图像恢复方法[J]. 数据采集与处理, 2002, 17(3):325-327.
	SHEN Y, WU J H, WU L S. Improvement of image restorationby constrained least square methods[J]. Journal of Data Acquisition and Processing, 2002, 17(3):325-327. (in Chinese)
[5]	RUDIN L, OSHER S, FATEMI E. Nonlinear total variation based noise removal algorithms[J]. Physica D:Nonlinear Phenomena, 1992, 60:259-268. doi: 10.1016/0167-2789(92)90242-F URL
[6]	RICHARDSON W H. Bayesian-based iterative method of image restoration[J]. Journal of the Optical Society of America(1917-1983), 1972, 62(1):55-59.
[7]	CHAKRABARTI A. A neural approach to blind motion deblurring[C]//Proceedings of European Conference on Computer Vision. Amsterdam,the Netherlands: Springer, 2016:221-235.
[8]	NOROOZI M, CHANDRAMOULI P, FAVARO P. Motion deblurring in the wild[C]//Proceedings of German Conference on Pattern Recognition. Basel,Switzerland: Springer, 2017:65-77.
[9]	NAH S, HYUN K T, MU L K. Deep multi-scale convolutional neural network for dynamic scene deblurring[C]// Proceedings of IEEE Conference on Computer Vision and Pattern Recognition. Honolulu, HI,US:IEEE, 2017:3883-3891.
[10]	TAO X, GAO H Y, SHEN X, et al. Scale-recurrent network for deep image deblurring[C]// Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Salt Lake City,UT,US:IEEE 2018:8174-8182.
[11]	ZHANG K, ZUO W M, CHEN Y J, et al. Beyond a gaussiandenoiser:residual learning of deep cnn for image denoising[J]. IEEE Transactions on Image Processing, 2017, 26(7):3142-3155. doi: 10.1109/TIP.83 URL
[12]	ZHANG K, ZUO W M, GU S H, et al. Learning deep CNN denoiser prior for image restoration[C]// Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.Washington,D.C.,US:IEEE, 2017:3929-3938.
[13]	DAI T, CAI J R, ZHANG Y B, et al. Second-order attention network for single image super-resolution[C]// Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition.Los Angle,CA,US:IEEE, 2019:11065-11074.
[14]	ZAMIR S W, ARORA A, KHAN S, et al. Cycleisp:real image restoration via improved data synthesis[C]// Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. Washington,D.C.,US:IEEE, 2020:2696-2705.
[15]	PAN X G, ZHAN X H, DAI B, et al. Exploiting deep generative prior for versatile image restoration and manipulation[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2021.DOI: 10.1109/TPAMI.2021.3115428.
[16]	GOODFELLOW I J, POUGET-ABADIE J, MIRZA M, et al. Generative adversarial nets[C]//Proceedings of International Conference on Neural Information Processing Systems. Kuching,Malaysia: MIT Press, 2014:2672-2680.
[17]	RADFORD A, METZ L, CHINTALA S. Unsupervised representation learning with deep convolutional generative adversarial networks:arXiv:1511.06434[R]. Ithaca,NY,US: Cornell University, 2015:1511.06434,DOI:10.48550/arXiv.1511.06434.
[18]	LEDIG C, THEIS L, HUSZAR F, et al. Photo-realistic single image super-resolution using a generative adversarial network[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Honolulu, HI,US: IEEE, 2017:4681-4690.
[19]	MIRZA M, OSINDERO S. Conditional generative adversarial nets[J]. Computer Science, 2014:2672-2680.
[20]	KUPYN O, BUDZAN V, MYKHAILYCH M, et al. DeblurGAN:blind motion deblurring using conditional adversarial networks[C]// Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. Salt Lake City,UT,US:IEEE, 2018:8183-8192.
[21]	KUPYN O, MARTYNIUK T, WU J, et al. Deblurgan-v2: eblurring (orders-of-magnitude) faster and better[C] //Proceedings of the IEEE/CVF International Conference on Computer Vision.Seoul, Korea:IEEE, 2019:8878-8887.
[22]	洪汉玉. 成像探测系统图像复原算法研究[D]. 武汉: 华中科技大学, 2004.
	HONG H Y. Research on image restoration algorithm of imaging detection system[D]. Wuhan: Huazhong University of Science and Technology, 2004. (inChinese)
[23]	梁志勇, 肖衡, 杨琳. 基于DeblurGAN对运动图像的去模糊化研究[J]. 现代计算机, 2019(31):25-27.
	LIANG Z Y, XIAO H, YANG L. Deblurring of moving images based on DeblurGAN[J]. Modern Computer, 2019(31):25-27. (in Chinese)
[24]	李从利, 张思雨, 韦哲, 等. 基于深度卷积生成对抗网络的航拍图像去厚云方法[J]. 兵工学报, 2019, 40(7):1434-1442. doi: 10.3969/j.issn.1000-1093.2019.07.013
	LI C L, ZHANG S Y, WEI Z, et al. Thickcloud removal for aerial images based on deep convolutional generative adversarial networks[J]. Acta Armamentarii, 2019, 40(7):1434-1442. (in Chinese)
[25]	YANG C, LU X, LIN Z, et al. High-resolution image inpainting using multi-scale neural patch synthesis[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Honolulu,HI,US: IEEE, 2017, 1(2):6721-6729.
[26]	MUELLER M, SMITH N, GHANEM B. A benchmark and simulator for uav tracking[C]//Proceedings of European Conference on Computer Vision. Amsterdam,the Netherlands: Springer, 2016: 445-461.
[27]	BORACCHI G, FOI A. Modeling the performance of image restoration from motion blur[J]. IEEE Transactions on Image Processing, 2012, 21(8): 3502-3517. doi: 10.1109/TIP.2012.2192126 pmid: 22481817
[28]	北京理工大学. 一种基于对抗生成网络的弹载图像去模糊方法:中国,CN202111246243.X[P].2022-01-18.
	Beijing Institute of Technology. A deblurring method for missile borne images based on generative adversarial networks:China, CN202111246243.X[P].2022-01-18. (in Chinese)
[29]	HOREA, ZIOU D. Image quality metrics:PSNRvs.SSIM[C]// Proceedings of the 20th International Conference on Pattern Recognition. Istanbul, Turkey: IEEE, 2010:2366-2369.
[30]	KINGMA D, BA J. Adam:a method for stochastic optimization:arXiv.1412.6980[R]. Ithaca,NY,US: Cornell University, 2014:1412.6980.