基于深度卷积生成对抗网络的航拍图像去厚云方法

doi:10.3969/j.issn.1000-1093.2019.07.013

摘要/Abstract

摘要： 针对航空图像中厚云去除的难题，提出一种基于深度卷积生成对抗网络的航拍图像去厚云方法。将图像中被云遮挡的区域看作图像修复问题中的缺失部分，利用卷积神经网络的对抗学习补偿缺失信息。设计了包括生成器-鉴别器的深度卷积生成对抗网络模型。生成器采用编码器-解码器结构，构建了包含重建损失、对抗损失和总变差损失的联合损失函数，不断训练以生成云区的预测图像；鉴别器衡量生成图像的真实性，以对抗损失作为损失函数。通过不断迭代联合优化生成器和鉴别器，以使网络预测性能提高。引入泊松图像编辑平滑边界，以降低颜色差异和边界伪迹的影响。在模拟含云图像与真实含云图像上实验结果表明，所提出方法的去云效果在峰值信噪比、结构相似性、自然图像无参考质量评价算法及其改进算法指标优于经典方法，更符合人眼主观感受，且具有较小的运算复杂度。

关键词: 航拍图像, 厚云去除, 深度卷积生成对抗网络, 泊松图像编辑

Abstract: A thick cloud removal method for aerial images based on deep convolution generative adversa- rial network (DCGAN) is proposed. In the proposed method, a region covered by cloud in image is regarded as a missing part in the image inpainting, and the adversarial learning of convolutional neural network is used to compensate the missing information. A DCGAN model including generator-discriminator is designed. The generator is an encoder-decoder structure with a joint loss function, including reconstruction loss, adversarial loss and total variation loss. The discriminator is used to measure the authenticity of the generated image with a loss function of adversarial loss. The prediction performance of network is improved by iterating the joint optimization of generator and discriminator. Poisson image editing is introduced to reduce the influences of color difference and boundary artifact. The experimental results on both simulated and real cloud images show that the proposed method is better than the classic method in terms of peak signal to noise ratio, structural similarity index measure, natural image no-reference quality evaluation algorithm and its improved algorithm, and has a small computational complexity. Key

Key words: aerialimage, thickcloudremoval, deepconvolutionalgenerativeadversarialnetwork, Poissonimageediting

中图分类号:

TP391.413

李从利，张思雨，韦哲，薛松. 基于深度卷积生成对抗网络的航拍图像去厚云方法[J]. 兵工学报, 2019, 40(7): 1434-1442.

LI Congli, ZHANG Siyu, WEI Zhe, XUE Song. Thick Cloud Removal for Aerial Images Based on Deep Convolutional Generative Adversarial Networks[J]. Acta Armamentarii, 2019, 40(7): 1434-1442.

参考文献

［1］ ZHANG C R, LI W D, TRAVIS D J. Restoration of clouded
pixels in multispectral remotely sensed imagery with cokriging[J]. International Journal of Remote Sensing, 2009, 30(9): 2173-2195.
[2］ WANG Z Y, JIN J Q, LIANG J W, et al. A new cloud removal algorithm for multi-spectral images[J]. Proceedings of SPIE, 2005, 6043: 230-240.
[3］ FENG C, MA J W, DAI Q, et al. An improved method for cloud removal in ASTER data change detection[C]∥Proceedings of 2004 IEEE International Geoscience and Remote Sensing Symposium. Anchorage, AK, US: IEEE, 2004: 3387-3389.
[4］ LI M, LIEW S C, KWOH L K. Automated production of cloud-free and cloud shadow-free image mosaics from cloudy satellite imagery[C]∥Proceedings of the XXth ISPRS Congress. Istanbul, Turkey: International Society for Photogrammetry and Remote Sensing, 2004: 12-13.
[5］ HELMER E H, RUEFENACHT B. Cloud-free satellite image mosaics with regression trees and histogram matching[J]. Photogrammetric Engineering & Remote Sensing, 2005, 71(9): 1079-1089.
[6］ LIN C H, TSAI P H, LAI K H, et al. Cloud removal from multitemporal satellite images using information cloning[J]. IEEE transactions on Geoscience and Remote Sensing, 2013, 51(1): 232-241.
[7］ ZHANG C Y, LI Z W, CHENG Q, et al. Cloud removal by fusing multi-source and multi-temporal images[C]∥Proceedings of IEEE International Geoscience and Remote Sensing Symposium. Fort Worth, TX, US: IEEE, 2017: 2577-2580.
[8］ CHEN F, ZHAO Z M, PENG L, et al. Clouds and cloud shadows removal from high-resolution aerial images[C]∥Proceedings of IEEE International Geoscience and Remote Sensing Symposium. Seoul, South Korea: IEEE, 2005:4256-4259.
[9］ MAALOUF A, CARR P, AUGEREAU B, et al. A bandelet-based inpainting technique for clouds removal from remotely sensed images[J]. IEEE Transactions on Geoscience and Remote Sensing, 2009, 47(7): 2363-2371.

[10］ LORENZI L, MELGANI F, MERCIER G. Inpainting strategies for reconstruction of missing data in VHR images[J]. IEEE Geoscience and Remote Sensing Letters, 2011, 8(5): 914-918.
[11］ CRIMINISI A, PREZ P, TOYAMA K. Region filling and object removal by exemplar-based image inpainting[J]. IEEE Transactions on Image Processing, 2004, 13(9): 1200-1212.
[12］ HAYS J, EFROS A A. Scene completion using millions of photographs[J]. Communications of the ACM, 2008, 51(10): 87-94.
[13］ SZEGEDY C, ZAREMBA W, SUTSKEVER I, et al. Intriguing properties of neural networks[EB/OL]. [2017-06-07]. https:∥arxiv.org/pdf/1312.6199.pdf.
[14］ GOODFELLOW I, POUGET-ABADIE J, MIRZA M, et al. Generative adversarial nets[C]∥Proceedings of Advances in Neural Information Processing Systems. Montreal, Canada: Curran Associates, Inc., 2014: 2672-2680.
[15］林懿伦, 戴星原, 李力, 等. 人工智能研究的新前线: 生成式对抗网络[J]. 自动化学报, 2018, 44(5): 775-792.
LIN Y L, DAI X Y, LI L, et al. The new frontier of AI research: generative adversarial networks[J]. Acta Automatica Sinica, 2018, 44(5): 775-792. (in Chinese)
[16］赵树阳, 李建武. 基于生成对抗网络的低秩图像生成方法[J]. 自动化学报, 2018, 44(5): 829-839.
ZHAO S Y, LI J W. Generative adversarial network for generating low-rank images[J]. Acta Automatica Sinica, 2018, 44(5): 829-839. (in Chinese)
[17］孔德江, 汤斯亮, 吴飞. 时空嵌入式生成对抗网络的地点预测方法[J]. 模式识别与人工智能, 2018, 31(1): 49-60.
KONG D J, TANG S L, WU F. Location prediction via generative adversarial network with spatial temporal embedding[J]. Pattern Recognition and Artificial Intelligence, 2018, 31(1): 49-60. (in Chinese)
[18］ GUPTA A, ZOU J. Feedback GAN (FBGAN) for DNA: a novel feedback-loop architecture for optimizing protein functions[EB/OL]. [2018-04-05]. https:∥arxiv.org/pdf/1804.01694.pdf.
[19] RADFORD A, METZ L, CHINTALA S. Unsupervised representation learning with deep convolutional generative adversarial networks[EB/OL]. [2016-01-07]. https:∥arxiv.org/pdf/1511.06434.pdf.
[20］ PATHAK D, KRAHENBUHL P, DONAHUE J, et al. Context encoders: feature learning by inpainting[C]∥Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Las Vegas, NV, US: IEEE, 2016: 2536-2544.
[21］ STRONG D, CHAN T. Edge-preserving and scale-dependent properties of total variation regularization[J]. Inverse Problems, 2003, 19(6): 165-187.
[22］ PREZ P, GANGNET M, BLAKE A. Poisson image editing[J]. ACM Transactions on Graphics, 2003, 22(3): 313-318.
[23］ RASMUS A, BERGLUND M, HONKALA M, et al. Semi-supervised learning with ladder networks[C]∥Proceedings of the 29th Annual Conference on Neural Information Processing Systems. Montreal, Canada: Curran Associates, Inc., 2015: 3546-3554.
[24］ KRIZHEVSKY A, SUTSKEVER I, HINTON G E. Imagenet classification with deep convolutional neural networks[C]∥Proceedings of the 25th International Conference on Neural Information Processing Systems. Lake Tahoe, NV, US: Curran Associates, Inc., 2012: 1097-1105.
[25］ YANG C, LU X, LIN Z, et al. High-resolution image inpainting using multi-scale neural patch synthesis[C]∥Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Honolulu, HI, US: IEEE, 2017, 1(2): 6721-6729.
[26］ DARABI S, SHECHTMAN E, BARNES C, et al. Image melding: combining inconsistent images using patch-based synthesis[J]. ACM Transactions on Graphics, 2012, 31(4): 82.
[27］ HUANG J B, KANG S B, AHUJA N, et al. Image completion using planar structure guidance[J]. ACM Transactions on graphics (TOG), 2014, 33(4): 129.
[28］ YEH R, CHEN C, LIM T Y, et al. Semantic image inpainting with perceptual and contextual losses[EB/OL]. [2016-07-26]. https:∥arxiv.org/pdf/1607.07539.pdf.
[29］ KINGMA D, BA J. Adam: a method for stochastic optimization[EB/OL]. [2017-01-30]. https:∥arxiv.org/pdf/1412.6980.pdf.
[30］ WANG Z, BOVIK A C, SHEIKH H R, et al. Image quality assessment: from error visibility to structural similarity[J]. IEEE Transactions on Image Processing, 2004, 13(4): 600-612.
[31］ MITTAL A, MURALIDHAR G S, GHOSH J, et al. Blind image quality assessment without human training using latent quality factors[J]. IEEE Signal Processing Letters, 2012, 19(2):75-78.
[32］ ZHANG L, ZHANG L, BOVIK A C. A feature-enriched completely blind image quality evaluator[J]. IEEE Transactions on Image Processing, 2015, 24(8):2579-2591.

第40卷第7期
2019 年7月兵工学报ACTA
ARMAMENTARIIVol.40No.7Jul.2019