
浏览全部资源
扫码关注微信
1. 东南大学 自动化学院, 江苏 南京 210096
2. 东南大学 复杂工程系统测量与控制教育部重点实验室, 江苏 南京 210096
3. 江苏自动化研究所, 江苏 连云港 222061
4. 南京理工大学 计算机科学与工程学院, 江苏 南京 210094
Received:05 July 2022,
Online First:15 December 2023,
Published:30 October 2023
移动端阅览
Qiang WANG, Letian WU, Hong LI, et al. An Infrared Small Target Detection Method via Dual Network Collaboration[J]. Acta Armamentarii, 2023, 44(10): 3165-3176.
Qiang WANG, Letian WU, Hong LI, et al. An Infrared Small Target Detection Method via Dual Network Collaboration[J]. Acta Armamentarii, 2023, 44(10): 3165-3176. DOI: 10.12382/bgxb.2022.0605.
红外弱小目标检测在预警系统和导弹制导中具有重要的作用
一直是红外图像处理中颇受关注的研究方向。由于红外弱小目标具有信杂比低、尺寸小、形状结构不明显和纹理弱等特点
现有的通用目标检测和语义分割网络直接应用到红外弱小目标检测效果不佳
为此提出一种基于双支网络协作的红外弱小目标检测网络(DualNet)。将检测任务划分成两个子任务
即降低漏检和降低虚警
进而设计两个不同的网络架构分别处理
并利用加权融合损失函数将两支网络信息整合
使得DualNet能够有效地平衡漏检率和虚警率。在自建数据集上的实验结果表明:DualNet相较于通用性能较好的FCN、DeepLabv3、cGAN以及U-net语义分割网络模型具备更高的准确率和鲁棒性
其在F1-measure指标上提高了8%;在SIRST公开数据集上的检测性能也显著超过了基于深度学习的红外目标检测模型ACM和MDvsFA-cGAN
以及多个经典的非深度学习红外弱小目标检测方法。研究结果表明
所提出的方法能够有效提高红外弱小目标的检测精度。
Infrared small target detection (ISTD) is a heated topic in infrared image processing
and it is intensively applied in early warning systems and missile guidance. ISTD faces significant challenges such as low signal-to-noise ratio (SNR)
small size
lack of distinct shape or structure
and weak texture
making it a demanding task. The performance of conventional object detection networks and semantic segmentation networks considerably deteriorates when applied directly to ISTD tasks. To address this issue
this paper proposes a new dual network collaboration-based image semantic segmentation network for ISTD
termed as DualNet. DualNet divides the task into two sub-tasks
namely reducing missed detections and reducing false alarms
with two sub-networks focusing on their respective targets (with cost reduced) by employing a weighted loss function to integrate sub-network information. DualNet effectively balances the miss detection rate and false alarm rate. Experimental results show that DualNet outperforms general neural network models (e.g. FCN
DeepLabv3
cGAN and U-net) on the ISTD task
with an improved F1-measure by 0.08. Furthermore
our model outperforms ACM and MDvsFA-cGAN
two most representative ISTD models based on deep learning
and several non-deep-learning-based ISTD methods.
NIE J Y , QU S C , WEI Y T , et al . An infrared small target detection method based on multiscale local homogeneity measure [J ] . Infrared Physics & Technology . 2018 , 90 : 186 - 194 .
ZHANG H , ZHANG L , YUAN D , et al . Infrared small target detection based on local intensity and gradient properties [J ] . Infrared Physics & Technology , 2018 , 89 : 88 - 96 .
赵耀 . 红外弱小目标检测算法研究 [D ] . 杭州 : 浙江大学 , 2015 .
ZHAO Y . Research on infrared small target detection algorithms [D ] . Hangzhou : Zhejiang University , 2015 . (in Chinese)
万丽丽 . 基于背景预测的红外目标检测 [D ] . 武汉 : 华中科技大学 , 2015 .
WAN L L . Infrared target detection based on background prediction [D ] . WuHan : Huazhong University of Science and Technology , 2015 . (in Chinese)
韩金辉 , 董兴浩 , 李知静 , 等 . 基于局部对比度机制的红外弱小目标检测算法 [J ] . 红外技术 , 2021 , 43 ( 4 ): 357 - 366 .
HAN J H , DONG X H , LI Z J , et al . Infrared small dim target detection based on local contrast mechanism [J ] . Infrared Technology , 2021 , 43 ( 4 ): 357 - 366 . (in Chinese)
危水根 , 王程伟 , 陈震 , 等 . 基于视觉注意机制的红外弱小目标检测 [J ] . 光子学报 , 2021 , 50 ( 1 ): 0110001 .
WEI S G , WANG C W , CHEN Z , et al . Infrared dim target detection based on human visual mechanism [J ] . Acta Photonic Sinicia , 2021 , 50 ( 1 ): 0110001 . (in Chinese)
朱硕雅 , 杨德振 , 贾鹏 , 等 . 时空联合红外小目标检测算法的设计与实现 [J ] . 红外与激光 , 2021 , 51 ( 3 ): 388 - 392 .
ZHU S Y , YANG D Z , JIA P , et al . Design and implementation of space-time combined infrared small target detection algorithm [J ] . Laser & Infrared , 2021 , 5 ( 3 ): 388 - 392 . (in Chinese)
娄康 . 基于深度学习与生物视觉机制的红外弱小目标检测与跟踪方法研究 [D ] . 镇江 : 江苏科技大学 , 2020 .
LOU K . Research on infrared weak and small target detection and tracking based on deep learning and biological vision mechanism [D ] . Zhenjiang : Jiangsu University of Science and Technology , 2020 . (in Chinese)
黄乐弘 . 基于深度学习的空中红外目标检测方法研究 [D ] . 北京 : 中国科学院大学 , 2020 .
HUANG L H . Research on infrared target detection in air based on deep learning [D ] . Beijiing : University of Chinese Academy of Sciences , 2020 . (in Chinese)
JOSEPH R , ALI F . YOLOv3: an incremental improvement: arXiv:1804.02767 [R/OL ] . Ithaca,NY , US : Cornell University , 2018 .(2018-04-08) [2020-04-01 ] . https://arxiv.org/abs/1804.02767. https://arxiv.org/abs/1804.02767 https://arxiv.org/abs/1804.02767
杨其利 , 周炳红 , 郑伟 . 基于全卷积递归网络的弱小目标检测方法 [J ] . 光学学报 , 2020 , 40 ( 13 ): 43 - 55 .
YANG Q L , ZHOU B H , ZHENG W . Dim and small target detection based on fully convolutional recursive network [J ] . ActaPhotonic Sinicia , 2020 , 40 ( 13 ): 43 - 55 . (in Chinese)
梁杰 , 李磊 , 任君 . 基于深度学习的红外图像遮挡干扰检测方法 [J ] . 兵工学报 , 2019 , 40 ( 7 ): 1401 - 1410 . DOI: 10.3969/j.issn.1000-1093.2019.07.009 http://doi.org/10.3969/j.issn.1000-1093.2019.07.009 红外成像体制进行目标探测和识别时,烟幕、云雾等遮挡类干扰会改变目标特征导致目标识别错误。通过对遮挡干扰区域进行定位和类型判断,在识别处理时进行针对性处理可大大降低识别虚警率,提高识别的抗干扰能力。为此,提出一种基于深度学习单通道检测器改进的红外图像厚云、烟幕遮挡干扰检测方法。该方法通过网络多层特征的复用和融合,实现了多尺度预测;利用动态锚框模块改进锚框机制,提高了检测精度;将网络中的卷积层与批归一化层合并,提高了检测速度;引入中心损失函数对分类函数进行优化,提高了网络对遮挡物的分类能力。在网络训练过程中,提出一种红外样本增广方法,对数据量进行有效扩充,解决了红外图像训练样本获取难的问题。实验结果表明,与未改进前的算法相比,在速度基本相同情况下改进的遮挡干扰检测方法检测精度提高3.7%,有效地解决了复杂环境下红外自动目标识别系统抗干扰能力较弱的问题。
LIANG J , LI L , REN J . Infrared image occlusion interference detection method based on deep learning [J ] . Acta Armamentarii , 2019 , 40 ( 7 ): 1401 - 1410 . (in Chinese) DOI: 10.3969/j.issn.1000-1093.2019.07.009 http://doi.org/10.3969/j.issn.1000-1093.2019.07.009 The occlusion interference of smoke screen and cloud can change the target characteristics and cause the target identification errors when an infrared imaging system detects and identifies a target. The targeted processing during the identification process can greatly reduce the identification false alarm rate and improve the anti-interference ability of identification by performing the positioning and type judgment of occlusion interference area. To this end, an infrared image thick cloud and smoke screen occlusion interference detection method based on improved deep learning single channel detector is proposed. In the proposed method, the multi-scale prediction is realized by multiplexing and merging the multi-layer features of network, and the detection precision is inceased by using the dynamic anchor frame module to improve the anchor frame mechanism. The detection speed is inceased by merging the convolutional layer and the batch normalization layer in the network, and the classification ability of network for the obstruction is improved by introducing the central loss function to optimize the classification function. An infrared sample augmentation method is proposed for network training, which effectively expands the data volume and solves the problem of difficult acquisition of infrared image training samples. The experimental results show that the proposed method is used to improve the detection accuracy by 3.7% at the same speed, which effectively solves the problem of weak anti-interference ability of infrared automatic target recognition system in complex environment. Key
田萱 , 王亮 , 丁琪 . 基于深度学习的图像语义分割方法综述 [J ] . 软件学报 , 2019 , 30 ( 2 ): 440 - 468 .
TIAN X , WANG L , DING Q . Review of image semantic segmentation based on deep learning [J ] . Journal of Software , 2019 , 30 ( 2 ): 440 - 468 . (in Chinese)
SHELHAMER E , LONG J , DARRELL T . Fully convolutional networks for semantic segmentation [J ] . IEEE Transactions on Pattern Analysis & Machine Intelligence , 2014 , 39 ( 4 ): 640 - 651 .
RONNEBERGER O , FISCHER P , BROX T . U-Net: convolutional networks for biomedical image segmentation [C ] // Proceedings of Medical Image Computing and Computer-Assisted Intervention. Cham, Switzerland:Springer , 2015 : 234 - 241 .
GIRSHICK R , DONAHUE J , DARRELL T , et al . Rich feature hierarchies for accurate object detection and semantic segmentation [C ] // Proceedings of 2014 IEEE Conference on Computer Vision and Pattern Recognition. Columbus, OH, US:IEEE , 2014 : 580 - 587 .
CHEN L C , PAPANDREOU G , KOKKINOS I , et al . DeepLab: Semantic image segmentation with deep convolutional nets, atrous convolution, and fully connected CRFs [J ] . IEEE Transactions on Pattern Analysis & Machine Intelligence , 2018 , 40 ( 4 ): 834 - 848
YU F , KOLTUN V . Multi-scale context aggregation by dilated convolutions: arXiv:1511.07122 [R/OL ] . Ithaca, NY , US : Cornell University , 2016 (2016-04-30) [2020-04-01 ] . https://arxiv.org/abs/1511.07122. https://arxiv.org/abs/1511.07122 https://arxiv.org/abs/1511.07122
CHEN L C , PAPANDREOU G , SCHROFF F , et al . Rethinking atrous convolution for semantic image segmentation: arXiv:1706.05587 [R/OL ] . Ithaca, NY , US : Cornell University , 2017 (2017-12-05) [2020-05-01 ] . https://arxiv.org/abs/1706.05587. https://arxiv.org/abs/1706.05587 https://arxiv.org/abs/1706.05587
LIU S , QI X J , SHI J P , et al . Multi-scale patch aggregation (MPA) for simultaneous detection and segmentation [C ] // Proceedings of 2016 IEEE Conference on Computer Vision and Pattern Recognition. Las Vegas, NV, US:IEEE , 2016 : 3141 - 3149 .
HE K M , ZHANG X , REN S , et al . Spatial pyramid pooling in deep convolutional networks for visual recognition [J ] . IEEE Transactions on Pattern Analysis and Machine Intelligence , 2015 , 37 ( 9 ): 1904 - 1916 . DOI: 10.1109/TPAMI.2015.2389824 http://doi.org/10.1109/TPAMI.2015.2389824 Existing deep convolutional neural networks (CNNs) require a fixed-size (e.g., 224 × 224) input image. This requirement is "artificial" and may reduce the recognition accuracy for the images or sub-images of an arbitrary size/scale. In this work, we equip the networks with another pooling strategy, "spatial pyramid pooling", to eliminate the above requirement. The new network structure, called SPP-net, can generate a fixed-length representation regardless of image size/scale. Pyramid pooling is also robust to object deformations. With these advantages, SPP-net should in general improve all CNN-based image classification methods. On the ImageNet 2012 dataset, we demonstrate that SPP-net boosts the accuracy of a variety of CNN architectures despite their different designs. On the Pascal VOC 2007 and Caltech101 datasets, SPP-net achieves state-of-the-art classification results using a single full-image representation and no fine-tuning. The power of SPP-net is also significant in object detection. Using SPP-net, we compute the feature maps from the entire image only once, and then pool features in arbitrary regions (sub-images) to generate fixed-length representations for training the detectors. This method avoids repeatedly computing the convolutional features. In processing test images, our method is 24-102 × faster than the R-CNN method, while achieving better or comparable accuracy on Pascal VOC 2007. In ImageNet Large Scale Visual Recognition Challenge (ILSVRC) 2014, our methods rank #2 in object detection and #3 in image classification among all 38 teams. This manuscript also introduces the improvement made for this competition.
CHOLLET F . Xception:deep learning with depthwise separable convolutions [C ] // Proceedings of 2017 IEEE International Conference on Computer Vision.Venice, Italy:IEEE , 2017 : 764 - 773 .
WANG P Q , CHEN P F , YUAN Y , et al . Understanding convolution for semantic segmentation [C ] // Proceedings of 2018 IEEE Winter Conference on Applications of Computer Vision.Istanbul, Turkey:IEEE , 2018 : 1451 - 1460 .
DAI J F , QI H Z , XIONG Y W , et al . Deformable convolutional networks [C ] // Proceedings of 2017 IEEE International Conference on Computer Vision. Venice, Italy:IEEE , 2017 : 764 - 773 .
BADRINARAYANAN V , KENDALL A , CIPOLLA R . Segnet:a deep convolutional encoder-decoder architecture for image segmentation [J ] . IEEE Transactions on Pattern Analysis and Machine Intelligence , 2015 , 39 ( 12 ): 2481 - 2495 . DOI: 10.1109/TPAMI.34 http://doi.org/10.1109/TPAMI.34 https://ieeexplore.ieee.org/xpl/RecentIssue.jsp?punumber=34 https://ieeexplore.ieee.org/xpl/RecentIssue.jsp?punumber=34
NOH H , HONG S , HAN B . Learning deconvolution network for semantic segmentation [C ] // Proceedings of 2015 IEEE International Conference on Computer Vision. Santiago, Chile:IEEE , 2015 : 1520 - 1528 .
LIU Z W , LI X X , LUO P , et al . Semantic image segmentation via deep parsing network [C ] // Proceedings of 2015 IEEE International Conference on Computer Vision.Santiago, Chile:IEEE , 2015 : 1377 - 1385
CHEN L C , YANG Y , WANG J , et al . Attention to scale: Scale-aware semantic image segmentation [C ] // Proceedings of 2016 IEEE Conference on Computer Vision and Pattern Recognition. Las Vegas, NV, US:IEEE , 2016 : 3640 - 3649 .
ROY A G , NAVAB N , WACHINGER C . Concurrent Spatial and Channel Squeeze & Excitation in Fully Convolutional Networks [C ] // Proceedings of the 21st International Conference on Medical Image Computing and Computer Assisted Intervention.Granada, Spain:IEEE , 2018 : 421 - 429 .
YU Q H , XIE L X , WANG Y , et al . Recurrent saliency transformation network: incorporating multi-stage visual cues for small organ segmentation [C ] // Proceedings of 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.Salt Lake City,UT, US:IEEE , 2018 : 8280 - 8289 .
LUC P , COUPRIE C , CHINTALA S , et al . Semantic segmentation using adversarial networks:arXiv:1611.08408 [R/OL ] . Ithaca, NY , US : Cornell University , 2016 (2016-11-25)[2020-06-01 ] . https://arxiv.org/abs/1611.08408. https://arxiv.org/abs/1611.08408 https://arxiv.org/abs/1611.08408
ZHAO H S , SHI J P , QI X J . Pyramid scene parsing network [C ] // Proceedings of 2017 IEEE Conference on Computer Vision and Pattern Recognition. Honolulu, HI, US:IEEE , 2017 : 6230 - 6239 .
LIN TY , GOYAL P , GIRSHICK R , et al . Focal loss for dense object detection [J ] . IEEE Transactions on Pattern Analysis and Machine Intelligence , 2020 , 42 ( 2 ): 318 - 327 . DOI: 10.1109/TPAMI.34 http://doi.org/10.1109/TPAMI.34 https://ieeexplore.ieee.org/xpl/RecentIssue.jsp?punumber=34 https://ieeexplore.ieee.org/xpl/RecentIssue.jsp?punumber=34
RIVEST J F , FORTIN R . Detection of dim targets in digital infrared imagery by morphological image processing [J ] . Optical Engineering , 1996 , 35 ( 7 ): 1886 - 1893 . DOI: 10.1117/1.600620 http://doi.org/10.1117/1.600620 http://opticalengineering.spiedigitallibrary.org/article.aspx?doi=10.1117/1.600620 http://opticalengineering.spiedigitallibrary.org/article.aspx?doi=10.1117/1.600620
DESHPANDE S D , ER M H , VENKATESWARLU R , et al . Max-mean and max-median filters for detection of small targets [C ] // Proceedings of Signal and Data Processing of Small Targets 1999.Denver,CO,US:SPIE , 1999 , 3809 : 74 - 83 .
HAN J , MORADI S , FARAMARZI I , et al . Infrared small target detection based on the weighted strengthened local contrast measure [J ] . IEEE Geoscience and Remote Sensing Letters , 2021 , 18 ( 9 ): 1670 - 1674 . DOI: 10.1109/LGRS.2020.3004978 http://doi.org/10.1109/LGRS.2020.3004978 https://ieeexplore.ieee.org/document/9130832/ https://ieeexplore.ieee.org/document/9130832/
HAN J , MORADI S , FARAMARZI I , et al . A local contrast method for infrared small-target detection utilizing a tri-layer window [J ] . IEEE Geoscience and Remote Sensing Letters , 2019 , 17 ( 10 ): 1822 - 1826 . DOI: 10.1109/LGRS.8859 http://doi.org/10.1109/LGRS.8859 https://ieeexplore.ieee.org/xpl/RecentIssue.jsp?punumber=8859 https://ieeexplore.ieee.org/xpl/RecentIssue.jsp?punumber=8859
GAO C Q , MENG D Y , YANG Y , et al . Infrared patch-image model for small target detection in a single image [J ] . IEEE Transactions on Image Processing , 2013 , 22 ( 12 ): 4996 - 5009 . The robust detection of small targets is one of the key techniques in infrared search and tracking applications. A novel small target detection method in a single infrared image is proposed in this paper. Initially, the traditional infrared image model is generalized to a new infrared patch-image model using local patch construction. Then, because of the non-local self-correlation property of the infrared background image, based on the new model small target detection is formulated as an optimization problem of recovering low-rank and sparse matrices, which is effectively solved using stable principle component pursuit. Finally, a simple adaptive segmentation method is used to segment the target image and the segmentation result can be refined by post-processing. Extensive synthetic and real data experiments show that under different clutter backgrounds the proposed method not only works more stably for different target sizes and signal-to-clutter ratio values, but also has better detection performance compared with conventional baseline methods.
ZHANG L D , PENG L B , ZHANG T F , et al . Infrared small target detection via non-convex rank approximation minimization joint l 2,1 norm [J ] . Remote Sensing , 2018 , 10 ( 11 ): 1821 . DOI: 10.3390/rs10111821 http://doi.org/10.3390/rs10111821 http://www.mdpi.com/2072-4292/10/11/1821 http://www.mdpi.com/2072-4292/10/11/1821 To improve the detection ability of infrared small targets in complex backgrounds, a novel method based on non-convex rank approximation minimization joint l2,1 norm (NRAM) was proposed. Due to the defects of the nuclear norm and l1 norm, the state-of-the-art infrared image-patch (IPI) model usually leaves background residuals in the target image. To fix this problem, a non-convex, tighter rank surrogate and weighted l1 norm are instead utilized, which can suppress the background better while preserving the target efficiently. Considering that many state-of-the-art methods are still unable to fully suppress sparse strong edges, the structured l2,1 norm was introduced to wipe out the strong residuals. Furthermore, with the help of exploiting the structured norm and tighter rank surrogate, the proposed model was more robust when facing various complex or blurry scenes. To solve this non-convex model, an efficient optimization algorithm based on alternating direction method of multipliers (ADMM) plus difference of convex (DC) programming was designed. Extensive experimental results illustrate that the proposed method not only shows superiority in background suppression and target enhancement, but also reduces the computational complexity compared with other baselines.
DAI Y M , WU Y Q . Reweighted infrared patch-tensor model with both nonlocal and local priors for single-frame small target detection [J ] . IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing , 2017 , 10 ( 8 ): 3752 - 3767 . DOI: 10.1109/JSTARS.4609443 http://doi.org/10.1109/JSTARS.4609443 https://ieeexplore.ieee.org/xpl/RecentIssue.jsp?punumber=4609443 https://ieeexplore.ieee.org/xpl/RecentIssue.jsp?punumber=4609443
ZHANG L D , PENG Z M . Infrared small target detection based on partial sum of the tensor nuclear norm [J ] . Remote Sensing , 2019 , 11 ( 4 ): 382 . DOI: 10.3390/rs11040382 http://doi.org/10.3390/rs11040382 http://www.mdpi.com/2072-4292/11/4/382 http://www.mdpi.com/2072-4292/11/4/382 Excellent performance, real time and strong robustness are three vital requirements for infrared small target detection. Unfortunately, many current state-of-the-art methods merely achieve one of the expectations when coping with highly complex scenes. In fact, a common problem is that real-time processing and great detection ability are difficult to coordinate. Therefore, to address this issue, a robust infrared patch-tensor model for detecting an infrared small target is proposed in this paper. On the basis of infrared patch-tensor (IPT) model, a novel nonconvex low-rank constraint named partial sum of tensor nuclear norm (PSTNN) joint weighted l1 norm was employed to efficiently suppress the background and preserve the target. Due to the deficiency of RIPT which would over-shrink the target with the possibility of disappearing, an improved local prior map simultaneously encoded with target-related and background-related information was introduced into the model. With the help of a reweighted scheme for enhancing the sparsity and high-efficiency version of tensor singular value decomposition (t-SVD), the total algorithm complexity and computation time can be reduced dramatically. Then, the decomposition of the target and background is transformed into a tensor robust principle component analysis problem (TRPCA), which can be efficiently solved by alternating direction method of multipliers (ADMM). A series of experiments substantiate the superiority of the proposed method beyond state-of-the-art baselines.
SUN Y , YANG J G , AN W . Infrared dim and small target detection via multiple subspace learning and spatial-temporal patch-tensor model [J ] . IEEE Transactions on Geoscience and Remote Sensing , 2021 , 59 ( 5 ): 3737 - 3752 . DOI: 10.1109/TGRS.2020.3022069 http://doi.org/10.1109/TGRS.2020.3022069 https://ieeexplore.ieee.org/document/9203993/ https://ieeexplore.ieee.org/document/9203993/
DAI Y M , WU Y Q , ZHOU Y , et al . Asymmetric contextual modulation for infrared small target detection [C ] // Proceedings of 2021 IEEE Winter Conference on Applications of Computer Vision. Waikoloa, HI, US:IEEE , 2021 : 950 - 959 .
WANG H , ZHOU L P , WANG L . Miss detection vs. false alarm: adversarial learning for small object segmentation in infrared images [C ] // Proceedings of 2019 IEEE/CVF International Conference on Computer Vision.Seoul , Korea : IEEE , 2019 : 8509 - 8518 .
0
Views
407
下载量
0
CNKI被引量
Publicity Resources
Related Articles
Related Author
Related Institution
京公网安备11010802024360号