An Infrared Small Target Detection Method via Dual Network Collaboration

doi:10.12382/bgxb.2022.0605

Abstract

Abstract:

Infrared small target detection (ISTD) is a heated topic in infrared image processing, and it is intensively applied in early warning systems and missile guidance. ISTD faces significant challenges such as low signal-to-noise ratio (SNR), small size, lack of distinct shape or structure, and weak texture, making it a demanding task. The performance of conventional object detection networks and semantic segmentation networks considerably deteriorates when applied directly to ISTD tasks. To address this issue, this paper proposes a new dual network collaboration-based image semantic segmentation network for ISTD, termed as DualNet. DualNet divides the task into two sub-tasks, namely reducing missed detections and reducing false alarms, with two sub-networks focusing on their respective targets (with cost reduced) by employing a weighted loss function to integrate sub-network information. DualNet effectively balances the miss detection rate and false alarm rate. Experimental results show that DualNet outperforms general neural network models (e.g. FCN, DeepLabv3, cGAN and U-net) on the ISTD task, with an improved F1-measure by 0.08. Furthermore, our model outperforms ACM and MDvsFA-cGAN, two most representative ISTD models based on deep learning, and several non-deep-learning-based ISTD methods.

Key words: infrared dim small target, target detection, dual network collaboration, semantic segmentation, deep learning

CLC Number:

TP391.41

WANG Qiang, WU Letian, LI Hong, WANG Yong, WANG Huan, YANG Wankou. An Infrared Small Target Detection Method via Dual Network Collaboration[J]. Acta Armamentarii, 2023, 44(10): 3165-3176.

Figures/Tables 16

References 44

[1]	NIE J Y, QU S C, WEI Y T, et al. An infrared small target detection method based on multiscale local homogeneity measure[J]. Infrared Physics & Technology. 2018, 90: 186-194.
[2]	ZHANG H, ZHANG L, YUAN D, et al. Infrared small target detection based on local intensity and gradient properties[J]. Infrared Physics & Technology, 2018, 89: 88-96.
[3]	赵耀. 红外弱小目标检测算法研究[D]. 杭州: 浙江大学, 2015.
	ZHAO Y. Research on infrared small target detection algorithms[D]. Hangzhou: Zhejiang University, 2015. (in Chinese)
[4]	万丽丽. 基于背景预测的红外目标检测[D]. 武汉: 华中科技大学, 2015.
	WAN L L. Infrared target detection based on background prediction[D]. WuHan: Huazhong University of Science and Technology, 2015. (in Chinese)
[5]	韩金辉, 董兴浩, 李知静, 等. 基于局部对比度机制的红外弱小目标检测算法[J]. 红外技术, 2021, 43(4): 357-366.
	HAN J H, DONG X H, LI Z J, et al. Infrared small dim target detection based on local contrast mechanism[J]. Infrared Technology, 2021, 43(4): 357-366. (in Chinese)
[6]	危水根, 王程伟, 陈震, 等. 基于视觉注意机制的红外弱小目标检测[J]. 光子学报, 2021, 50(1): 0110001.
	WEI S G, WANG C W, CHEN Z, et al. Infrared dim target detection based on human visual mechanism[J]. Acta Photonic Sinicia, 2021, 50(1): 0110001. (in Chinese)
[7]	朱硕雅, 杨德振, 贾鹏, 等. 时空联合红外小目标检测算法的设计与实现[J]. 红外与激光, 2021, 51(3): 388-392.
	ZHU S Y, YANG D Z, JIA P, et al. Design and implementation of space-time combined infrared small target detection algorithm[J]. Laser & Infrared, 2021, 5(3): 388-392. (in Chinese)
[8]	娄康. 基于深度学习与生物视觉机制的红外弱小目标检测与跟踪方法研究[D]. 镇江: 江苏科技大学, 2020.
	LOU K. Research on infrared weak and small target detection and tracking based on deep learning and biological vision mechanism[D]. Zhenjiang: Jiangsu University of Science and Technology, 2020. (in Chinese)
[9]	黄乐弘. 基于深度学习的空中红外目标检测方法研究[D]. 北京: 中国科学院大学, 2020.
	HUANG L H. Research on infrared target detection in air based on deep learning[D]. Beijiing: University of Chinese Academy of Sciences, 2020. (in Chinese)
[10]	JOSEPH R, ALI F. YOLOv3: an incremental improvement: arXiv:1804.02767[R/OL]. Ithaca,NY, US: Cornell University, 2018.(2018-04-08) [2020-04-01]. https://arxiv.org/abs/1804.02767.
[11]	杨其利, 周炳红, 郑伟. 基于全卷积递归网络的弱小目标检测方法[J]. 光学学报, 2020, 40(13): 43-55.
	YANG Q L, ZHOU B H, ZHENG W. Dim and small target detection based on fully convolutional recursive network[J]. ActaPhotonic Sinicia, 2020, 40(13): 43-55. (in Chinese)
[12]	梁杰, 李磊, 任君. 基于深度学习的红外图像遮挡干扰检测方法[J]. 兵工学报, 2019, 40(7): 1401-1410. doi: 10.3969/j.issn.1000-1093.2019.07.009
	LIANG J, LI L, REN J. Infrared image occlusion interference detection method based on deep learning[J]. Acta Armamentarii, 2019, 40(7): 1401-1410. (in Chinese) doi: 10.3969/j.issn.1000-1093.2019.07.009
[13]	田萱, 王亮, 丁琪. 基于深度学习的图像语义分割方法综述[J]. 软件学报, 2019, 30(2): 440-468.
	TIAN X, WANG L, DING Q. Review of image semantic segmentation based on deep learning[J]. Journal of Software, 2019, 30(2): 440-468. (in Chinese)
[14]	SHELHAMER E, LONG J, DARRELL T. Fully convolutional networks for semantic segmentation[J]. IEEE Transactions on Pattern Analysis & Machine Intelligence, 2014, 39(4): 640-651.
[15]	RONNEBERGER O, FISCHER P, BROX T. U-Net: convolutional networks for biomedical image segmentation[C]// Proceedings of Medical Image Computing and Computer-Assisted Intervention. Cham, Switzerland:Springer, 2015: 234-241.
[16]	GIRSHICK R, DONAHUE J, DARRELL T, et al. Rich feature hierarchies for accurate object detection and semantic segmentation[C]// Proceedings of 2014 IEEE Conference on Computer Vision and Pattern Recognition. Columbus, OH, US:IEEE, 2014: 580-587.
[17]	CHEN L C, PAPANDREOU G, KOKKINOS I, et al. DeepLab: Semantic image segmentation with deep convolutional nets, atrous convolution, and fully connected CRFs[J]. IEEE Transactions on Pattern Analysis & Machine Intelligence, 2018, 40(4): 834-848
[18]	YU F, KOLTUN V. Multi-scale context aggregation by dilated convolutions: arXiv:1511.07122[R/OL]. Ithaca, NY, US: Cornell University, 2016(2016-04-30) [2020-04-01]. https://arxiv.org/abs/1511.07122.
[19]	CHEN L C, PAPANDREOU G, SCHROFF F, et al. Rethinking atrous convolution for semantic image segmentation: arXiv:1706.05587[R/OL]. Ithaca, NY, US: Cornell University, 2017(2017-12-05) [2020-05-01]. https://arxiv.org/abs/1706.05587.
[20]	LIU S, QI X J, SHI J P, et al. Multi-scale patch aggregation (MPA) for simultaneous detection and segmentation[C]// Proceedings of 2016 IEEE Conference on Computer Vision and Pattern Recognition. Las Vegas, NV, US:IEEE, 2016: 3141-3149.
[21]	HE K M, ZHANG X, REN S, et al. Spatial pyramid pooling in deep convolutional networks for visual recognition[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2015, 37(9): 1904-1916. doi: 10.1109/TPAMI.2015.2389824 pmid: 26353135
[22]	CHOLLET F. Xception:deep learning with depthwise separable convolutions[C]// Proceedings of 2017 IEEE International Conference on Computer Vision.Venice, Italy:IEEE, 2017: 764-773.
[23]	WANG P Q, CHEN P F, YUAN Y, et al. Understanding convolution for semantic segmentation[C]// Proceedings of 2018 IEEE Winter Conference on Applications of Computer Vision.Istanbul, Turkey:IEEE, 2018: 1451-1460.
[24]	DAI J F, QI H Z, XIONG Y W, et al. Deformable convolutional networks[C]// Proceedings of 2017 IEEE International Conference on Computer Vision. Venice, Italy:IEEE, 2017: 764-773.
[25]	BADRINARAYANAN V, KENDALL A, CIPOLLA R. Segnet:a deep convolutional encoder-decoder architecture for image segmentation[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2015, 39(12): 2481-2495. doi: 10.1109/TPAMI.34 URL
[26]	NOH H, HONG S, HAN B. Learning deconvolution network for semantic segmentation[C]// Proceedings of 2015 IEEE International Conference on Computer Vision. Santiago, Chile:IEEE, 2015: 1520-1528.
[27]	LIU Z W, LI X X, LUO P, et al. Semantic image segmentation via deep parsing network[C]// Proceedings of 2015 IEEE International Conference on Computer Vision.Santiago, Chile:IEEE, 2015: 1377-1385
[28]	CHEN L C, YANG Y, WANG J, et al. Attention to scale: Scale-aware semantic image segmentation[C]// Proceedings of 2016 IEEE Conference on Computer Vision and Pattern Recognition. Las Vegas, NV, US:IEEE, 2016: 3640-3649.
[29]	ROY A G, NAVAB N, WACHINGER C. Concurrent Spatial and Channel Squeeze & Excitation in Fully Convolutional Networks[C]// Proceedings of the 21st International Conference on Medical Image Computing and Computer Assisted Intervention.Granada, Spain:IEEE, 2018: 421-429.
[30]	YU Q H, XIE L X, WANG Y, et al. Recurrent saliency transformation network: incorporating multi-stage visual cues for small organ segmentation[C]// Proceedings of 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.Salt Lake City,UT, US:IEEE, 2018: 8280-8289.
[31]	LUC P, COUPRIE C, CHINTALA S, et al. Semantic segmentation using adversarial networks:arXiv:1611.08408[R/OL]. Ithaca, NY, US: Cornell University, 2016 (2016-11-25)[2020-06-01]. https://arxiv.org/abs/1611.08408.
[32]	ZHAO H S, SHI J P, QI X J. Pyramid scene parsing network[C]// Proceedings of 2017 IEEE Conference on Computer Vision and Pattern Recognition. Honolulu, HI, US:IEEE, 2017: 6230-6239.
[33]	LIN TY, GOYAL P, GIRSHICK R, et al. Focal loss for dense object detection[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2020, 42(2): 318-327. doi: 10.1109/TPAMI.34 URL
[34]	RIVEST J F, FORTIN R. Detection of dim targets in digital infrared imagery by morphological image processing[J]. Optical Engineering, 1996, 35(7): 1886-1893. doi: 10.1117/1.600620 URL
[35]	DESHPANDE S D, ER M H, VENKATESWARLU R, et al. Max-mean and max-median filters for detection of small targets[C]// Proceedings of Signal and Data Processing of Small Targets 1999.Denver,CO,US:SPIE, 1999, 3809: 74-83.
[36]	HAN J, MORADI S, FARAMARZI I, et al. Infrared small target detection based on the weighted strengthened local contrast measure[J]. IEEE Geoscience and Remote Sensing Letters, 2021, 18(9): 1670-1674. doi: 10.1109/LGRS.2020.3004978 URL
[37]	HAN J, MORADI S, FARAMARZI I, et al. A local contrast method for infrared small-target detection utilizing a tri-layer window[J]. IEEE Geoscience and Remote Sensing Letters, 2019, 17(10): 1822-1826. doi: 10.1109/LGRS.8859 URL
[38]	GAO C Q, MENG D Y, YANG Y, et al. Infrared patch-image model for small target detection in a single image[J]. IEEE Transactions on Image Processing, 2013, 22(12): 4996-5009. pmid: 24043387
[39]	ZHANG L D, PENG L B, ZHANG T F, et al. Infrared small target detection via non-convex rank approximation minimization joint l_2,1 norm[J]. Remote Sensing, 2018, 10(11): 1821. doi: 10.3390/rs10111821 URL
[40]	DAI Y M, WU Y Q. Reweighted infrared patch-tensor model with both nonlocal and local priors for single-frame small target detection[J]. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 2017, 10(8): 3752-3767. doi: 10.1109/JSTARS.4609443 URL
[41]	ZHANG L D, PENG Z M. Infrared small target detection based on partial sum of the tensor nuclear norm[J]. Remote Sensing, 2019, 11(4): 382. doi: 10.3390/rs11040382 URL
[42]	SUN Y, YANG J G, AN W. Infrared dim and small target detection via multiple subspace learning and spatial-temporal patch-tensor model[J]. IEEE Transactions on Geoscience and Remote Sensing, 2021, 59(5): 3737-3752. doi: 10.1109/TGRS.2020.3022069 URL
[43]	DAI Y M, WU Y Q, ZHOU Y, et al. Asymmetric contextual modulation for infrared small target detection[C]// Proceedings of 2021 IEEE Winter Conference on Applications of Computer Vision. Waikoloa, HI, US:IEEE, 2021: 950-959.
[44]	WANG H, ZHOU L P, WANG L. Miss detection vs. false alarm: adversarial learning for small object segmentation in infrared images[C]// Proceedings of 2019 IEEE/CVF International Conference on Computer Vision.Seoul, Korea: IEEE, 2019: 8509-8518.

数据集	算法	Precision↑	Recall↑	F1-measure↑
	FCN-RSTN^[30]	0.12	0.75	0.21
	cGAN^[31]	0.13	0.60	0.21
All Seqs	U-net^[15]	0.01	0.01	0.01
	DeepLabv3^[19]	0.17	0.59	0.27
	DualNet	0.36	0.44	0.40
	FCN-RSTN^[30]	0.54	0.39	0.45
	cGAN^[31]	0.28	0.71	0.40
Single	U-net^[15]	0.01	0.02	0.01
	DeepLabv3^[19]	0.22	0.57	0.31
	DualNet	0.52	0.54	0.53

数据集	算法	Precision↑	Recall↑	F1-measure↑
	FCN-RSTN^[30]	0.12	0.75	0.21
	cGAN^[31]	0.13	0.60	0.21
All Seqs	U-net^[15]	0.01	0.01	0.01
	DeepLabv3^[19]	0.17	0.59	0.27
	DualNet	0.36	0.44	0.40
	FCN-RSTN^[30]	0.54	0.39	0.45
	cGAN^[31]	0.28	0.71	0.40
Single	U-net^[15]	0.01	0.02	0.01
	DeepLabv3^[19]	0.22	0.57	0.31
	DualNet	0.52	0.54	0.53

模型	数据集	Precision↑	Recall↑	F1-measure↑
Dilated-Conv	All Seqs	0.32	0.36	0.34
	Single	0.45	0.44	0.45
ASPP	All Seqs	0.34	0.42	0.38
	Single	0.48	0.46	0.47
ASPP+PPM	All Seqs	0.32	0.38	0.35
	Single	0.46	0.45	0.45
ASPP+SPP	All Seqs	0.34	0.48	0.40
	Single	0.49	0.53	0.51
ASPP+SPP+	All Seqs	0.46	0.44	0.40
scSE	Single	0.52	0.53	0.52
ASPP+SPP+	All Seqs	0.36	0.44	0.40
scSE+DCNv2	Single	0.52	0.54	0.53

模型	数据集	Precision↑	Recall↑	F1-measure↑
Dilated-Conv	All Seqs	0.32	0.36	0.34
	Single	0.45	0.44	0.45
ASPP	All Seqs	0.34	0.42	0.38
	Single	0.48	0.46	0.47
ASPP+PPM	All Seqs	0.32	0.38	0.35
	Single	0.46	0.45	0.45
ASPP+SPP	All Seqs	0.34	0.48	0.40
	Single	0.49	0.53	0.51
ASPP+SPP+	All Seqs	0.46	0.44	0.40
scSE	Single	0.52	0.53	0.52
ASPP+SPP+	All Seqs	0.36	0.44	0.40
scSE+DCNv2	Single	0.52	0.54	0.53

数据集	α	β	F1-measure↑
	0.05	0.95	0.38
	0.1	0.9	0.40
	0.2	0.8	0.36
	0.3	0.7	0.37
All Seqs	0.4	0.6	0.34
	0.5	0.5	0.33
	0.6	0.4	0.31
	0.7	0.3	0.31
	0.8	0.2	0.30
	0.05	0.95	0.51
	0.1	0.9	0.53
	0.2	0.8	0.51
	0.3	0.7	0.51
Single	0.4	0.6	0.50
	0.5	0.5	0.50
	0.6	0.4	0.49
	0.7	0.3	0.49
	0.8	0.2	0.47