基于弱监督的红外小目标分割方法

doi:10.12382/bgxb.2024.0573

摘要/Abstract

摘要：

针对目前在红外小目标分割领域数据集稀缺问题,提出根据红外小目标特点设计图像分割弱监督方法来解决该问题。所提方法利用红外小目标边界框掩码作为监督,结合框外损失、框内损失和基于双模型的指数滑动平均迭代算法,为红外小目标分割提供一种更为经济高效的训练方式,在不需要大规模标注的情况下,让模型从有限的弱标签中获取知识,从而在红外小目标分割任务中表现出色。实验结果表明:所提方法成功克服了人工标注方式的繁琐和成本高的问题,为红外小目标分割任务提供了更高效和可持续的训练方式。

关键词: 图像分割, 红外小目标, 密集连接, 弱监督, 损失函数

Abstract:

To address the challenge of limited training data in infrared small target segmentation, an image segmentation weak supervision method (BoxInf) is proposed. The method leverages bounding box annotations, a readily available weak supervision source, and incorporates out-of-box loss, in-box loss, and an exponential moving average (EMA) iterative algorithm within a dual-model framework. By effectively utilizing these elements, BoxInf offers a cost-effective training paradigm. The model learns from limited weak labels, circumventing the need for extensive and expensive pixel-wise annotations. Consequently, BoxInf demonstrates robustness in infrared small target segmentation tasks. The experimental validation confirms the effectiveness of the proposed method in mitigating the laborious and costly burden of manual annotation, paving the way for a more efficient and sustainable training strategy in this domain.

Key words: image segmentation, infrared small target, dense connections, weak supervision, loss function

中图分类号:

TP391

王烨茹, 陈殿坤, 秦飞巍, 徐华杰, 刘述, 赵龙. 基于弱监督的红外小目标分割方法[J]. 兵工学报, 2024, 45(S1): 120-134.

WANG Yeru, CHEN Diankun, QIN Feiwei, XU Huajie, LIU Shu, ZHAO Long. Weak Supervision-based Infrared Small Target Segmentation Method[J]. Acta Armamentarii, 2024, 45(S1): 120-134.

图/表 20

图1 图像分割弱监督领域的四种标注方式

Fig.1 Four annotation methods for weakly supervised image segmentation

图2 红外小目标弱监督算法BoxInf总体框架注:图中PS-Loss表示红外小目标弱监督损失函数;SoftIoU-Loss表示伪掩码使用的一般图像分割损失函数

Fig.2 Overall framework of the infrared small target weak supervision algorithm BoxInf

图3 框外损失和框内损失作用域

Fig.3 Domain of Project-Loss and Shape-Loss

图4 真实掩码和边界框掩码的映射关系

Fig.4 Mapping relationship between ground truth mask and bounding box mask

图5 框外损失函数Project-Loss流程图

Fig.5 Flow chart of Project-Loss function

图6 红外小目标边缘形状生成原理

Fig.6 Principle of edge shape generation of small infrared targets

图7 框内损失边缘点和内部点

Fig.7 Edge points and internal points of Shape-Loss

图8 框内损失Shape-Loss流程图

Fig.8 Flow chart of Shape-Loss

图9 红外小目标数据集上不同阈值的分割数据的假阳率

Fig.9 False positive rate of segmented data with different thresholds on infrared small target dataset

图10 基于EMA算法的红外小目标弱监督分割迭代算法

Fig.10 EMA-based iteration algorithm for weakly supervised segmentation of small infrared targets

图11 实验所用数据集展示

Fig.11 The datasets used in the experiment

图12 红外小目标数据集转换成弱监督所需数据集

Fig.12 Conversion of infrared small target datasets into datasets required for weak supervision

表1 与其他图像分割弱监督方法的实验对比结果

Table 1 Comparison of experimental results of the proposed method and other weakly supervised image segmentation methods

数据集	方法	IoU	nIoU	准确率	召回率	F₁
	全监督	0.8836	0.8931	0.9453	0.9313	0.9382
	边界框掩码	0.5677	0.6419	0.6223	0.8661	0.8242
	GrabCut	0.6547	0.6781	0.6871	0.9329	0.7913
NUDT-SIRST	BoxSup	0.6676	0.6848	0.7007	0.934	0.8007
	BoxInst	0.7762	0.7902	0.8345	0.9175	0.8740
	BoxTeacher	0.7775	0.7821	0.8701	0.8796	0.8749
	BoxInf(本文)	0.8044	0.8264	0.8867	0.8965	0.8916
	全监督	0.7733	0.7528	0.8512	0.8941	0.8721
	边界框掩码	0.6873	0.6967	0.7483	0.8940	0.8147
	GrabCut	0.6928	0.6917	0.7565	0.8916	0.8185
NUAA-SIRST	BoxSup	0.7003	0.6965	0.8398	0.7901	0.8141
	BoxInst	0.7311	0.7312	0.8663	0.8240	0.8446
	BoxTeacher	0.7361	0.7287	0.8566	0.8396	0.8480
	BoxInf(本文)	0.7460	0.7349	0.8671	0.8423	0.8545
	全监督	0.7279	0.6856	0.8355	0.8488	0.8421
	边界框掩码	0.5734	0.5306	0.6117	0.9008	0.7286
	GrabCut	0.5916	0.5731	0.6428	0.8805	0.7431
IRSTD-1k	BoxSup	0.6053	0.6370	0.7644	0.7237	0.7435
	BoxInst	0.6515	0.6552	0.7655	0.8134	0.7655
	BoxTeacher	0.6553	0.6748	0.7947	0.7882	0.7914
	BoxInf(本文)	0.6796	0.6716	0.8013	0.8166	0.8089

图13 弱监督算法与其他典型算法对比

Fig.13 Comparison of weak supervision algorithm with other typical algorithms

表2 数据集NUDT-SIRST上不同损失函数组合的消融实验

Table 2 Ablation experiments with different loss function combinations

方法	损失函数	IoU	nIoU	准确率	召回率	F₁
边界框掩码	DiceIoU-Loss	0.5580	0.6585	0.5810	0.9337	0.7163
边界框掩码	SoftIoU-Loss	0.5677	0.6419	0.6223	0.8661	0.8242
BoxInf	DiceIoU-Loss	0.8044	0.8264	0.8867	0.8965	0.8916
BoxInf	SoftIoU-Loss	0.7585	0.7728	0.8854	0.8410	0.8626

表3 数据集NUDT-SIRST上BoxInf模型模块组成的消融实验

Table 3 Ablation experiment of BoxInf model modules

Project-Loss	Shape-Loss	EMA	IoU	nIoU	准确率	召回率	F₁
边界框掩码			0.5677	0.6419	0.6223	0.8661	0.8242
√			0.7702	0.7886	0.8620	0.8786	0.8702
√	√		0.7919	0.8142	0.8957	0.8723	0.8838
√	√	√	0.8044	0.8264	0.8867	0.8965	0.8916

表4 不同预处理方式的最终训练效果

Table 4 Training results of different preprocessing methods

数据集	方法	IoU	nIoU	准确率	召回率	F₁
NUDT-SIRST	Box	0.5677	0.6419	0.6223	0.8661	0.7242
	BoxGrabCut	0.6547	0.6781	0.6871	0.9329	0.7913
NUAA-SIRST	Box	0.6873	0.6967	0.7483	0.8940	0.8147
	BoxGrabCut	0.6928	0.6917	0.7565	0.8916	0.8185
NUAA-SIRST	Box	0.5734	0.5306	0.6117	0.9008	0.7286
	BoxGrabCut	0.5916	0.5731	0.6428	0.8805	0.7431

表5 数据集NUDT-SIRST上EMA迭代算法中不同λ对方法的最终效果

Table 5 The effect of different EMA iteration algorithms on the poroposed method

λ	IoU	nIoU	准确率	召回率	F₁
1/N	0.8044	0.8264	0.8867	0.8965	0.8916
2/N	0.8024	0.8262	0.8840	0.9048	0.8943
3/N	0.8009	0.8206	0.8766	0.8990	0.8876

表6 数据集NUDT-SIRST上损失函数组合的消融实验

Table 6 Ablation experiments on loss function combinations

偏移量	IoU	nIoU	准确率	召回率	F₁
0	0.8044	0.8264	0.8867	0.8965	0.8916
+1	0.3830	0.3845	0.3900	0.9554	0.5539
+2	0.2563	0.2382	0.2586	0.9666	0.4080
+3	0.1893	0.1765	0.1916	0.9403	0.3183

表7 数据集NUDT-SIRST上PS-Loss损失函数加权值的消融实验

Table 7 Ablation experiment of weighted value of PS-Loss

β	IoU	nIoU	准确率	召回率	F₁
0.1	0.8044	0.8264	0.8867	0.8965	0.8916
0.2	0.8033	0.8245	0.8767	0.8954	0.8859
0.3	0.7968	0.8182	0.8586	0.8866	0.8723

参考文献 41

[1]	DESHPANDE S D, ER M H, VENKATESWARLU R, et al. Max-mean and max-median filters for detection of small targets[C]//Proceedings of the SPIE 3809, Signal and Data Processing of Small Targets 1999. Denver, CO, US:SPIE, 1999:74-83.
[2]	ZENG M, LI J X, PENG Z. The design of top-hat morphological filter and application to infrared target detection[J]. Infrared Physics & Technology, 2006, 48(1): 67-76.
[3]	BAI K, WANG Y H, SONG Q, Patch similarity based edge-preserving background estimation for single frame infrared small target detection[C]// Proceedings of the 2016 IEEE International Conference on Image Processing, Phoenix, Arizona, US: IEEE, 2016: 181-185.
[4]	CHEN C L P, LI H, WEI Y, et al. A local contrast method for small infrared target detection[J]. IEEE Transactions on Geoscience and Remote Sensing, 2013, 52(1): 574-581.
[5]	HE D, SUN X P, LIU M L, et al. Small infrared target detection based on weighted local difference measure[J]. IEEE Transactions on Geoscience and Remote Sensing, 2016, 54(7): 4204-4214.
[6]	XIA C Q, LI X R, ZHAO L, et al. Infrared small target detection based on multiscale local contrast measure using local energy factor[J]. IEEE Geoscience and Remote Sensing Letters, 2019, 17(1): 157-161.
[7]	GAO C Q, MENG D Y, YANG Y, et al. Infrared patch-image model for small target detection in a single image[J]. IEEE Transactions on Image Processing, 2013, 22(12): 4996-5009. pmid: 24043387
[8]	DAI Y M, WU Y Q, SONG Y. Infrared small target and background separation via column-wise weighted robust principal component analysis[J]. Infrared Physics & Technology, 2016, 77: 421-430.
[9]	DAI Y M, WU Y Q. Reweighted infrared patch-tensor model with both nonlocal and local priors for single-frame small target detection[J]. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 2017, 10(8): 3752-3767.
[10]	王宇翔, 韩振铎, 王宏敏. 基于多向差异度的红外弱小目标检测算法[J]. 红外技术, 2012, 34(6):351-355.
	WANG Y X, HAN Z Y, WANG H M. Detection algorithm for dim infrared target based on multi-difference factor[J]. Infrared Technology, 2012, 34(6):351-355. (in Chinese)
[11]	周姣, 辛云宏. 基于显著性与尺度空间的红外弱小目标检测[J]. 激光与红外, 2015, 45(4):452-456.
	ZHOU J, XIN Y H. Infrared dim small target detection based on saliency and scale-space[J]. Infrared Technology, 2015, 45(4):452-456. (in Chinese)
[12]	NIE J Y, QU S C, WEI Y T, et al. An infrared small target detection method based on multiscale local homogeneity measure[J]. Infrared Physics & Technology, 2018, 90: 186-194.
[13]	张天成. 复杂场景下的红外图像小目标检测方法研究[D]. 南京: 南京航空航天大学, 2023.
	ZHANG T C. Research on detection method for infrared small targets in complex scenarios[D]. Nanjing: Nanjing University of Aeronautics and Astronautics, 2023. (in Chinese)
[14]	夏超群. 基于局部和全局特征表示的红外小目标检测算法研究[D]. 杭州: 浙江大学, 2021.
	XIA C Q. Research on infrared small target detection algorithm based on local and global feature representation[D]. Hangzhou: Zhejiang University, 2021. (in Chinese)
[15]	谢江荣. 基于深度学习的空中红外目标检测关键技术研究[D]. 上海: 中国科学院大学(中国科学院上海技术物理研究所), 2020.
	XIE J R. Research on key techniques of airborne infrared target detection based on deep learning[D]. Shanghai: University of Chinese Academy of Sciences (Shanghai Institute of Technical Physics, Chinese Academy of Sciences), 2020. (in Chinese)
[16]	WANG H, ZHOU L P, WANG L. Miss detection vs. False alarm: adversarial learning for small object segmentation in infrared images[C]// Proceedings of the IEEE/CVF International Conference on Computer Vision. Seoul, Korea (South): IEEE, 2019: 8509-8518.
[17]	DAI Y M, WU Y Q, ZHOU F, et al. Asymmetric contextual modulation for infrared small target detection[C]// Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision. Waikoloa, HI, US: IEEE, 2021: 950-959.
[18]	DAI Y M, WU Y Q, ZHOU F, et al. Attentional local contrast networks for infrared small target detection[J]. IEEE Transactions on Geoscience and Remote Sensing, 2021, 59(11): 9813-9824.
[19]	ZHANG M J, ZHANG R, YANG Y X, et al. ISNet: Shape matters for infrared small target detection[C]// Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. New Orleans, Louisiana, US: IEEE, 2022: 877-886.
[20]	LI B Y, XIAO C, WANG L G, et al. Dense nested attention network for infrared small target detection[J]. IEEE Transactions on Image Processing, 2022, 32: 1745-1758.
[21]	WU X, HONG D F, CHANUSSOT J. UIU-Net: U-Net in U-Net for infrared small object detection[J]. IEEE Transactions on Image Processing, 2022, 32: 364-376.
[22]	WU F Y, ZHANG T F, LI L, et al. RPCANet: deep unfolding RPCA based infrared small target detection[C]// Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision. Waikoloa, Hawaii, US: IEEE, 2024: 4809-4818.
[23]	仉新, 张旭阳, 毛宇新, 等. 基于弱监督学习的图像语义分割算法综述[J]. 长江信息通信, 2023, 36(5):105-108.
	ZHANG X, ZHANG X Y, MAO Y X, et al. Research on medical image segmentation method based on weakly supervised learning[J]. Changjiang Information & Communications, 2023, 36(5):105-108. (in Chinese)
[24]	程舸帆, 何良华. 基于弱监督学习的医学图像分割方法研究[J]. 电脑知识与技术, 2022, 18(3):7-9.
	CHENG G F, HE L H. A review of image semantic segmentation algorithms based on weakly supervised learning[J]. Computer Knowledge and Technology, 2022, 18(3):7-9. (in Chinese)
[25]	丁一鹏, 赵璐. 遥感图像语义分割中的弱监督域自适应算法[J]. 计算机工程与应用, 2022, 58(22):195-202. doi: 10.3778/j.issn.1002-8331.2104-0259
	DING Y P, ZHAO L. Weakly-supervised domain adaptive algorithm for remote sensing image semantic segmentation[J]. Computer Engineering and Applications, 2022, 58(22):195-202. (in Chinese) doi: 10.3778/j.issn.1002-8331.2104-0259
[26]	王强, 黄小明, 佟强, 等. 基于边界框标注的弱监督显著性目标检测算法[J]. 计算机应用, 2023, 43(6):1910-1918. doi: 10.11772/j.issn.1001-9081.2022050706
	WANG Q, HUANG X M, DONG Q,etal. Weakly supervised salient object detection algorithm based on bounding box annotation[J]. Journal of Computer Applications, 2023, 43(6):1910-1918. (in Chinese) doi: 10.11772/j.issn.1001-9081.2022050706
[27]	PONT-TUSET J, ARBELAEZ P, BARRON J T, et al. Multiscale combinatorial grouping for image segmentation and object proposal generation[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2016, 39(1): 128-140.
[28]	ROTHER C, KOLMOGOROV V, BLAKE A. “GrabCut” interactive foreground extraction using iterated graph cuts[J]. ACM Transactions on Graphics, 2004, 23(3): 309-314.
[29]	DAI J F, HE K M, SUN J. Boxsup: exploiting bounding boxes to supervise convolutional networks for semantic segmentation[C]// Proceedings of the IEEE International Conference on Computer Vision. Santiago, Chile: IEEE, 2015: 1635-1643.
[30]	龙建武, 刘东, 宋鑫磊. 前背景信息一致的边界框弱监督息肉分割网络[J]. 重庆理工大学学报(自然科学), 2023, 37(12):210-221.
	LONG J W, LIU D, SONG X L. Weak supervision polyp segmentation network with consistent front background information with boundary boxes[J]. Journal of Chongqing University of Technology:Natural Science, 2023, 37(12):210-221.
[31]	TIAN Z, SHEN C H, WANG X L, et al. BoxInst: high-performance instance segmentation with box annotations[C]// Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. Nashville, TN, US: IEEE, 2021: 5443-5452.
[32]	CHENG T H, WANG X G, CHEN S Y, et al. Boxteacher: exploring high-quality pseudo labels for weakly supervised instance segmentation[C]// Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. Vancouver. BC, Canada: IEEE, 2023: 3145-3154.
[33]	穆柯舟. 基于弱监督学习的显著性物体检测方法研究[D]. 哈尔滨: 哈尔滨工业大学, 2022.
	MU K Z. Research on the significant object detection method based on weak supervised learning[D]. Harbin:Harbin Institute of Technology, 2022. (in Chinese)
[34]	LIN D, DAI J F, JIA J Y, et al. ScribbleSup: scribble-supervised convolutional networks for semantic segmentation[C]// Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Las Vegas, NV, US: IEEE, 2016: 3159-3167.
[35]	刘昕竺. 基于弱监督学习的人体解析算法研究[D]. 北京: 北京交通大学, 2023.
[36]	BEARMAN A, RUSSAKOVSKY O, FERRARI V, et al. What’s the point: semantic segmentation with point supervision[C]// Proceedings of the 14th European Conference on Computer Vision.Amsterdam. The Netherlands: Springer International Publishing, 2016: 549-565.
[37]	胡孔涛. 基于弱监督学习的癌症病理图像分割[D]. 南京: 东南大学, 2020.
	HU K T. Cancer pathology image segmentation based on weak supervised learning[D]. Nanjing: Southeast University, 2020. (in Chinese)
[38]	刘永胜. 基于深度神经网络的弱监督学习方法在图像领域的研究[D]. 成都: 电子科技大学, 2020.
	LIU Y S. Research on weakly supervised learning method based on deep neural network in image field[D]. Chendu: University of Electronic Science and Technology of China, 2020. (in Chinese)
[39]	KIRILLOV A, MINTUN E, RAVI N, et al. Segment anything[C]// Proceedings of the IEEE/CVF International Conference on Computer Vision. Paris, France: IEEE, 2023: 4015-4026.
[40]	HE K M, CHEN X L, XIE S N, et al. Masked autoencoders are scalable vision learners[C]// Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. New Orleans, LA, US: IEEE, 2022: 16000-16009.
[41]	姜昕. 基于图像级标注的弱监督海雾检测研究[D]. 北京: 北京邮电大学, 2024.
	JIANG X. Research on weakly supervised sea fog detection based on image-level labeling[D]. Beijing: Beijing University of Posts and Telecommunications, 2024. (in Chinese)