Weak Supervision-based Infrared Small Target Segmentation Method

doi:10.12382/bgxb.2024.0573

Abstract

Abstract:

To address the challenge of limited training data in infrared small target segmentation, an image segmentation weak supervision method (BoxInf) is proposed. The method leverages bounding box annotations, a readily available weak supervision source, and incorporates out-of-box loss, in-box loss, and an exponential moving average (EMA) iterative algorithm within a dual-model framework. By effectively utilizing these elements, BoxInf offers a cost-effective training paradigm. The model learns from limited weak labels, circumventing the need for extensive and expensive pixel-wise annotations. Consequently, BoxInf demonstrates robustness in infrared small target segmentation tasks. The experimental validation confirms the effectiveness of the proposed method in mitigating the laborious and costly burden of manual annotation, paving the way for a more efficient and sustainable training strategy in this domain.

Key words: image segmentation, infrared small target, dense connections, weak supervision, loss function

CLC Number:

TP391

WANG Yeru, CHEN Diankun, QIN Feiwei, XU Huajie, LIU Shu, ZHAO Long. Weak Supervision-based Infrared Small Target Segmentation Method[J]. Acta Armamentarii, 2024, 45(S1): 120-134.

Figures/Tables 20

References 41

[1]	DESHPANDE S D, ER M H, VENKATESWARLU R, et al. Max-mean and max-median filters for detection of small targets[C]//Proceedings of the SPIE 3809, Signal and Data Processing of Small Targets 1999. Denver, CO, US:SPIE, 1999:74-83.
[2]	ZENG M, LI J X, PENG Z. The design of top-hat morphological filter and application to infrared target detection[J]. Infrared Physics & Technology, 2006, 48(1): 67-76.
[3]	BAI K, WANG Y H, SONG Q, Patch similarity based edge-preserving background estimation for single frame infrared small target detection[C]// Proceedings of the 2016 IEEE International Conference on Image Processing, Phoenix, Arizona, US: IEEE, 2016: 181-185.
[4]	CHEN C L P, LI H, WEI Y, et al. A local contrast method for small infrared target detection[J]. IEEE Transactions on Geoscience and Remote Sensing, 2013, 52(1): 574-581.
[5]	HE D, SUN X P, LIU M L, et al. Small infrared target detection based on weighted local difference measure[J]. IEEE Transactions on Geoscience and Remote Sensing, 2016, 54(7): 4204-4214.
[6]	XIA C Q, LI X R, ZHAO L, et al. Infrared small target detection based on multiscale local contrast measure using local energy factor[J]. IEEE Geoscience and Remote Sensing Letters, 2019, 17(1): 157-161.
[7]	GAO C Q, MENG D Y, YANG Y, et al. Infrared patch-image model for small target detection in a single image[J]. IEEE Transactions on Image Processing, 2013, 22(12): 4996-5009. pmid: 24043387
[8]	DAI Y M, WU Y Q, SONG Y. Infrared small target and background separation via column-wise weighted robust principal component analysis[J]. Infrared Physics & Technology, 2016, 77: 421-430.
[9]	DAI Y M, WU Y Q. Reweighted infrared patch-tensor model with both nonlocal and local priors for single-frame small target detection[J]. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 2017, 10(8): 3752-3767.
[10]	王宇翔, 韩振铎, 王宏敏. 基于多向差异度的红外弱小目标检测算法[J]. 红外技术, 2012, 34(6):351-355.
	WANG Y X, HAN Z Y, WANG H M. Detection algorithm for dim infrared target based on multi-difference factor[J]. Infrared Technology, 2012, 34(6):351-355. (in Chinese)
[11]	周姣, 辛云宏. 基于显著性与尺度空间的红外弱小目标检测[J]. 激光与红外, 2015, 45(4):452-456.
	ZHOU J, XIN Y H. Infrared dim small target detection based on saliency and scale-space[J]. Infrared Technology, 2015, 45(4):452-456. (in Chinese)
[12]	NIE J Y, QU S C, WEI Y T, et al. An infrared small target detection method based on multiscale local homogeneity measure[J]. Infrared Physics & Technology, 2018, 90: 186-194.
[13]	张天成. 复杂场景下的红外图像小目标检测方法研究[D]. 南京: 南京航空航天大学, 2023.
	ZHANG T C. Research on detection method for infrared small targets in complex scenarios[D]. Nanjing: Nanjing University of Aeronautics and Astronautics, 2023. (in Chinese)
[14]	夏超群. 基于局部和全局特征表示的红外小目标检测算法研究[D]. 杭州: 浙江大学, 2021.
	XIA C Q. Research on infrared small target detection algorithm based on local and global feature representation[D]. Hangzhou: Zhejiang University, 2021. (in Chinese)
[15]	谢江荣. 基于深度学习的空中红外目标检测关键技术研究[D]. 上海: 中国科学院大学(中国科学院上海技术物理研究所), 2020.
	XIE J R. Research on key techniques of airborne infrared target detection based on deep learning[D]. Shanghai: University of Chinese Academy of Sciences (Shanghai Institute of Technical Physics, Chinese Academy of Sciences), 2020. (in Chinese)
[16]	WANG H, ZHOU L P, WANG L. Miss detection vs. False alarm: adversarial learning for small object segmentation in infrared images[C]// Proceedings of the IEEE/CVF International Conference on Computer Vision. Seoul, Korea (South): IEEE, 2019: 8509-8518.
[17]	DAI Y M, WU Y Q, ZHOU F, et al. Asymmetric contextual modulation for infrared small target detection[C]// Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision. Waikoloa, HI, US: IEEE, 2021: 950-959.
[18]	DAI Y M, WU Y Q, ZHOU F, et al. Attentional local contrast networks for infrared small target detection[J]. IEEE Transactions on Geoscience and Remote Sensing, 2021, 59(11): 9813-9824.
[19]	ZHANG M J, ZHANG R, YANG Y X, et al. ISNet: Shape matters for infrared small target detection[C]// Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. New Orleans, Louisiana, US: IEEE, 2022: 877-886.
[20]	LI B Y, XIAO C, WANG L G, et al. Dense nested attention network for infrared small target detection[J]. IEEE Transactions on Image Processing, 2022, 32: 1745-1758.
[21]	WU X, HONG D F, CHANUSSOT J. UIU-Net: U-Net in U-Net for infrared small object detection[J]. IEEE Transactions on Image Processing, 2022, 32: 364-376.
[22]	WU F Y, ZHANG T F, LI L, et al. RPCANet: deep unfolding RPCA based infrared small target detection[C]// Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision. Waikoloa, Hawaii, US: IEEE, 2024: 4809-4818.
[23]	仉新, 张旭阳, 毛宇新, 等. 基于弱监督学习的图像语义分割算法综述[J]. 长江信息通信, 2023, 36(5):105-108.
	ZHANG X, ZHANG X Y, MAO Y X, et al. Research on medical image segmentation method based on weakly supervised learning[J]. Changjiang Information & Communications, 2023, 36(5):105-108. (in Chinese)
[24]	程舸帆, 何良华. 基于弱监督学习的医学图像分割方法研究[J]. 电脑知识与技术, 2022, 18(3):7-9.
	CHENG G F, HE L H. A review of image semantic segmentation algorithms based on weakly supervised learning[J]. Computer Knowledge and Technology, 2022, 18(3):7-9. (in Chinese)
[25]	丁一鹏, 赵璐. 遥感图像语义分割中的弱监督域自适应算法[J]. 计算机工程与应用, 2022, 58(22):195-202. doi: 10.3778/j.issn.1002-8331.2104-0259
	DING Y P, ZHAO L. Weakly-supervised domain adaptive algorithm for remote sensing image semantic segmentation[J]. Computer Engineering and Applications, 2022, 58(22):195-202. (in Chinese) doi: 10.3778/j.issn.1002-8331.2104-0259
[26]	王强, 黄小明, 佟强, 等. 基于边界框标注的弱监督显著性目标检测算法[J]. 计算机应用, 2023, 43(6):1910-1918. doi: 10.11772/j.issn.1001-9081.2022050706
	WANG Q, HUANG X M, DONG Q,etal. Weakly supervised salient object detection algorithm based on bounding box annotation[J]. Journal of Computer Applications, 2023, 43(6):1910-1918. (in Chinese) doi: 10.11772/j.issn.1001-9081.2022050706
[27]	PONT-TUSET J, ARBELAEZ P, BARRON J T, et al. Multiscale combinatorial grouping for image segmentation and object proposal generation[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2016, 39(1): 128-140.
[28]	ROTHER C, KOLMOGOROV V, BLAKE A. “GrabCut” interactive foreground extraction using iterated graph cuts[J]. ACM Transactions on Graphics, 2004, 23(3): 309-314.
[29]	DAI J F, HE K M, SUN J. Boxsup: exploiting bounding boxes to supervise convolutional networks for semantic segmentation[C]// Proceedings of the IEEE International Conference on Computer Vision. Santiago, Chile: IEEE, 2015: 1635-1643.
[30]	龙建武, 刘东, 宋鑫磊. 前背景信息一致的边界框弱监督息肉分割网络[J]. 重庆理工大学学报(自然科学), 2023, 37(12):210-221.
	LONG J W, LIU D, SONG X L. Weak supervision polyp segmentation network with consistent front background information with boundary boxes[J]. Journal of Chongqing University of Technology:Natural Science, 2023, 37(12):210-221.
[31]	TIAN Z, SHEN C H, WANG X L, et al. BoxInst: high-performance instance segmentation with box annotations[C]// Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. Nashville, TN, US: IEEE, 2021: 5443-5452.
[32]	CHENG T H, WANG X G, CHEN S Y, et al. Boxteacher: exploring high-quality pseudo labels for weakly supervised instance segmentation[C]// Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. Vancouver. BC, Canada: IEEE, 2023: 3145-3154.
[33]	穆柯舟. 基于弱监督学习的显著性物体检测方法研究[D]. 哈尔滨: 哈尔滨工业大学, 2022.
	MU K Z. Research on the significant object detection method based on weak supervised learning[D]. Harbin:Harbin Institute of Technology, 2022. (in Chinese)
[34]	LIN D, DAI J F, JIA J Y, et al. ScribbleSup: scribble-supervised convolutional networks for semantic segmentation[C]// Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Las Vegas, NV, US: IEEE, 2016: 3159-3167.
[35]	刘昕竺. 基于弱监督学习的人体解析算法研究[D]. 北京: 北京交通大学, 2023.
[36]	BEARMAN A, RUSSAKOVSKY O, FERRARI V, et al. What’s the point: semantic segmentation with point supervision[C]// Proceedings of the 14th European Conference on Computer Vision.Amsterdam. The Netherlands: Springer International Publishing, 2016: 549-565.
[37]	胡孔涛. 基于弱监督学习的癌症病理图像分割[D]. 南京: 东南大学, 2020.
	HU K T. Cancer pathology image segmentation based on weak supervised learning[D]. Nanjing: Southeast University, 2020. (in Chinese)
[38]	刘永胜. 基于深度神经网络的弱监督学习方法在图像领域的研究[D]. 成都: 电子科技大学, 2020.
	LIU Y S. Research on weakly supervised learning method based on deep neural network in image field[D]. Chendu: University of Electronic Science and Technology of China, 2020. (in Chinese)
[39]	KIRILLOV A, MINTUN E, RAVI N, et al. Segment anything[C]// Proceedings of the IEEE/CVF International Conference on Computer Vision. Paris, France: IEEE, 2023: 4015-4026.
[40]	HE K M, CHEN X L, XIE S N, et al. Masked autoencoders are scalable vision learners[C]// Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. New Orleans, LA, US: IEEE, 2022: 16000-16009.
[41]	姜昕. 基于图像级标注的弱监督海雾检测研究[D]. 北京: 北京邮电大学, 2024.
	JIANG X. Research on weakly supervised sea fog detection based on image-level labeling[D]. Beijing: Beijing University of Posts and Telecommunications, 2024. (in Chinese)

数据集	方法	IoU	nIoU	准确率	召回率	F₁
	全监督	0.8836	0.8931	0.9453	0.9313	0.9382
	边界框掩码	0.5677	0.6419	0.6223	0.8661	0.8242
	GrabCut	0.6547	0.6781	0.6871	0.9329	0.7913
NUDT-SIRST	BoxSup	0.6676	0.6848	0.7007	0.934	0.8007
	BoxInst	0.7762	0.7902	0.8345	0.9175	0.8740
	BoxTeacher	0.7775	0.7821	0.8701	0.8796	0.8749
	BoxInf(本文)	0.8044	0.8264	0.8867	0.8965	0.8916
	全监督	0.7733	0.7528	0.8512	0.8941	0.8721
	边界框掩码	0.6873	0.6967	0.7483	0.8940	0.8147
	GrabCut	0.6928	0.6917	0.7565	0.8916	0.8185
NUAA-SIRST	BoxSup	0.7003	0.6965	0.8398	0.7901	0.8141
	BoxInst	0.7311	0.7312	0.8663	0.8240	0.8446
	BoxTeacher	0.7361	0.7287	0.8566	0.8396	0.8480
	BoxInf(本文)	0.7460	0.7349	0.8671	0.8423	0.8545
	全监督	0.7279	0.6856	0.8355	0.8488	0.8421
	边界框掩码	0.5734	0.5306	0.6117	0.9008	0.7286
	GrabCut	0.5916	0.5731	0.6428	0.8805	0.7431
IRSTD-1k	BoxSup	0.6053	0.6370	0.7644	0.7237	0.7435
	BoxInst	0.6515	0.6552	0.7655	0.8134	0.7655
	BoxTeacher	0.6553	0.6748	0.7947	0.7882	0.7914
	BoxInf(本文)	0.6796	0.6716	0.8013	0.8166	0.8089

数据集	方法	IoU	nIoU	准确率	召回率	F₁
	全监督	0.8836	0.8931	0.9453	0.9313	0.9382
	边界框掩码	0.5677	0.6419	0.6223	0.8661	0.8242
	GrabCut	0.6547	0.6781	0.6871	0.9329	0.7913
NUDT-SIRST	BoxSup	0.6676	0.6848	0.7007	0.934	0.8007
	BoxInst	0.7762	0.7902	0.8345	0.9175	0.8740
	BoxTeacher	0.7775	0.7821	0.8701	0.8796	0.8749
	BoxInf(本文)	0.8044	0.8264	0.8867	0.8965	0.8916
	全监督	0.7733	0.7528	0.8512	0.8941	0.8721
	边界框掩码	0.6873	0.6967	0.7483	0.8940	0.8147
	GrabCut	0.6928	0.6917	0.7565	0.8916	0.8185
NUAA-SIRST	BoxSup	0.7003	0.6965	0.8398	0.7901	0.8141
	BoxInst	0.7311	0.7312	0.8663	0.8240	0.8446
	BoxTeacher	0.7361	0.7287	0.8566	0.8396	0.8480
	BoxInf(本文)	0.7460	0.7349	0.8671	0.8423	0.8545
	全监督	0.7279	0.6856	0.8355	0.8488	0.8421
	边界框掩码	0.5734	0.5306	0.6117	0.9008	0.7286
	GrabCut	0.5916	0.5731	0.6428	0.8805	0.7431
IRSTD-1k	BoxSup	0.6053	0.6370	0.7644	0.7237	0.7435
	BoxInst	0.6515	0.6552	0.7655	0.8134	0.7655
	BoxTeacher	0.6553	0.6748	0.7947	0.7882	0.7914
	BoxInf(本文)	0.6796	0.6716	0.8013	0.8166	0.8089

方法	损失函数	IoU	nIoU	准确率	召回率	F₁
边界框掩码	DiceIoU-Loss	0.5580	0.6585	0.5810	0.9337	0.7163
边界框掩码	SoftIoU-Loss	0.5677	0.6419	0.6223	0.8661	0.8242
BoxInf	DiceIoU-Loss	0.8044	0.8264	0.8867	0.8965	0.8916
BoxInf	SoftIoU-Loss	0.7585	0.7728	0.8854	0.8410	0.8626

方法	损失函数	IoU	nIoU	准确率	召回率	F₁
边界框掩码	DiceIoU-Loss	0.5580	0.6585	0.5810	0.9337	0.7163
边界框掩码	SoftIoU-Loss	0.5677	0.6419	0.6223	0.8661	0.8242
BoxInf	DiceIoU-Loss	0.8044	0.8264	0.8867	0.8965	0.8916
BoxInf	SoftIoU-Loss	0.7585	0.7728	0.8854	0.8410	0.8626

Project-Loss	Shape-Loss	EMA	IoU	nIoU	准确率	召回率	F₁
边界框掩码			0.5677	0.6419	0.6223	0.8661	0.8242
√			0.7702	0.7886	0.8620	0.8786	0.8702
√	√		0.7919	0.8142	0.8957	0.8723	0.8838
√	√	√	0.8044	0.8264	0.8867	0.8965	0.8916