基于COSNet的伪装目标分割

doi:10.12382/bgxb.2022.0067

摘要/Abstract

摘要：

近年来,对伪装目标进行精准识别的军事需求不断加大,使得伪装目标分割(COS)技术应运而生。由于伪装目标与背景的融合度较高,COS比传统的目标分割难度更大。为更加精准地分割出伪装目标,构建完备的军用伪装目标数据集(MiCOD),并提出一种基于人类视觉系统的COS网络—COSNet。COSNet由特征提取模块、聚焦放大模块、多尺度特征图融合模块3部分组成。针对性设计的聚焦放大模块包含关键点聚焦模块和感受野放大模块,关键点聚焦模块通过模拟人类注意力高度集中的观察过程减少虚警率,而感受野放大模块通过仿生人类视觉感受野机制以增大观测范围、提升分割精度。损失函数方面,依据聚焦放大模块设计了更适用于伪装目标识别的关键点区域加权感知损失,以给予伪装目标更高的关注度。大量定量和定性实验结果表明:在自建数据集MiCOD上,与其他目标分割模型对比,COSNet在8个评价指标上均达到最优效果,分割精度明显提升;当模拟真实的战场环境时,COSNet平均灵敏度Sen_mean为0.622,平均特异度Spe_mean为0.670,漏检率和虚警率均低于其他算法。

关键词: 伪装目标分割, 计算机视觉, 图像分割, 关键点, 数据集

Abstract:

In recent years, the increasing military need for accurate identification of camouflaged objects has brought camouflaged object segmentation (COS) technology into existence. COS is more difficult than traditional object segmentation because of the high “integration” of camouflaged objects with the background. In order to segment the camouflaged objects more accurately, we first construct a complete military camouflaged object dataset (MiCOD), and then propose a human vision system-based camouflaged object segmentation network called COSNet. COSNet consists of three parts: featrue extraction module, focus and magnification module, and multi-scale feature fusion module. The focus and magnification module consists of two key serial modules, namely, the key point focus module and the receptive field magnification module. The key point focus module reduces the false alarm rate by simulating the observation process with high human attention, while the receptive field magnification module increases the observation range to improve the segmentation accuracy by imitating the human visual receptive field mechanism. As for the loss function, key point weighted perceptual loss is designed based on the focus and magnification module, which is more suitable for the recognition of camouflaged objects. A large number of quantitative and qualitative experiments on MiCOD demonstrate that COSNet achieves optimal results in eight evaluation metrics and significantly improves the segmentation accuracy. When simulating real battlefield environment, Sen_mean is 0.622, Spe_mean is 0.670, and the missed detection rate and false alarm rate are lower compared to other algorithms.

Key words: camouflaged object segmentation, computer vision, image segmentation, key point, dataset

蒋昕昊, 蔡伟, 张志利, 姜波, 杨志勇, 王鑫. 基于COSNet的伪装目标分割[J]. 兵工学报, 2023, 44(5): 1456-1468.

JIANG Xinhao, CAI Wei, ZHANG Zhili, JIANG Bo, YANG Zhiyong, WANG Xin. Camouflaged Object Segmentation Based on COSNet[J]. Acta Armamentarii, 2023, 44(5): 1456-1468.

图/表 14

图1 军用伪装目标示例

Fig.1 Example of military camouflaged object

表1 变换矩阵

Table 1 Transformation matrix

平移矩阵	旋转矩阵
H_shift= $10 d x 01 d y 001$	H_rotation= $c o s θ - s i n θ 0 s i n θ c o s θ 0 001$
水平翻转矩阵	垂直翻转矩阵
H_horizon= $- 1 00 010001$	H_vertical= $100 0 - 1 0 001$
缩放矩阵	错切矩阵
H_scale= $s x 00 0 s y 0 001$	H_shear= $1 s h y 0 s h x 10 001$

表1 变换矩阵

Table 1 Transformation matrix

平移矩阵	旋转矩阵
H_shift= $10 d x 01 d y 001$	H_rotation= $c o s θ - s i n θ 0 s i n θ c o s θ 0 001$
水平翻转矩阵	垂直翻转矩阵
H_horizon= $- 1 00 010001$	H_vertical= $100 0 - 1 0 001$
缩放矩阵	错切矩阵
H_scale= $s x 00 0 s y 0 001$	H_shear= $1 s h y 0 s h x 10 001$

表2 数据集概况

Table 2 Dataset overview

属性	描述	数量
	春季山林	800
伪装人员	夏季山林	900
	秋季山林	400
	冬季山林	500
伪装坦克	复杂环境	100
自然环境	不含伪装目标	1300

图2 数据集示例(左为RGB图像,右为真值图)

Fig.2 Example of dataset (RGB images on the left, GT maps on the right)

图3 COSNet目标分割模型网络结构聚焦放大模块真值图

Fig.3 Network structure of COSNet object segmentation model

图4 关键点聚焦模块结构图

Fig.4 Diagram of KPFM structure

图5 感受野放大模块结构图

Fig.5 Diagram of RFAM structure

表3 在自建MiCOD数据集上的对比实验结果1(每幅图片均含有伪装目标)

Table 3 Comparative experiment result 1 on self-built MiCOD dataset (Each picture contains camouflage objects)

算法	来源	Dic_mean	IoU_mean	Sen_mean	Spe_mean	S_m	MAE	adpE_m	adpF_m
UNet++□	DLMIA’17	0.513	0.421	0.471	0.747	0.717	0.009	0.736	0.532
SCRN◇	ICCV’19	0.687	0.575	0.726	0.955	0.847	0.010	0.677	0.761
BASNet◇	CVPR’19	0.763	0.666	0.758	0.950	0.865	0.008	0.928	0.782
F3Net◇	AAAI’20	0.816	0.716	0.846	0.972	0.889	0.005	0.944	0.824
PraNet□	MICCAI’20	0.802	0.696	0.834	0.977	0.887	0.006	0.915	0.814
SINet-V1▯	CVPR’20	0.810	0.706	0.842	0.977	0.876	0.005	0.965	0.813
PFNet▯	CVPR’21	0.785	0.682	0.804	0.965	0.873	0.006	0.941	0.794
UACANet-L□	ACM MM’21	0.817	0.715	0.853	0.979	0.880	0.004	0.963	0.820
COSNet▯	本文算法	0.864	0.775	0.913	0.989	0.921	0.003	0.944	0.872

表4 在自建MiCOD数据集上的对比实验结果2(每幅图片不总是含有伪装目标)

Table 4 Comparative experiment result 2 on self-built MiCOD dataset (The camouflaged object are not always present)

算法	来源	Dic_mean	IoU_mean	Sen_mean	Spe_mean	S_m	MAE	adpE_m	adpF_m
UNet++□	DLMIA’17	0.346	0.284	0.318	0.504	0.809	0.006	0.543	0.359
SCRN◇	ICCV’19	0.442	0.366	0.470	0.638	0.816	0.069	0.777	0.496
BASNet◇	CVPR’19	0.516	0.451	0.522	0.631	0.903	0.014	0.917	0.527
F3Net◇	AAAI’20	0.536	0.469	0.572	0.644	0.919	0.004	0.649	0.542
PraNet□	MICCAI’20	0.534	0.463	0.553	0.656	0.914	0.008	0.931	0.540
SINet-V1▯	CVPR’20	0.542	0.471	0.563	0.656	0.907	0.017	0.956	0.546
PFNet▯	CVPR’21	0.528	0.459	0.542	0.647	0.915	0.004	0.812	0.534
UACANet-L□	ACM MM’21	0.552	0.484	0.572	0.661	0.919	0.004	0.978	0.554
COSNet▯	本文算法	0.608	0.559	0.622	0.670	0.956	0.003	0.991	0.610

表5 在自建MiCOD数据集上的可视化对比结果

Table 5 Visual comparison results on self-built MiCOD dataset

表6 在通用伪装目标数据集上的对比实验结果

Table 6 Comparative experimental results on general camouflaged object dataset

算法	来源	Dic_mean	IoU_mean	Sen_mean	Spe_mean	S_m	MAE	adpE_m	adpF_m
UNet++□	DLMIA’17	0.529	0.416	0.553	0.859	0.678	0.067	0.763	0.549
BASNet◇	CVPR’19	0.490	0.381	0.611	0.865	0.663	0.097	0.732	0.505
SCRN◇	ICCV’19	0.640	0.529	0.676	0.926	0.791	0.052	0.799	0.709
F3Net◇	AAAI’20	0.675	0.565	0.709	0.940	0.781	0.049	0.851	0.687
PraNet□	MICCAI’20	0.700	0.595	0.737	0.939	0.799	0.045	0.866	0.726
SINet-V1▯	CVPR’20	0.714	0.608	0.737	0.948	0.806	0.039	0.883	0.724
PFNet▯	CVPR’21	0.722	0.617	0.749	0.952	0.810	0.038	0.886	0.732
UACANet-L□	ACM MM’21	0.745	0.646	0.763	0.945	0.816	0.034	0.901	0.748
COSNet▯	本文算法	0.765	0.659	0.769	0.956	0.837	0.032	0.901	0.771

表7 在通用伪装目标数据集上的可视化对比结果

Table 7 Visual comparison results on general camouflaged object dataset

表8 在自建MiCOD数据集上的消融实验结果

Table 8 Ablation experimental results on self-built MiCOD dataset

基线网络	添加KPFM	添加RFAM	Dic_mean	IoU_mean	Sen_mean	Spe_mean	S_m	MAE	adpE_m	adpF_m
√			0.335	0.232	0.813	0.745	0.507	0.179	0.498	0.515
√	√		0.413	0.346	0.767	0.720	0.523	0.191	0.638	0.622
√		√	0.814	0.712	0.835	0.983	0.886	0.005	0.952	0.821
√	√	√	0.864	0.775	0.913	0.989	0.921	0.003	0.944	0.872

表9 在自建MiCOD数据集上的消融实验可视化对比结果

Table 9 Visual comparison results of ablation experiment on self-built MiCOD dataset

参考文献 35

[1]	STEVENS M, MERILAITA S. Animal camouflage: current issues and new perspectives[J]. Philosophical Transactions of the Royal Society B Biological Sciences, 2009, 364(1516):423-427. doi: 10.1098/rstb.2008.0217 URL
[2]	STUART-FOX D, MOUSSALLI A, WHITING M J. Predator-specific camouflage in chameleons[J]. Biology Letters, 2008, 4(4):326-329. doi: 10.1098/rsbl.2008.0173 URL
[3]	PUZIKOVA N P, UVAROVA E V, FILYAEV I M, et al. Principles of an approach for coloring military camouflage[J]. Fibre Chemistry, 2008, 40(2):155-159. doi: 10.1007/s10692-008-9030-9 URL
[4]	DING Y, ZHAO X F, ZHANG Z L, et al. Semi-supervised locality preserving dense graph neural network with ARMA filters and context-aware learning for hyperspectral image classification[J/OL]. IEEE Transactions on Geoscience and Remote Sensing.(2021-08-04)[2021-11-06]. https:∥doi.org/10.1109/TGRS.2021.3100578. doi: 10.1109/TGRS.2021.3100578
[5]	ZHANG J J, ZHANG X, LI T, et al. Visible light polarization image desmogging via cycle convolutional neural network[J/OL]. Multimedia Systems.(2021-04-30)[2021-11-06]. https:∥doi.org/10.1007/s00530-021-00802-9. doi: 10.1007/s00530-021-00802-9
[6]	JIANG X H, CAI W, YANG Z Y, et al. IARet: a lightweight multiscale infrared aerocraft recognition algorithm[J/OL]. Arabian Journal for Science and Engineerin.(2021-09-24)[2021-11-06].
[7]	周静, 窦一民, 李金屏. 基于光流场分割的伪装色运动目标检测[J]. 济南大学学报(自然科学版), 2020, 34(4):328-334.
	ZHOU J, DOU Y M, LI J P. Camouflage color moving object detection based on optical flow field segmentation[J]. Journal of University of Jinan (Science and Technology), 2020, 34(4):328-334. (in Chinese)
[8]	FAN D P, JI G P, SUN G L, et al. Camouflaged object detection[C]∥Proceedings of IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway, NJ, US:IEEE, 2020:2774-2784.
[9]	MEI H Y, JI G P, WEI Z Q, et al. Camouflaged object segmentation with distraction mining[C]∥Proceedings of IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway, NJ, US:IEEE, 2021:8768-8777.
[10]	HOU J, GRAHAM B, NIESNERM, et al. Exploring data-efficient 3D scene understanding with contrastive scene contexts[C]∥Proceedings of IEEE/CVF Conference on Computer Vision and Pattern Recognition.Piscataway, NJ, US:IEEE, 2021:15582-15592.
[11]	HUANG J H, WANG H Q, BIRDAL T, et al. MultiBodySync: multi-body segmentation and motion estimation via 3D scan synchronization[C]∥Proceedings of IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway, NJ, US:IEEE, 2021:7104-7114.
[12]	LIU Z Z, QI X J, FU C W. One thing one click: a self-training approach for weakly supervised 3D semantic segmentation[C]∥Proceedings of IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway, NJ, US:IEEE, 2021:1726-1736.
[13]	CHEN X K, YUAN Y H, ZENG G, et al. Semi-supervised semantic segmentation with cross pseudo supervision[C]∥Proceedings of IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway, NJ, US:IEEE, 2021:2613-2622.
[14]	YAO Y Z, CHEN T, XIE G S, et al. Non-salient region object mining for weakly supervised semantic segmentation[C]∥Proceedings of IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway, NJ, US:IEEE, 2021:2623-2632.
[15]	FU Y, YANG L J, LIU D, et al. CompFeat: comprehensive feature aggregation for video instance segmentation[C]∥Proceedings of the AAAI Conference on Artificial Intelligence. Menlo Park, NJ, US:AAAI, 2019, 35(2):1361-1369.
[16]	WEI J, HU Y W, ZHANG R M, et al. Shallow attention network for polyp segmentation[EB/OL]. (2021-09-24)[2021-11-06].
[17]	WANG J F, SONG L, LI Z M, et al. End-to-end object detection with fully convolutional network[C]∥Proceedings of IEEE/CVF Conference on Computer Vision and Pattern Recognition.Piscataway, NJ, US:IEEE, 2021:15844-15853.
[18]	PATEL K, BUR A M, WANG G H, et al. Enhanced U-Net:a feature enhancement network for polyp segmentation[C]∥Proceedings of the 2021 18th Conference on Robots and Vision.Piscataway, NJ, US:IEEE, 2021:181-188.
[19]	FAN H, MEI X, PROKHOROV D, et al. RGB-D scene labeling with multimodal recurrent neural networks[C]∥Proceedings of 2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).Piscataway, NJ, US:IEEE, 2017:203-211.
[20]	LIU K H, YE Z H, GUO H Y, et al. FISS GAN: a generative adversarial network for foggy image semantic segmentation[J]. IEEE/CAA Journal of Automatica Sinica, 2021, 8(8):1428-1439. doi: 10.1109/JAS.2021.1004057 URL
[21]	LE T N, CAO Y, NGUYEN T C, et al. Camouflaged instance segmentation in-the-wild: dataset, method, and benchmark suite[J]. IEEE Transactions on Image Processing, 2022, 31: 287-300. doi: 10.1109/TIP.2021.3130490 URL
[22]	梁新宇, 林浩坤, 杨辉, 等. 军用伪装目标图像语义分割数据集的构建[J]. 激光与光电子学进展, 2021, 58(4):0410015.
	LIANG X Y, LIN H K, YANG H, et al. Construction of semantic segmentation dataset of camouflage target image[J]. Laser & Optoelectronics Progress, 2021, 58(4):0410015. (in Chinese)
[23]	HALL J R, CUTHILL I C, BADDELEY R J, et al. Camouflage, detection and identification of moving targets[J]. Proceedings of the Royal Society B: Biological Sciences, 2013, 280(1758):20130064. doi: 10.1098/rspb.2013.0064 URL
[24]	GAO S, CHENG M M, ZHAO K, et al. Res2net: a new multi-scale backbone architecture[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2019, 43(2):652-662. doi: 10.1109/TPAMI.34 URL
[25]	WEI J H, WANG S H, HUANG Q M. F³Net: fusion, feedback and focus for salient object detection[C]∥Proceedings of the AAAI Conference on Artificial Intelligence. Menlo Park, NJ, US:AAAI, 2021, 34(7):12321-12328.
[26]	PERAZZI F, KRÄHENBÜHL P, PRITCH Y, et al. Saliency filters: contrast based filtering for salient region detection[C]∥Proceedings of 2012 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway, NJ, US:IEEE, 2012:733-740.
[27]	FAN D P, CHENG M M, LIU Y, et al. Structure-measure: a new way to evaluate foreground maps[C]∥Proceedings of 2017 IEEE International Conference on Computer Vision. Piscataway, NJ, US:IEEE, 2017:4558-4567.
[28]	FAN D P, GONG C, CAO Y, et al. Enhanced-alignment measure for binary foreground map evaluation[C]∥Proceedings of the Twenty-Seventh International Joint Conference on Artificial Intelligence. San Francisco,CA, US: Margan Kaufmann, 2018:698-704.
[29]	MARGOLIN R, ZELNIK-MANOR L, TAL A. How to evaluate foreground maps[C]∥Proceedings of 2014 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway, NJ, US:IEEE, 2014:248-255.
[30]	MILLETARI F, NASSIR N, SEYED-AHMAD A. V-Net: fully convolutional neural networks for volumetric medical image segmentation[C]∥Proceedings of the 2016 4th International Conference on 3D Vision (3DV). CA, US: Computer Society’s Conference Publishing Services, 2016: 565-571.
[31]	ZHOU Z, IDDIQUEE M M, TAJBAKHSH N, et al. UNet++: redesigning skip connections to exploit multiscale features in image segmentation[J]. IEEE Transactions on Medical Imaging, 2020, 39:1856-1867. doi: 10.1109/TMI.2019.2959609 pmid: 31841402
[32]	FAN D P, JI G P, ZHOU T, et al. Pranet: parallel reverse attention network for polyp segmentation[C]∥Proceedings of International Conference on Medical Image Computing and Computer-Assisted Inter-vention.Berlin, Germany:Springer, 2020:263-273.
[33]	KIM T, LEE H, KIM D. UACANet: uncertainty augmented context attention for polyp segmentation[C]∥Proceedings of the 29th ACM International Conference on Multimedia. New York, NY, US:ACM, 2021:2167-2175.
[34]	QIN X B, ZHANG Z V, HUANG C Y, et al. BASNet: Boundary-Aware Salient Object Detection[C]∥Proceedings of 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway,NJ, US:IEEE, 2019:7471-7481.
[35]	WU Z, SU L, HUANG Q M. Stacked cross refinement network for edge-aware salient object detection[C]∥Proceedings of 2019 IEEE/CVF International Conference on Computer Vision. Piscataway, NJ, US:IEEE, 2019:7263-7272.