Camouflaged Object Segmentation Based on COSNet

doi:10.12382/bgxb.2022.0067

Abstract

Abstract:

In recent years, the increasing military need for accurate identification of camouflaged objects has brought camouflaged object segmentation (COS) technology into existence. COS is more difficult than traditional object segmentation because of the high “integration” of camouflaged objects with the background. In order to segment the camouflaged objects more accurately, we first construct a complete military camouflaged object dataset (MiCOD), and then propose a human vision system-based camouflaged object segmentation network called COSNet. COSNet consists of three parts: featrue extraction module, focus and magnification module, and multi-scale feature fusion module. The focus and magnification module consists of two key serial modules, namely, the key point focus module and the receptive field magnification module. The key point focus module reduces the false alarm rate by simulating the observation process with high human attention, while the receptive field magnification module increases the observation range to improve the segmentation accuracy by imitating the human visual receptive field mechanism. As for the loss function, key point weighted perceptual loss is designed based on the focus and magnification module, which is more suitable for the recognition of camouflaged objects. A large number of quantitative and qualitative experiments on MiCOD demonstrate that COSNet achieves optimal results in eight evaluation metrics and significantly improves the segmentation accuracy. When simulating real battlefield environment, Sen_mean is 0.622, Spe_mean is 0.670, and the missed detection rate and false alarm rate are lower compared to other algorithms.

Key words: camouflaged object segmentation, computer vision, image segmentation, key point, dataset

JIANG Xinhao, CAI Wei, ZHANG Zhili, JIANG Bo, YANG Zhiyong, WANG Xin. Camouflaged Object Segmentation Based on COSNet[J]. Acta Armamentarii, 2023, 44(5): 1456-1468.

Figures/Tables 14

Fig.1 Example of military camouflaged object

Table 1 Transformation matrix

平移矩阵	旋转矩阵
H_shift= $10 d x 01 d y 001$	H_rotation= $c o s θ - s i n θ 0 s i n θ c o s θ 0 001$
水平翻转矩阵	垂直翻转矩阵
H_horizon= $- 1 00 010001$	H_vertical= $100 0 - 1 0 001$
缩放矩阵	错切矩阵
H_scale= $s x 00 0 s y 0 001$	H_shear= $1 s h y 0 s h x 10 001$

Table 1 Transformation matrix

平移矩阵	旋转矩阵
H_shift= $10 d x 01 d y 001$	H_rotation= $c o s θ - s i n θ 0 s i n θ c o s θ 0 001$
水平翻转矩阵	垂直翻转矩阵
H_horizon= $- 1 00 010001$	H_vertical= $100 0 - 1 0 001$
缩放矩阵	错切矩阵
H_scale= $s x 00 0 s y 0 001$	H_shear= $1 s h y 0 s h x 10 001$

Table 2 Dataset overview

属性	描述	数量
	春季山林	800
伪装人员	夏季山林	900
	秋季山林	400
	冬季山林	500
伪装坦克	复杂环境	100
自然环境	不含伪装目标	1300

Fig.2 Example of dataset (RGB images on the left, GT maps on the right)

Fig.3 Network structure of COSNet object segmentation model

Fig.4 Diagram of KPFM structure

Fig.5 Diagram of RFAM structure

Table 3 Comparative experiment result 1 on self-built MiCOD dataset (Each picture contains camouflage objects)

算法	来源	Dic_mean	IoU_mean	Sen_mean	Spe_mean	S_m	MAE	adpE_m	adpF_m
UNet++□	DLMIA’17	0.513	0.421	0.471	0.747	0.717	0.009	0.736	0.532
SCRN◇	ICCV’19	0.687	0.575	0.726	0.955	0.847	0.010	0.677	0.761
BASNet◇	CVPR’19	0.763	0.666	0.758	0.950	0.865	0.008	0.928	0.782
F3Net◇	AAAI’20	0.816	0.716	0.846	0.972	0.889	0.005	0.944	0.824
PraNet□	MICCAI’20	0.802	0.696	0.834	0.977	0.887	0.006	0.915	0.814
SINet-V1▯	CVPR’20	0.810	0.706	0.842	0.977	0.876	0.005	0.965	0.813
PFNet▯	CVPR’21	0.785	0.682	0.804	0.965	0.873	0.006	0.941	0.794
UACANet-L□	ACM MM’21	0.817	0.715	0.853	0.979	0.880	0.004	0.963	0.820
COSNet▯	本文算法	0.864	0.775	0.913	0.989	0.921	0.003	0.944	0.872

Table 4 Comparative experiment result 2 on self-built MiCOD dataset (The camouflaged object are not always present)

算法	来源	Dic_mean	IoU_mean	Sen_mean	Spe_mean	S_m	MAE	adpE_m	adpF_m
UNet++□	DLMIA’17	0.346	0.284	0.318	0.504	0.809	0.006	0.543	0.359
SCRN◇	ICCV’19	0.442	0.366	0.470	0.638	0.816	0.069	0.777	0.496
BASNet◇	CVPR’19	0.516	0.451	0.522	0.631	0.903	0.014	0.917	0.527
F3Net◇	AAAI’20	0.536	0.469	0.572	0.644	0.919	0.004	0.649	0.542
PraNet□	MICCAI’20	0.534	0.463	0.553	0.656	0.914	0.008	0.931	0.540
SINet-V1▯	CVPR’20	0.542	0.471	0.563	0.656	0.907	0.017	0.956	0.546
PFNet▯	CVPR’21	0.528	0.459	0.542	0.647	0.915	0.004	0.812	0.534
UACANet-L□	ACM MM’21	0.552	0.484	0.572	0.661	0.919	0.004	0.978	0.554
COSNet▯	本文算法	0.608	0.559	0.622	0.670	0.956	0.003	0.991	0.610

Table 5 Visual comparison results on self-built MiCOD dataset

Table 6 Comparative experimental results on general camouflaged object dataset

算法	来源	Dic_mean	IoU_mean	Sen_mean	Spe_mean	S_m	MAE	adpE_m	adpF_m
UNet++□	DLMIA’17	0.529	0.416	0.553	0.859	0.678	0.067	0.763	0.549
BASNet◇	CVPR’19	0.490	0.381	0.611	0.865	0.663	0.097	0.732	0.505
SCRN◇	ICCV’19	0.640	0.529	0.676	0.926	0.791	0.052	0.799	0.709
F3Net◇	AAAI’20	0.675	0.565	0.709	0.940	0.781	0.049	0.851	0.687
PraNet□	MICCAI’20	0.700	0.595	0.737	0.939	0.799	0.045	0.866	0.726
SINet-V1▯	CVPR’20	0.714	0.608	0.737	0.948	0.806	0.039	0.883	0.724
PFNet▯	CVPR’21	0.722	0.617	0.749	0.952	0.810	0.038	0.886	0.732
UACANet-L□	ACM MM’21	0.745	0.646	0.763	0.945	0.816	0.034	0.901	0.748
COSNet▯	本文算法	0.765	0.659	0.769	0.956	0.837	0.032	0.901	0.771

Table 7 Visual comparison results on general camouflaged object dataset

Table 8 Ablation experimental results on self-built MiCOD dataset

基线网络	添加KPFM	添加RFAM	Dic_mean	IoU_mean	Sen_mean	Spe_mean	S_m	MAE	adpE_m	adpF_m
√			0.335	0.232	0.813	0.745	0.507	0.179	0.498	0.515
√	√		0.413	0.346	0.767	0.720	0.523	0.191	0.638	0.622
√		√	0.814	0.712	0.835	0.983	0.886	0.005	0.952	0.821
√	√	√	0.864	0.775	0.913	0.989	0.921	0.003	0.944	0.872

Table 9 Visual comparison results of ablation experiment on self-built MiCOD dataset

References 35

[1]	STEVENS M, MERILAITA S. Animal camouflage: current issues and new perspectives[J]. Philosophical Transactions of the Royal Society B Biological Sciences, 2009, 364(1516):423-427. doi: 10.1098/rstb.2008.0217 URL
[2]	STUART-FOX D, MOUSSALLI A, WHITING M J. Predator-specific camouflage in chameleons[J]. Biology Letters, 2008, 4(4):326-329. doi: 10.1098/rsbl.2008.0173 URL
[3]	PUZIKOVA N P, UVAROVA E V, FILYAEV I M, et al. Principles of an approach for coloring military camouflage[J]. Fibre Chemistry, 2008, 40(2):155-159. doi: 10.1007/s10692-008-9030-9 URL
[4]	DING Y, ZHAO X F, ZHANG Z L, et al. Semi-supervised locality preserving dense graph neural network with ARMA filters and context-aware learning for hyperspectral image classification[J/OL]. IEEE Transactions on Geoscience and Remote Sensing.(2021-08-04)[2021-11-06]. https:∥doi.org/10.1109/TGRS.2021.3100578. doi: 10.1109/TGRS.2021.3100578
[5]	ZHANG J J, ZHANG X, LI T, et al. Visible light polarization image desmogging via cycle convolutional neural network[J/OL]. Multimedia Systems.(2021-04-30)[2021-11-06]. https:∥doi.org/10.1007/s00530-021-00802-9. doi: 10.1007/s00530-021-00802-9
[6]	JIANG X H, CAI W, YANG Z Y, et al. IARet: a lightweight multiscale infrared aerocraft recognition algorithm[J/OL]. Arabian Journal for Science and Engineerin.(2021-09-24)[2021-11-06].
[7]	周静, 窦一民, 李金屏. 基于光流场分割的伪装色运动目标检测[J]. 济南大学学报(自然科学版), 2020, 34(4):328-334.
	ZHOU J, DOU Y M, LI J P. Camouflage color moving object detection based on optical flow field segmentation[J]. Journal of University of Jinan (Science and Technology), 2020, 34(4):328-334. (in Chinese)
[8]	FAN D P, JI G P, SUN G L, et al. Camouflaged object detection[C]∥Proceedings of IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway, NJ, US:IEEE, 2020:2774-2784.
[9]	MEI H Y, JI G P, WEI Z Q, et al. Camouflaged object segmentation with distraction mining[C]∥Proceedings of IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway, NJ, US:IEEE, 2021:8768-8777.
[10]	HOU J, GRAHAM B, NIESNERM, et al. Exploring data-efficient 3D scene understanding with contrastive scene contexts[C]∥Proceedings of IEEE/CVF Conference on Computer Vision and Pattern Recognition.Piscataway, NJ, US:IEEE, 2021:15582-15592.
[11]	HUANG J H, WANG H Q, BIRDAL T, et al. MultiBodySync: multi-body segmentation and motion estimation via 3D scan synchronization[C]∥Proceedings of IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway, NJ, US:IEEE, 2021:7104-7114.
[12]	LIU Z Z, QI X J, FU C W. One thing one click: a self-training approach for weakly supervised 3D semantic segmentation[C]∥Proceedings of IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway, NJ, US:IEEE, 2021:1726-1736.
[13]	CHEN X K, YUAN Y H, ZENG G, et al. Semi-supervised semantic segmentation with cross pseudo supervision[C]∥Proceedings of IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway, NJ, US:IEEE, 2021:2613-2622.
[14]	YAO Y Z, CHEN T, XIE G S, et al. Non-salient region object mining for weakly supervised semantic segmentation[C]∥Proceedings of IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway, NJ, US:IEEE, 2021:2623-2632.
[15]	FU Y, YANG L J, LIU D, et al. CompFeat: comprehensive feature aggregation for video instance segmentation[C]∥Proceedings of the AAAI Conference on Artificial Intelligence. Menlo Park, NJ, US:AAAI, 2019, 35(2):1361-1369.
[16]	WEI J, HU Y W, ZHANG R M, et al. Shallow attention network for polyp segmentation[EB/OL]. (2021-09-24)[2021-11-06].
[17]	WANG J F, SONG L, LI Z M, et al. End-to-end object detection with fully convolutional network[C]∥Proceedings of IEEE/CVF Conference on Computer Vision and Pattern Recognition.Piscataway, NJ, US:IEEE, 2021:15844-15853.
[18]	PATEL K, BUR A M, WANG G H, et al. Enhanced U-Net:a feature enhancement network for polyp segmentation[C]∥Proceedings of the 2021 18th Conference on Robots and Vision.Piscataway, NJ, US:IEEE, 2021:181-188.
[19]	FAN H, MEI X, PROKHOROV D, et al. RGB-D scene labeling with multimodal recurrent neural networks[C]∥Proceedings of 2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).Piscataway, NJ, US:IEEE, 2017:203-211.
[20]	LIU K H, YE Z H, GUO H Y, et al. FISS GAN: a generative adversarial network for foggy image semantic segmentation[J]. IEEE/CAA Journal of Automatica Sinica, 2021, 8(8):1428-1439. doi: 10.1109/JAS.2021.1004057 URL
[21]	LE T N, CAO Y, NGUYEN T C, et al. Camouflaged instance segmentation in-the-wild: dataset, method, and benchmark suite[J]. IEEE Transactions on Image Processing, 2022, 31: 287-300. doi: 10.1109/TIP.2021.3130490 URL
[22]	梁新宇, 林浩坤, 杨辉, 等. 军用伪装目标图像语义分割数据集的构建[J]. 激光与光电子学进展, 2021, 58(4):0410015.
	LIANG X Y, LIN H K, YANG H, et al. Construction of semantic segmentation dataset of camouflage target image[J]. Laser & Optoelectronics Progress, 2021, 58(4):0410015. (in Chinese)
[23]	HALL J R, CUTHILL I C, BADDELEY R J, et al. Camouflage, detection and identification of moving targets[J]. Proceedings of the Royal Society B: Biological Sciences, 2013, 280(1758):20130064. doi: 10.1098/rspb.2013.0064 URL
[24]	GAO S, CHENG M M, ZHAO K, et al. Res2net: a new multi-scale backbone architecture[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2019, 43(2):652-662. doi: 10.1109/TPAMI.34 URL
[25]	WEI J H, WANG S H, HUANG Q M. F³Net: fusion, feedback and focus for salient object detection[C]∥Proceedings of the AAAI Conference on Artificial Intelligence. Menlo Park, NJ, US:AAAI, 2021, 34(7):12321-12328.
[26]	PERAZZI F, KRÄHENBÜHL P, PRITCH Y, et al. Saliency filters: contrast based filtering for salient region detection[C]∥Proceedings of 2012 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway, NJ, US:IEEE, 2012:733-740.
[27]	FAN D P, CHENG M M, LIU Y, et al. Structure-measure: a new way to evaluate foreground maps[C]∥Proceedings of 2017 IEEE International Conference on Computer Vision. Piscataway, NJ, US:IEEE, 2017:4558-4567.
[28]	FAN D P, GONG C, CAO Y, et al. Enhanced-alignment measure for binary foreground map evaluation[C]∥Proceedings of the Twenty-Seventh International Joint Conference on Artificial Intelligence. San Francisco,CA, US: Margan Kaufmann, 2018:698-704.
[29]	MARGOLIN R, ZELNIK-MANOR L, TAL A. How to evaluate foreground maps[C]∥Proceedings of 2014 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway, NJ, US:IEEE, 2014:248-255.
[30]	MILLETARI F, NASSIR N, SEYED-AHMAD A. V-Net: fully convolutional neural networks for volumetric medical image segmentation[C]∥Proceedings of the 2016 4th International Conference on 3D Vision (3DV). CA, US: Computer Society’s Conference Publishing Services, 2016: 565-571.
[31]	ZHOU Z, IDDIQUEE M M, TAJBAKHSH N, et al. UNet++: redesigning skip connections to exploit multiscale features in image segmentation[J]. IEEE Transactions on Medical Imaging, 2020, 39:1856-1867. doi: 10.1109/TMI.2019.2959609 pmid: 31841402
[32]	FAN D P, JI G P, ZHOU T, et al. Pranet: parallel reverse attention network for polyp segmentation[C]∥Proceedings of International Conference on Medical Image Computing and Computer-Assisted Inter-vention.Berlin, Germany:Springer, 2020:263-273.
[33]	KIM T, LEE H, KIM D. UACANet: uncertainty augmented context attention for polyp segmentation[C]∥Proceedings of the 29th ACM International Conference on Multimedia. New York, NY, US:ACM, 2021:2167-2175.
[34]	QIN X B, ZHANG Z V, HUANG C Y, et al. BASNet: Boundary-Aware Salient Object Detection[C]∥Proceedings of 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway,NJ, US:IEEE, 2019:7471-7481.
[35]	WU Z, SU L, HUANG Q M. Stacked cross refinement network for edge-aware salient object detection[C]∥Proceedings of 2019 IEEE/CVF International Conference on Computer Vision. Piscataway, NJ, US:IEEE, 2019:7263-7272.