基于卷积和注意力机制的小样本目标检测

doi:10.12382/bgxb.2022.1167

摘要/Abstract

摘要：

小样本目标检测(FSOD)旨在使检测器只用少量的训练样本就能适应未见的类别。典型的FSOD方法使用Faster R-CNN作为基本检测框架,利用卷积神经网络提取图像特征,而卷积神经网络中采用的旨在捕获尽可能多的图像信息的池化操作将不可避免地导致图像信息的丢失。在主干网络中引入混合扩张卷积,以确保更大的感受野并最大限度地减少图像信息的损失。在k-shot设置中,为充分利用给定的支持数据,提出支持特征动态融合模块,以每个支持特征和查询特征之间的相关性为权重,自适应地融合支持特征,以获得更强大的支持线索。实验结果表明,新方法在公共Pascal VOC和MS-COCO数据集上实现了较好的FSOD性能。

关键词: 小样本目标检测, 混合扩张卷积, 支持特征动态融合

Abstract:

Few-shot object detection(FSOD) aims to enable the detector with a small number of training samples. Typical FSOD method takes Faster R-CNN as the basic detection framework, and uses a convolutional neural network to extract the image features. However, the pooling operation used in the convolutional neural network inevitably leads to the loss of image information. Therefore, a hybrid dilated convolution is introduced into the backbone network to ensure a larger receptive field and minimize the loss of image information. A support feature dynamic fusion module is proposed to further utilize the given support data in k-shot setting, which adaptively fuses the support features with the weight of the correlation between each support feature and query feature to obtain stronger support clues. Experimental results show that rhe proposed method achieves good and state-of-the-art FSOD performance on public Pascal VOC and MS-COCO datasets.

Key words: few-shot object detection, hybrid dilated convolution, support feature dynamic fusion

中图分类号:

TP183

郭永红, 牛海涛, 史超, 郭铖. 基于卷积和注意力机制的小样本目标检测[J]. 兵工学报, 2023, 44(11): 3508-3515.

GUO Yonghong, NIU Haitao, SHI Chao, GUO Cheng. Few-shot Object Detection Based on Convolution Network and Attention Mechanism[J]. Acta Armamentarii, 2023, 44(11): 3508-3515.

图/表 6

参考文献 25

[1]	王芳, 王海晏, 寇添, 等. 多维特征点空间的红外弱小目标检测方法[J]. 应用光学, 2020, 41(6):1268-1276. doi: 10.5768/JAO202041.0606002
	WANG F, WANG H Y, KOU T, et al. Multi-dimensional feature point space infrared dim target detection method[J]. Journal of Applied Optics, 2020, 41(6):1268-1276. (in Chinese) doi: 10.5768/JAO202041.0606002
[2]	牛畅, 尹奎英, 黄银和. 无人机对地目标自动检测与跟踪技术[J]. 应用光学, 2020, 41(6): 1153-1160. doi: 10.5768/JAO202041.0601003
	NIU C, YIN K Y, HUANG Y H. Automatic target detecting and tracking technology based on UAV ground target images[J]. Journal of Applied Optics, 2020, 41(6): 1153-1160. (in Chinese) doi: 10.5768/JAO202041.0601003
[3]	杜鹏飞, 李小勇, 高雅丽. 多模态视觉语言表征学习研究综述[J]. 软件学报, 2021, 32(2): 327-348.
	DU P F, LI X Y, GAO Y L. Survey on multimodal visual language representation learning[J]. Journal of Software, 2021, 32(2): 327-348. (in Chinese)
[4]	KANG B Y, LIU Z, WANG X, et al. Few-shot object detection via feature reweighting[C]// Proceedings of the IEEE/CVF International Conference on Computer Vision. Changshu, China: IEEE, 2019: 8420-8429.
[5]	梁杰, 任君, 李磊, 等. 基于典型几何形状精确回归的机场跑道检测方法[J]. 兵工学报, 2020, 41(10): 2045-2054.
	LIANG J, REN J, LI L, et al. Airport runway detection algorithm based on accurate regression of typical geometric shapes[J]. Acta Armamentarii, 2020, 41(10): 2045-2054. (in Chinese)
[6]	WANG Y X, RAMANAN D, HEBERT M. Meta-learning to detect rare objects[C]// Proceedings of the 2019 IEEE/CVF International Conference on Computer Vision. Seoul, Korea: IEEE, 2019: 9924-9933.
[7]	ZHANG G J, LUO Z P, CUI K W, et al. Meta-DETR: image-level few-shot detection with inter-class correlation Exploitation[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2023, 45(11): 12832-12843.2, DOI: 10.1109/TPAMI.2022.3195735.
[8]	VILALTA R, DRISSI Y. A perspective view and survey of meta-learning[J]. Artificial Intelligence Review, 2002, 18(2):77-95. doi: 10.1023/A:1019956318069 URL
[9]	WEISS K, KHOSHGOFTAAR T M, WANG D D. A survey of transfer learning[J]. Journal of Big Data, 2016, 3(1):1-40. doi: 10.1186/s40537-015-0036-x URL
[10]	HAN G X, HUANG S Y, MA J W J, et al. Meta faster R-CNN:towards accurate few-shot object detection with attentive feature alignment[C]// Proceedings of the 36th AAAI Conference on Artificial Intelligence. Reston, VA,US:AIAA, 2022, 36(1): 780-789.
[11]	REN S Q, HE K M, GIRSHICK R, et al. Towards real-time object detection with region proposal networks[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2016, 39(6): 1137-1149. doi: 10.1109/TPAMI.2016.2577031 URL
[12]	QIAO L M, ZHAO Y X, LI Z Y, et al. DeFRCN:Decoupled faster R-CNN for few-shot object detection[C]// Proceedings of the IEEE/CVF International Conference on Computer Vision. Paris, France: IEEE, 2021: 8681-8690.
[13]	WANG P Q, CHEN P F, YUAN Y, et al. Understanding convolution for semantic segmentation[C]// Proceedings of 2018 IEEE Winter Conference on Applications of Computer Vision. Lake Tahoe, NV, US: IEEE, 2018: 1451-1460.
[14]	LIU W, ANGUELOV D, ERHAN D, et al. SSD:single shot multibox detector[C]// Proceedings of the 14th European Conference on Computer Vision. Amsterdam,the Netherlands: Springer, 2016: 21-37.
[15]	REDMON J, DIVVALA S, GIRSHICK R, et al. You only look once:unified, real-time object detection[C]// Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Lyon, France: IEEE, 2016: 779-788.
[16]	GIRSHICK R. Fast R-CNN[C]// Proceedings of the 2015 IEEE International Conference on Computer Vision. Santiago,Chile:IEEE, 2015:1440-1448.
[17]	WU J X, LIU S T, HUANG D, et al. Multi-scale positive sample refinement for few-shot object detection[C]// Proceedings of the 16th European Conference on Computer Vision. Glasgow, UK: Springer, 2020: 456-472.
[18]	YAN X P, CHEN Z, XU A, et al. Towards general solver for instance-level low-shot learning[C]// Proceedings of the 2019 IEEE/CVF International Conference on Computer Vision. Seoul, Korea: IEEE, 2019: 9577-9586.
[19]	WANG X, HUANG T E, DARRELL T, et al. Frustratingly simple few-shot object detection:arXiv:2003.06957[R]. Ithaca, NY, US: Cornell University, 2020:2003.06957.
[20]	SUN B, LI B H, CAI S C, et al. Few-shot object detection via contrastive proposal encoding[C]// Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. Paris, France: IEEE, 2021: 7352-7362.
[21]	LI B H, YANG B Y, LIU C, et al. Beyond max-margin: class margin equilibrium for few-shot object detection[C]// Proceedings of the 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Nashville, TN, US: IEEE, 2021: 7359-7368.
[22]	CHEN L C, PAPANDROU G, KOKKINOS I, et al. DeepLab: semantic image segmentation with deep convolutional nets, atrous convolution, and fully connected CRFs[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2017, 40(4): 834-848. doi: 10.1109/TPAMI.2017.2699184 URL
[23]	EVERINGHAM M, COOL L V, WILLIAMS C K I, et al. The pascal visual object classes(VOC) challenge[J]. International Journal of Computer Vision, 2010, 88(2): 303-338. doi: 10.1007/s11263-009-0275-4 URL
[24]	LIN T Y, MAIRE M, BELONGIE S, et al. Microsoft coco: common objects in context[C]// Proceedings of the 13rd European Conference on Computer Vision. Zurich, Switzerland: Springer, 2014: 740-755.
[25]	RUSSAKOVSKY O, DENG J, SU H, et al. Imagenet large scale visual recognition challenge[J]. International Journal of Computer Vision, 2015, 115(3): 211-252. doi: 10.1007/s11263-015-0816-y URL

算法	新类子集	样本数
算法	新类子集	1	2	3	5	10
	split1	6.6	10.7	12.5	24.8	38.6
YOLO-ft^[7]	split2	12.5	4.2	11.6	16.1	33.9
	split3	13	15.9	15	32.2	38.4
	split1	13.8	19.6	32.8	41.5	45.6
FRCN-ft^[18]	split2	7.9	15.3	26.2	31.6	39.1
	split3	9.8	11.3	19.1	35	45.1
	split1	14.8	15.5	26.7	33.9	47.2
FSRW^[7]	split2	15.7	15.2	22.7	30.1	40.5
	split3	21.3	25.6	28.4	42.8	45.9
	split1	18.9	20.6	30.2	36.8	49.6
MetaDet^[6]	split2	21.8	23.1	27.8	31.7	43
	split3	20.6	23.9	29.4	43.9	44.1
	split1	19.9	25.5	35	45.7	51.5
Meta R-CNN^[18]	split2	10.4	19.4	29.6	34.8	45.4
	split3	14.3	18.2	27.5	41.2	48.1
	split1	39.8	36.1	44.7	55.7	56
TFA^[19]	split2	23.5	26.9	34.1	35.1	39.1
	split3	30.8	34.8	42.8	49.5	49.8
	split1	41.7		51.4	55.2	61.8
MPSR^[13]	split2	24.4		39.2	39.9	47.8
	split3	35.6		42.3	48	49.7
	split1	44.2	43.8	51.4	61.9	63.4
FSCE^[20]	split2	27.3	29.5	43.5	44.2	50.2
	split3	37.2	41.9	47.5	54.6	58.5
	split1	41.5	47.5	50.4	58.2	60.9
CME^[21]	split2	27.2	30.2	41.4	42.5	46.8
	split3	34.3	39.6	45.1	48.3	51.5
	split1	53.6	57.5	61.5	64.1	60.8
DeFRCN^[12]	split2	30.1	38.1	47	53.3	47.9
	split3	48.4	50.9	52.3	54.9	57.4
	split1	55.1	57.4	61.1	64.6	61.5
DeFRCN^★[12]	split2	32.6	39.6	48.1	53.8	49.2
	split3	48.9	51.9	52.3	55.7	59
	split1	41.8	46.7	52.7	59.6	62.3
Meta-FR-CNN^[7]	split2	26.1	33.6	43.8	47.8	50.1
	split3	35.6	42.1	45.8	53.4	52.3
	split1	40.6	51.4	58.0	59.2	63.6
Meta-DETR^[10]	split2	37.0	36.6	43.7	49.1	54.6
	split3	41.6	45.9	52.7	58.9	60.6
	split1	56	59.3	63.1	66	62.6
本文方法	split2	33.3	41	49.4	55.2	49.9
	split3	49.8	53.8	53.9	56.5	59.9

算法	新类子集	样本数
算法	新类子集	1	2	3	5	10
	split1	6.6	10.7	12.5	24.8	38.6
YOLO-ft^[7]	split2	12.5	4.2	11.6	16.1	33.9
	split3	13	15.9	15	32.2	38.4
	split1	13.8	19.6	32.8	41.5	45.6
FRCN-ft^[18]	split2	7.9	15.3	26.2	31.6	39.1
	split3	9.8	11.3	19.1	35	45.1
	split1	14.8	15.5	26.7	33.9	47.2
FSRW^[7]	split2	15.7	15.2	22.7	30.1	40.5
	split3	21.3	25.6	28.4	42.8	45.9
	split1	18.9	20.6	30.2	36.8	49.6
MetaDet^[6]	split2	21.8	23.1	27.8	31.7	43
	split3	20.6	23.9	29.4	43.9	44.1
	split1	19.9	25.5	35	45.7	51.5
Meta R-CNN^[18]	split2	10.4	19.4	29.6	34.8	45.4
	split3	14.3	18.2	27.5	41.2	48.1
	split1	39.8	36.1	44.7	55.7	56
TFA^[19]	split2	23.5	26.9	34.1	35.1	39.1
	split3	30.8	34.8	42.8	49.5	49.8
	split1	41.7		51.4	55.2	61.8
MPSR^[13]	split2	24.4		39.2	39.9	47.8
	split3	35.6		42.3	48	49.7
	split1	44.2	43.8	51.4	61.9	63.4
FSCE^[20]	split2	27.3	29.5	43.5	44.2	50.2
	split3	37.2	41.9	47.5	54.6	58.5
	split1	41.5	47.5	50.4	58.2	60.9
CME^[21]	split2	27.2	30.2	41.4	42.5	46.8
	split3	34.3	39.6	45.1	48.3	51.5
	split1	53.6	57.5	61.5	64.1	60.8
DeFRCN^[12]	split2	30.1	38.1	47	53.3	47.9
	split3	48.4	50.9	52.3	54.9	57.4
	split1	55.1	57.4	61.1	64.6	61.5
DeFRCN^★[12]	split2	32.6	39.6	48.1	53.8	49.2
	split3	48.9	51.9	52.3	55.7	59
	split1	41.8	46.7	52.7	59.6	62.3
Meta-FR-CNN^[7]	split2	26.1	33.6	43.8	47.8	50.1
	split3	35.6	42.1	45.8	53.4	52.3
	split1	40.6	51.4	58.0	59.2	63.6
Meta-DETR^[10]	split2	37.0	36.6	43.7	49.1	54.6
	split3	41.6	45.9	52.7	58.9	60.6
	split1	56	59.3	63.1	66	62.6
本文方法	split2	33.3	41	49.4	55.2	49.9
	split3	49.8	53.8	53.9	56.5	59.9

算法	样本数
算法	10	30
FRCN-ft^[18]	6.5	11.1
FSRW^[7]	5.6	9.1
MetaDet^[6]	7.1	11.3
Meta R-CNN^[18]	8.7	12.4
TFA^[19]	10.0	13.7
MPSR^[13]	9.8	14.1
FSDetView^[18]	12.5	14.7
FSCE^[20]	11.9	16.4
CME^[21]	15.1	16.9
DeFRCN^[12]	18.5	22.6
DeFRCN^★[12]	18.4	22.7
本文方法	19.1	23.3

算法	样本数
算法	10	30
FRCN-ft^[18]	6.5	11.1
FSRW^[7]	5.6	9.1
MetaDet^[6]	7.1	11.3
Meta R-CNN^[18]	8.7	12.4
TFA^[19]	10.0	13.7
MPSR^[13]	9.8	14.1
FSDetView^[18]	12.5	14.7
FSCE^[20]	11.9	16.4
CME^[21]	15.1	16.9
DeFRCN^[12]	18.5	22.6
DeFRCN^★[12]	18.4	22.7
本文方法	19.1	23.3

DeFRCN	HDC	SFDF	样本数
DeFRCN	HDC	SFDF	1	2	3	5	10
√			55.1	57.4	61.1	64.6	61.5
√	√		56	58.3	62	65.3	62.3
√		√	55.1	58.7	62.6	65.4	62
√	√	√	56	59.3	63.1	66	62.6