Few-shot Object Detection Based on Convolution Network and Attention Mechanism

doi:10.12382/bgxb.2022.1167

Abstract

Abstract:

Few-shot object detection(FSOD) aims to enable the detector with a small number of training samples. Typical FSOD method takes Faster R-CNN as the basic detection framework, and uses a convolutional neural network to extract the image features. However, the pooling operation used in the convolutional neural network inevitably leads to the loss of image information. Therefore, a hybrid dilated convolution is introduced into the backbone network to ensure a larger receptive field and minimize the loss of image information. A support feature dynamic fusion module is proposed to further utilize the given support data in k-shot setting, which adaptively fuses the support features with the weight of the correlation between each support feature and query feature to obtain stronger support clues. Experimental results show that rhe proposed method achieves good and state-of-the-art FSOD performance on public Pascal VOC and MS-COCO datasets.

Key words: few-shot object detection, hybrid dilated convolution, support feature dynamic fusion

CLC Number:

TP183

GUO Yonghong, NIU Haitao, SHI Chao, GUO Cheng. Few-shot Object Detection Based on Convolution Network and Attention Mechanism[J]. Acta Armamentarii, 2023, 44(11): 3508-3515.

Figures/Tables 6

References 25

[1]	王芳, 王海晏, 寇添, 等. 多维特征点空间的红外弱小目标检测方法[J]. 应用光学, 2020, 41(6):1268-1276. doi: 10.5768/JAO202041.0606002
	WANG F, WANG H Y, KOU T, et al. Multi-dimensional feature point space infrared dim target detection method[J]. Journal of Applied Optics, 2020, 41(6):1268-1276. (in Chinese) doi: 10.5768/JAO202041.0606002
[2]	牛畅, 尹奎英, 黄银和. 无人机对地目标自动检测与跟踪技术[J]. 应用光学, 2020, 41(6): 1153-1160. doi: 10.5768/JAO202041.0601003
	NIU C, YIN K Y, HUANG Y H. Automatic target detecting and tracking technology based on UAV ground target images[J]. Journal of Applied Optics, 2020, 41(6): 1153-1160. (in Chinese) doi: 10.5768/JAO202041.0601003
[3]	杜鹏飞, 李小勇, 高雅丽. 多模态视觉语言表征学习研究综述[J]. 软件学报, 2021, 32(2): 327-348.
	DU P F, LI X Y, GAO Y L. Survey on multimodal visual language representation learning[J]. Journal of Software, 2021, 32(2): 327-348. (in Chinese)
[4]	KANG B Y, LIU Z, WANG X, et al. Few-shot object detection via feature reweighting[C]// Proceedings of the IEEE/CVF International Conference on Computer Vision. Changshu, China: IEEE, 2019: 8420-8429.
[5]	梁杰, 任君, 李磊, 等. 基于典型几何形状精确回归的机场跑道检测方法[J]. 兵工学报, 2020, 41(10): 2045-2054.
	LIANG J, REN J, LI L, et al. Airport runway detection algorithm based on accurate regression of typical geometric shapes[J]. Acta Armamentarii, 2020, 41(10): 2045-2054. (in Chinese)
[6]	WANG Y X, RAMANAN D, HEBERT M. Meta-learning to detect rare objects[C]// Proceedings of the 2019 IEEE/CVF International Conference on Computer Vision. Seoul, Korea: IEEE, 2019: 9924-9933.
[7]	ZHANG G J, LUO Z P, CUI K W, et al. Meta-DETR: image-level few-shot detection with inter-class correlation Exploitation[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2023, 45(11): 12832-12843.2, DOI: 10.1109/TPAMI.2022.3195735.
[8]	VILALTA R, DRISSI Y. A perspective view and survey of meta-learning[J]. Artificial Intelligence Review, 2002, 18(2):77-95. doi: 10.1023/A:1019956318069 URL
[9]	WEISS K, KHOSHGOFTAAR T M, WANG D D. A survey of transfer learning[J]. Journal of Big Data, 2016, 3(1):1-40. doi: 10.1186/s40537-015-0036-x URL
[10]	HAN G X, HUANG S Y, MA J W J, et al. Meta faster R-CNN:towards accurate few-shot object detection with attentive feature alignment[C]// Proceedings of the 36th AAAI Conference on Artificial Intelligence. Reston, VA,US:AIAA, 2022, 36(1): 780-789.
[11]	REN S Q, HE K M, GIRSHICK R, et al. Towards real-time object detection with region proposal networks[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2016, 39(6): 1137-1149. doi: 10.1109/TPAMI.2016.2577031 URL
[12]	QIAO L M, ZHAO Y X, LI Z Y, et al. DeFRCN:Decoupled faster R-CNN for few-shot object detection[C]// Proceedings of the IEEE/CVF International Conference on Computer Vision. Paris, France: IEEE, 2021: 8681-8690.
[13]	WANG P Q, CHEN P F, YUAN Y, et al. Understanding convolution for semantic segmentation[C]// Proceedings of 2018 IEEE Winter Conference on Applications of Computer Vision. Lake Tahoe, NV, US: IEEE, 2018: 1451-1460.
[14]	LIU W, ANGUELOV D, ERHAN D, et al. SSD:single shot multibox detector[C]// Proceedings of the 14th European Conference on Computer Vision. Amsterdam,the Netherlands: Springer, 2016: 21-37.
[15]	REDMON J, DIVVALA S, GIRSHICK R, et al. You only look once:unified, real-time object detection[C]// Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Lyon, France: IEEE, 2016: 779-788.
[16]	GIRSHICK R. Fast R-CNN[C]// Proceedings of the 2015 IEEE International Conference on Computer Vision. Santiago,Chile:IEEE, 2015:1440-1448.
[17]	WU J X, LIU S T, HUANG D, et al. Multi-scale positive sample refinement for few-shot object detection[C]// Proceedings of the 16th European Conference on Computer Vision. Glasgow, UK: Springer, 2020: 456-472.
[18]	YAN X P, CHEN Z, XU A, et al. Towards general solver for instance-level low-shot learning[C]// Proceedings of the 2019 IEEE/CVF International Conference on Computer Vision. Seoul, Korea: IEEE, 2019: 9577-9586.
[19]	WANG X, HUANG T E, DARRELL T, et al. Frustratingly simple few-shot object detection:arXiv:2003.06957[R]. Ithaca, NY, US: Cornell University, 2020:2003.06957.
[20]	SUN B, LI B H, CAI S C, et al. Few-shot object detection via contrastive proposal encoding[C]// Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. Paris, France: IEEE, 2021: 7352-7362.
[21]	LI B H, YANG B Y, LIU C, et al. Beyond max-margin: class margin equilibrium for few-shot object detection[C]// Proceedings of the 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Nashville, TN, US: IEEE, 2021: 7359-7368.
[22]	CHEN L C, PAPANDROU G, KOKKINOS I, et al. DeepLab: semantic image segmentation with deep convolutional nets, atrous convolution, and fully connected CRFs[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2017, 40(4): 834-848. doi: 10.1109/TPAMI.2017.2699184 URL
[23]	EVERINGHAM M, COOL L V, WILLIAMS C K I, et al. The pascal visual object classes(VOC) challenge[J]. International Journal of Computer Vision, 2010, 88(2): 303-338. doi: 10.1007/s11263-009-0275-4 URL
[24]	LIN T Y, MAIRE M, BELONGIE S, et al. Microsoft coco: common objects in context[C]// Proceedings of the 13rd European Conference on Computer Vision. Zurich, Switzerland: Springer, 2014: 740-755.
[25]	RUSSAKOVSKY O, DENG J, SU H, et al. Imagenet large scale visual recognition challenge[J]. International Journal of Computer Vision, 2015, 115(3): 211-252. doi: 10.1007/s11263-015-0816-y URL

算法	新类子集	样本数
算法	新类子集	1	2	3	5	10
	split1	6.6	10.7	12.5	24.8	38.6
YOLO-ft^[7]	split2	12.5	4.2	11.6	16.1	33.9
	split3	13	15.9	15	32.2	38.4
	split1	13.8	19.6	32.8	41.5	45.6
FRCN-ft^[18]	split2	7.9	15.3	26.2	31.6	39.1
	split3	9.8	11.3	19.1	35	45.1
	split1	14.8	15.5	26.7	33.9	47.2
FSRW^[7]	split2	15.7	15.2	22.7	30.1	40.5
	split3	21.3	25.6	28.4	42.8	45.9
	split1	18.9	20.6	30.2	36.8	49.6
MetaDet^[6]	split2	21.8	23.1	27.8	31.7	43
	split3	20.6	23.9	29.4	43.9	44.1
	split1	19.9	25.5	35	45.7	51.5
Meta R-CNN^[18]	split2	10.4	19.4	29.6	34.8	45.4
	split3	14.3	18.2	27.5	41.2	48.1
	split1	39.8	36.1	44.7	55.7	56
TFA^[19]	split2	23.5	26.9	34.1	35.1	39.1
	split3	30.8	34.8	42.8	49.5	49.8
	split1	41.7		51.4	55.2	61.8
MPSR^[13]	split2	24.4		39.2	39.9	47.8
	split3	35.6		42.3	48	49.7
	split1	44.2	43.8	51.4	61.9	63.4
FSCE^[20]	split2	27.3	29.5	43.5	44.2	50.2
	split3	37.2	41.9	47.5	54.6	58.5
	split1	41.5	47.5	50.4	58.2	60.9
CME^[21]	split2	27.2	30.2	41.4	42.5	46.8
	split3	34.3	39.6	45.1	48.3	51.5
	split1	53.6	57.5	61.5	64.1	60.8
DeFRCN^[12]	split2	30.1	38.1	47	53.3	47.9
	split3	48.4	50.9	52.3	54.9	57.4
	split1	55.1	57.4	61.1	64.6	61.5
DeFRCN^★[12]	split2	32.6	39.6	48.1	53.8	49.2
	split3	48.9	51.9	52.3	55.7	59
	split1	41.8	46.7	52.7	59.6	62.3
Meta-FR-CNN^[7]	split2	26.1	33.6	43.8	47.8	50.1
	split3	35.6	42.1	45.8	53.4	52.3
	split1	40.6	51.4	58.0	59.2	63.6
Meta-DETR^[10]	split2	37.0	36.6	43.7	49.1	54.6
	split3	41.6	45.9	52.7	58.9	60.6
	split1	56	59.3	63.1	66	62.6
本文方法	split2	33.3	41	49.4	55.2	49.9
	split3	49.8	53.8	53.9	56.5	59.9

算法	新类子集	样本数
算法	新类子集	1	2	3	5	10
	split1	6.6	10.7	12.5	24.8	38.6
YOLO-ft^[7]	split2	12.5	4.2	11.6	16.1	33.9
	split3	13	15.9	15	32.2	38.4
	split1	13.8	19.6	32.8	41.5	45.6
FRCN-ft^[18]	split2	7.9	15.3	26.2	31.6	39.1
	split3	9.8	11.3	19.1	35	45.1
	split1	14.8	15.5	26.7	33.9	47.2
FSRW^[7]	split2	15.7	15.2	22.7	30.1	40.5
	split3	21.3	25.6	28.4	42.8	45.9
	split1	18.9	20.6	30.2	36.8	49.6
MetaDet^[6]	split2	21.8	23.1	27.8	31.7	43
	split3	20.6	23.9	29.4	43.9	44.1
	split1	19.9	25.5	35	45.7	51.5
Meta R-CNN^[18]	split2	10.4	19.4	29.6	34.8	45.4
	split3	14.3	18.2	27.5	41.2	48.1
	split1	39.8	36.1	44.7	55.7	56
TFA^[19]	split2	23.5	26.9	34.1	35.1	39.1
	split3	30.8	34.8	42.8	49.5	49.8
	split1	41.7		51.4	55.2	61.8
MPSR^[13]	split2	24.4		39.2	39.9	47.8
	split3	35.6		42.3	48	49.7
	split1	44.2	43.8	51.4	61.9	63.4
FSCE^[20]	split2	27.3	29.5	43.5	44.2	50.2
	split3	37.2	41.9	47.5	54.6	58.5
	split1	41.5	47.5	50.4	58.2	60.9
CME^[21]	split2	27.2	30.2	41.4	42.5	46.8
	split3	34.3	39.6	45.1	48.3	51.5
	split1	53.6	57.5	61.5	64.1	60.8
DeFRCN^[12]	split2	30.1	38.1	47	53.3	47.9
	split3	48.4	50.9	52.3	54.9	57.4
	split1	55.1	57.4	61.1	64.6	61.5
DeFRCN^★[12]	split2	32.6	39.6	48.1	53.8	49.2
	split3	48.9	51.9	52.3	55.7	59
	split1	41.8	46.7	52.7	59.6	62.3
Meta-FR-CNN^[7]	split2	26.1	33.6	43.8	47.8	50.1
	split3	35.6	42.1	45.8	53.4	52.3
	split1	40.6	51.4	58.0	59.2	63.6
Meta-DETR^[10]	split2	37.0	36.6	43.7	49.1	54.6
	split3	41.6	45.9	52.7	58.9	60.6
	split1	56	59.3	63.1	66	62.6
本文方法	split2	33.3	41	49.4	55.2	49.9
	split3	49.8	53.8	53.9	56.5	59.9

算法	样本数
算法	10	30
FRCN-ft^[18]	6.5	11.1
FSRW^[7]	5.6	9.1
MetaDet^[6]	7.1	11.3
Meta R-CNN^[18]	8.7	12.4
TFA^[19]	10.0	13.7
MPSR^[13]	9.8	14.1
FSDetView^[18]	12.5	14.7
FSCE^[20]	11.9	16.4
CME^[21]	15.1	16.9
DeFRCN^[12]	18.5	22.6
DeFRCN^★[12]	18.4	22.7
本文方法	19.1	23.3

算法	样本数
算法	10	30
FRCN-ft^[18]	6.5	11.1
FSRW^[7]	5.6	9.1
MetaDet^[6]	7.1	11.3
Meta R-CNN^[18]	8.7	12.4
TFA^[19]	10.0	13.7
MPSR^[13]	9.8	14.1
FSDetView^[18]	12.5	14.7
FSCE^[20]	11.9	16.4
CME^[21]	15.1	16.9
DeFRCN^[12]	18.5	22.6
DeFRCN^★[12]	18.4	22.7
本文方法	19.1	23.3

DeFRCN	HDC	SFDF	样本数
DeFRCN	HDC	SFDF	1	2	3	5	10
√			55.1	57.4	61.1	64.6	61.5
√	√		56	58.3	62	65.3	62.3
√		√	55.1	58.7	62.6	65.4	62
√	√	√	56	59.3	63.1	66	62.6