基于Ghost-TiFPN的轻量化快速目标跟踪算法

doi:10.12382/bgxb.2022.1272

摘要/Abstract

摘要：

针对传统孪生目标跟踪算法体量大、难以在嵌入式设备部署以及其在目标尺度变化大、有相似物干扰等条件下效果不佳的问题,提出一种新的轻量化快速跟踪(Ghost fast Tracking with TiFPN and Retriever, GTtracker)算法。引入Ghost机制,对Resnet网络进行重新设计,构建一种轻量化G-Resnet网络对跟踪目标进行快速特征提取。设计轻量自适应加权融合(Tiny adaptive weighted fusion algorithm Feature Pyramid Network, TiFPN)算法,进一步加强特征信息的融合,解决相似物干扰问题。设计一种轻量化区域回归网络(Ghost Decoupled Net, GDnet),以实现目标分类、交并比(Intersection-over-Union,IoU)计算以及边界框回归,并在跟踪阶段应用一种新的目标寻回器提升算法跟踪的成功率。在OTB100数据集和VOT2020数据集上进行算法验证,并移植算法到嵌入式设备Jetson Xavier NX上进行性能测试。实验结果均表明算法的有效性和优越性,相比经典孪生目标跟踪(SiamCAR)算法,新方法在精度和期望平均重叠率(Expected Average Overlap,EAO)指标均相似的前提下,能够实现更快的运行速度,可在Jetson Xavier NX上实时运行,达到30 帧/s,且能有效解决相似物干扰、尺度变化大等问题。

关键词: 目标跟踪, 孪生网络, 轻量化, 嵌入式设备

Abstract:

In response to the problems that traditional siamese target tracking algorithm is bulky and difficult to deploy in embedded devices, and has poor effect under the conditions of large changes in target scale and similar object interference, a lightweight and fast target tracking algorithm GTtracker is proposed. The Resnet network is redesigned to build a lightweight G-Resnet network by introducing the Ghost mechanism for fast feature extraction of tracked targets. And then the fusion of feature information is further enhanced by designing a lightweight adaptive weighted fusion algorithm TiFPN to solve the problem of similar object interference. After that, a lightweight area regression network GDNet is introduced for target classification, IoU calculation, and bounding box regression, which applies a new target finder in the tracking stage to enhance the success rate of algorithm tracking. Finally, the algorithm is validated on OTB100 dataset and VOT2020 dataset, and ported to Jetson Xavier NX embedded device for performance testing. Experimental results show the effectiveness and superiority of the proposed algorithm, In comparison with classical siamese target tracking algorithm (SiamCAR), the proposed algorithm can achieve faster operation speed and real-time operation on Jetson Xavier NX, reaching 30 frames/s, under the conditions of the same accuracy and EAO metrics, which can effectively solve the problems of similar object interference and large scale variation.

Key words: target tracking, siamese network, lightweight, embedded device

中图分类号:

TP391.41

阴国华, 齐咏生, 刘利强, 苏建强, 张丽杰. 基于Ghost-TiFPN的轻量化快速目标跟踪算法[J]. 兵工学报, 2024, 45(5): 1703-1716.

YIN Guohua, QI Yongsheng, LIU Liqiang, SU Jianqiang, ZHANG Lijie. Lightweight and Fast Target Tracking Algorithm Based on Ghost-TiFPN[J]. Acta Armamentarii, 2024, 45(5): 1703-1716.

图/表 15

图1 GTtracker网络结构

Fig.1 GTtracker network structure

图2 G-Resnet网络改进策略

Fig.2 Improvement strategy of G-Resnet network

图3 网络内部卷积结果可视化

Fig.3 Visualization of convolution results inside the network

表1 改进前后算法对比

Table 1 Algorithm comparison before and after improvement

算法名称	参数量/MB	速度/(帧·s^-1)	准确率/%
ResNet50	102.15	42	86.9
G-Resnet	48.62	89	87.4

图4 TiFPN融合模块

Fig.4 TiFPN fusion module

图5 GhostConv结构图

Fig.5 GhostConv structure

图6 GDnet网络结构

Fig.6 GDnet network structure

图7 目标寻回器工作示意图

Fig.7 Worlking schematic diagram of target finder

图8 GTtracker跟踪流程

Fig.8 GTtracker tracking process

表2 VOT2020数据集测试结果

Table 2 VOT2020 dataset test results

实验	基线算法	模块A	模块B	模块C	模块D	速度/(帧·s^-1)	参数量/Mb	精确度	鲁棒性	EAO
1	√	×	×	×	×	54.8	391	0.438	0.745	0.156
2	√	√	×	×	×	112.5	137	0.471	0.707	0.192
3	√	×	√	×	×	51.3	396	0.474	0.681	0.174
4	√	×	×	√	×	57	374	0.467	0.714	0.169
5	√	×	×	×	√	49	391	0.445	0.652	0.171
6	√	√	√	×	×	102	142	0.493	0.698	0.198
7	√	√	√	√	×	104	125	0.512	0.632	0.213
8	√	√	√	√	√	93	125	0.527	0.504	0.235

图9 OTB100数据集测试结果

Fig.9 OTB100 dataset test results

图10 基于OTB100数据集不同属性序列的实验结果

Fig.10 Experimental results based on different attribute sequences of OTB100 dataset

表3 基于VOT2020数据集的实验结果

Table 3 Experimental results based on VOT2020 dataset

指标	算法
指标	SiamFC	CFNet	SRDCF	SiamRPN	DaSiamRPN	Ocean	SiamCAR	本文算法
EAO	0.188	0.182	0.256	0.383	0.326	0.467	0.482	0.485
精确度	0.503	0.470	0.491	0.586	0.569	0.592	0.610	0.609
鲁棒性	0.585	0.543	0.356	0.276	0.337	0.169	0.150	0.156
运行速度	72	75	8	38	59	71	51.4	86

图11 OTB100数据集部分序列可视化结果

Fig.11 Visualization results of partial sequences in OTB100 dataset

图12 嵌入式设备上算法结果示意图

Fig.12 Schematic diagram of algorithm results on embedded devices

参考文献 21

[1]	刘芳, 杨安喆, 吴志威. 基于自适应Siamese网络的无人机目标跟踪算法[J]. 航空学报, 2020, 41(1): 243-255.
	LIU F, YANG A Z, WU Z W. Adaptive Siamese network based UAV target tracking algorithm[J]. Acta Aeronautica et Sinica, 2020, 41(1):243-255. (in Chinese)
[2]	崔洲涓, 安军社, 张羽丰, 等. 面向无人机的轻量级Siamese注意力网络目标跟踪[J]. 光学学报, 2020, 40(19): 132-144.
	CUI Z J, AN J S, ZHANG Y F, et al. Lightweight Siamese attention network target tracking for UAVs[J]. Journal of Optics. 2020, 40(19): 132-144. (in Chinese)
[3]	BERTINETTO L, VALMADRE J, HENRIQUES J F, et al. Fully-convolutional Siamese networks for object tracking[C]// Proceedings of European Conference on Computer Vision. Cham, Germany: Springer, 2016: 850-865.
[4]	LI B, YAN J J, WU W, et al. High performance visual tracking with siamese region proposal network[C]// Proceedings of IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscognition, NJ, US: IEEE Press,2018: 8971-8980.
[5]	LI B, WU W, WANG Q, et al. SiamRPN plus plus: evolution of Siamese visual tracking with very deep networks[C]// Proceedings of IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway, NJ, US: IEEE Press, 2019: 4282-4291.
[6]	CHEN Z D, ZHONG B N, LI G R, et al. Siamese box adaptive network for visual tracking[C]//Proceedings of the 2020 IEEE Conference on Computer Vision and Pattern Recognition. Seattle,WA, US:IEEE, 2020: 6667-6676.
[7]	GUO D Y, WANG J, CUI Y, et al. SiamCAR: siamese fully convolutional classification and regression for visual tracking[C]// Proceedings of 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Seattle, WA, US: IEEE, 2020: 6268-6276.
[8]	姜珊, 底晓强, 韩成. 融合时空特性的孪生网络视觉跟踪[J]. 兵工学报, 2021, 42(9):1940-1950. doi: 10.3969/j.issn.1000-1093.2021.09.015
	JIANG S, DI X Q, HAN C. Visual tracking of twin network based on spatiotemporal characteristics[J]. Acta Armamentarii, 2021, 42(9):1940-1950. (in Chinese)
[9]	周士琪, 王耀南, 钟杭. 融合视觉显著性再检测的孪生网络无人机目标跟踪算法[J]. 智能系统学报, 2021, 16(3): 584-594.
	ZHOU S Q, WANG Y N, ZHANG H. A twin network UAV target tracking algorithm incorporating visual saliency re-detection[J]. Journal of Intelligent Systems, 2021, 16(3): 584-594. (in Chinese)
[10]	SANDLER M, HOWARD A, ZHU M, et al. MobileNetV2: inverted residuals and linear bottlenecks[C]// Proceedings of 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.New York,NY, US: IEEE, 2018:4510-4520.
[11]	MA N, ZHANG X, ZHENG H T, et al. ShuffleNet V2: Practical Guidelines for Efficient CNN Architecture Design[C]// Proceedings of the European conference on computer vision. New York,NY, US: IEEE,2018: 116-131.
[12]	TAN M X, LE Q. Efficientnetv2: smaller models and faster training[C]//Proceedings of International Conference on Machine Learning. New York,NY, US:PMLR,2021: 10096-10106.
[13]	HE K M, ZHANG X Y, REN S Q, et al. Deep residual learning for image recognition[C]// Proceedings of 2016 IEEE Conference on Computer Vision and Pattern Recognition. New York,NY, US: IEEE, 2016: 770-778.
[14]	HAN K, WANG Y H, TIAN Q, et al. GhostNet: more features from cheap operations[C]//Proceedings of IEEE/CVF Conference on Computer Vision and Pattern Recognition,Electr Network.New York,NY, US:IEEE, 2020: 1577-1586.
[15]	VASWANI A, SHAZEER N, PARMAR N, et al. Attention is all you need[C]// Proceedings of the 31st International Conference on Neural Information Processing Systems New York,NY, US: Curran Associates Inc., 2017: 6000-6010.
[16]	LIU Z, MAO H, WU C Y, et al. A convnet for the 2020s[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. Washington,D.C., US:IEEE, 2022: 11976-11986.
[17]	ZHU Z, WANG Q, LI B, et al. Distractor-aware siamese networks for visual object tracking[C]// Proceedings of European Conference on Computer Vision.Munich, Germany: Springer, 2018: 103-119.
[18]	ZHANG Z P, PENG H W. Deeper and wider siamese networks for real-time visual tracking[C]// Proceedings of the 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition. New York,NY, US: IEEE, 2019: 4586-4595.
[19]	SHEN Z L, DAI Y C, RAO Z B. CFNet: cascade and fused cost volume for robust stereo matching[C]//Proceedings of 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition.New York,NY, US:IEEE, 2021:13901-13910.
[20]	DANELLJAN M, HAGER G, SHAHBAZ KHAN F, et al. Learning spatially regularized correlation filters for visual tracking[C]//Proceedings of the IEEE International Conference on Computer Vision.Washington,D.C., US:IEEE, 2015:4310-4318.
[21]	ZHANG Z P, PENG H W, FU J L, et al. Ocean:object-aware anchor-free tracking[C]// Proceedings of European Conference on Computer Vision.Glasgow, UK: Springer, 2020: 771-787.