基于多层特征嵌入的单目标跟踪算法

doi:10.12382/bgxb.2024.0062

摘要/Abstract

摘要：

针对现有视觉目标跟踪方法仅使用初始帧的目标单一外观特征,导致当背景复杂或外观发生剧烈变化时跟踪失效的问题,提出一种基于多层特征嵌入的单目标跟踪算法。增强目标的外观区分度,使用稀疏内嵌注意力机制编码器,嵌入具有高实例区分度的外观特征;采用类间特征聚合编码器嵌入目标的类别信息,在外观发生变化时保持类内的紧凑性;同时将预测的历史帧跟踪框坐标转化为目标运动轨迹特征嵌入,为算法提供高置信度的时间上下文特征。研究结果表明:所提算法在OTB100基准测试中成功率和准确率分别达到71.4%和92.6%,在GOT-10K、LaSOT、TrackingNet共3个大规模公开数据上取得了鲁棒的效果,成功率分别达到64.9%、72.0%和78.7%;基于多层特征嵌入的单目标跟踪算法有效地克服了现有算法的局限,具有较好的准确性和鲁棒性。

关键词: 目标跟踪, 稀疏内嵌注意力机制编码器, 类间特征聚合编码器, 运动特征嵌入

Abstract:

The existing visual object tracking methods often face the challenge of tracking failure when dealing with complex backgrounds or drastic appearance changes due to relying solely on the initial frame’s single appearance feature.To address this issue,a single object tracking algorithm based on multilayer feature embedding is proposed.To enhance the discriminability of the target’s appearance,a sparse embedded attention encoder is employed to embed the appearance features with high instance distinctiveness.Additionally,an category feature aggregation encoder is utilized to embed the target’s category information,maintaining the compactness within the class when the appearance changes occur.Simultaneously,the predicted historical frame tracking box coordinates are transformed into the embedded target motion trajectory features,providing the tracking algorithm with high-confidence temporal context features.Experimental results demonstrate that the proposed algorithm achieves a success rate of 71.4% and an accuracy of 92.6% in the OTB100 benchmark test.Moreover,it exhibits robust performance on three large-scale public datasets,namely GOT-10K,LaSOT,and TrackingNet,with success rates reaching 64.9%,72.0%,and 78.7%,respectively.The proposed algorithm effectively overcomes the limitations of the existing tracking algorithms,and has the enhanced tracking accuracy and robustness.

Key words: object tracking, sparse embedded attention encoder, category feature aggregation encoder, motion feature embedding

中图分类号:

TP391

才华, 周鸿策, 付强, 赵义武. 基于多层特征嵌入的单目标跟踪算法[J]. 兵工学报, 2025, 46(3): 240062-.

CAI Hua, ZHOU Hongce, FU Qiang, ZHAO Yiwu. Single Object Tracking Algorithm Based on Multilayer Feature Embedding[J]. Acta Armamentarii, 2025, 46(3): 240062-.

图/表 14

参考文献 9

[1]	陆康亮, 薛俊, 陶重犇. 融合空间掩膜预测与点云投影的多目标跟踪[J]. 光电工程, 2022, 49(9):49-63.
	LU K L, XUE J, TAO C B. Multi target tracking based on spatial mask prediction and point cloud projection[J]. Opto-Electronic Engineering, 2022, 49(9):49-63. (in Chinese)
[2]	才华, 寇婷婷, 杨依宁, 等. 基于轨迹优化的三维车辆多目标跟踪[J/OL]. 吉林大学学报(工学版), 2023(2023-05-09)[2024-01-11].https://doi.org/10.13229/j.cnki.Jdxbgxb.20221373.
	CAI H, KOU T T, YANG Y N, et al. Three-dimensional vehicle multi-target tracking based on trajectory optimization[J/OL]. Journal of Jilin University (Engineering Edition), 2023(2023-05-09)[2024-01-11].https://doi.org/10.13229/j.cnki.Jdxbgxb.20221373. (in Chinese)
[3]	姜珊, 底晓强, 韩成. 融合时空特性的孪生网络视觉跟踪[J]. 兵工学报, 2021, 42(9):1940-1950. doi: 10.3969/j.issn.1000-1093.2021.09.015
	JIANG S, DI X Q, HAN C. Siamese network for visual tracking with temporal-spatial property[J]. Acta Armamentarii, 2021, 42(9):1940-1950. (in Chinese) doi: 10.3969/j.issn.1000-1093.2021.09.015
[4]	梁杰, 李磊, 任君, 等. 基于深度学习的红外图像遮挡干扰检测方法[J]. 兵工学报, 2019, 40(7):1401-1410. doi: 10.3969/j.issn.1000-1093.2019.07.009
	LIANG J, LI L, REN J, et al. Infrared image occlusion interference detection method based on deep learning[J]. Acta Armamentarii, 2019, 40(7):1401-1410. (in Chinese) doi: 10.3969/j.issn.1000-1093.2019.07.009
[5]	陈瑞东, 秦会斌. 多特征融合与Kalman滤波的CAMShift跟踪算法[J]. 计算机仿真, 2024, 41(3):200-205,236.
	CHEN R D, QIN H B. CAMShift tracking algorithm based on multi-feature fusion and Kalman filter[J]. Computer Simulation, 2024, 41(3):200-205,236. (in Chinese)
[10]	孙家伟. 基于域不变投影的全天候目标跟踪方法研究[D]. 南京: 南京邮电大学, 2023.
	SUN J W. Research on all-day target tracking algorithm based on domain invariant projection[D]. Nanjing: Nanjing University of Posts and Telecommunications, 2023. (in Chinese)
[11]	ZHANG Z P, PENG H W. Deeper and wider Siamese networks for real-time visual tracking[C]// Proceedings of the 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Long Beach,CA,US: IEEE, 2019:4591-4600.
[12]	CHEN Z D, ZHONG B N, LI G R, et al. Siamese box adaptive network for visual tracking[C]// Proceedings of the 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Seattle,WA,US: IEEE, 2020:6668-6677.
[13]	GUO D Y, WANG J, CUI Y, et al. SiamCAR:Siamese fully convolutional classification and regression for visual tracking[C]// Proceedings of the 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Seattle,WA,US: IEEE, 2020:6269-6277.
[14]	HE A F, LUO C, TIAN X M, et al. A twofold siamese network for real-time object tracking[C]// Proceedings of the 2018 IEEE Conference on Computer Vision and Pattern Recognition. Seattle,WA,US: IEEE, 2018:4834-4843.
[15]	ZHU Z, WANG Q, LI B, et al. Distractor-aware siamese networks for visual object tracking[C]// Proceedings of the 15th European Conference on Computer Vision. Munich,Germany: Springer, 2018:101-117.
[16]	PI Z X, WAN W T, SUN C, et al. Hierarchical feature embedding for visual tracking[C]// Proceedings of the 17th European Conference on Computer Vision. Tel Aviv,Israel: Springer, 2022:428-445.
[17]	才华, 王学伟, 付强, 等. 基于动态模板更新的孪生网络目标跟踪算法[J]. 吉林大学学报(工学版), 2022, 52(5):1106-1116.
	CAI H, WANG X W, FU Q, et al. Siamese network target tracking algorithm based on dynamic template updating[J]. Journal of Jilin University (Engineering Edition), 2022, 52(5):1106-1116. (in Chinese)
[18]	FU Z H, LIU Q J, FU Z H, et al. Stmtrack:template-free visual tracking with space-time memory networks[C]// Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. Nashville,TN,US: IEEE, 2021:13774-13783.
[19]	GAO S Y, ZHOU C L, MA C, et al. AiATrack:attention in attention for transformer visual tracking[C]// Proceedings of the 17th European Conference on Computer Vision. Tel Aviv,Israel: Springer, 2022:146-164.
[20]	刘广文, 谢欣月, 付强, 等. 基于时空模板焦点注意的Transformer目标跟踪算法[J/OL]. 吉林大学学报(工学版), 2024(2023-10-18)[2024-01-11].https://doi.org/10.13229/j.cnki.Jdxbgxb.20230544.
	LIU G W, XIE X Y, FU Q, et al. Spatiotemporal transformer with template attention for target tracking[J]. Journal of Jilin University (Engineering Edition), 2024(2023-10-18)[2024-01-11].https://doi.org/10.13229/j.cnki.Jdxbgxb.20230544. (in Chinese)
[21]	CHEN X, YAN B, ZHU J W, et al. Transformer tracking[C]// Proceedings of the 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Nashville,TN,US: IEEE, 2021:8126-8135.
[22]	WANG N, ZHOU W G, WANG J, et al. Transformer meets tracker:exploiting temporal context for robust visual tracking[C]// Proceedings of the 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Nashville,TN,US: IEEE, 2021:1571-1580.
[23]	YAN B, PENG H W, FU J L, et al. Learning spatio-temporal transformer for visual tracking[C]// Proceedings of the 2021 IEEE/CVF International Conference on Computer Vision. Nashville, TN,US: IEEE, 2021:10448-10457.
[24]	FU Z H, FU Z H, LIU Q J, et al. SparseTT:visual tracking with sparse transformers:arXiv:2205.03776[R/OL]. Ithaca,NY,US:Cornell University, 2022(2022-05-08).
[25]	GAO S Y, ZHOU C L, ZHANG J. Generalized relation modeling for transformer tracking[C]// Proceedings of the 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Vancouver,BC,Canada: IEEE, 2023:18686-18695.
[26]	LIN T Y, MAIRE M, BELONGIE S, et al. Microsoft coco:common objects in context[C]// Proceedings of the 13rd European Conference on Computer Vision. Zurich,Switzerland: Springer, 2014:740-755.
[27]	HUANG L H, ZHAO X, HUANG K Q. GOT-10k:a large high-diversity benchmark for generic object tracking in the wild[J]. IEEE Trans Pattern Anal Mach Intell, 2019, 43(5):1562-1577.
[28]	FAN H, BAI L X, LIN L T, et al. LaSOT:a high-quality benchmark for large-scale single object tracking[J]. International Journal of Computer Vision, 2020, 129:439-461.
[29]	WU Y, LIM J, YANG M H. Object tracking benchmark[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2015, 37(9):1834-1848. doi: 10.1109/TPAMI.2014.2388226 pmid: 26353130
[30]	MULLER M, BIBI A, GIANCOLA S, et al. TrackingNet:a large-scale dataset and benchmark for object tracking in the wild[C]// Proceedings of the 2018 European Conference on Computer Vision. Munich,Germany: Springer, 2018:300-317.
[6]	阴国华, 齐咏生, 刘利强, 等. 基于Ghost-TiFPN的轻量化快速目标跟踪算法[J]. 兵工学报, 45(5):1703-1716.
	YIN G H, QI Y S, LIU L Q, et al. Lightweight and fast target tracking algorithm based on ghost-TiFPN[J]. Acta Armamentarii, 45(5):1703-1716. (in Chinese)
[7]	BERTINETTO L, VALMADRE J, HENRIQUES J F, et al. Fully-convolutional Siamese networks for object tracking[C]// Proceedings of the European Conference on Computer Vision. Cham,Switzerland: Springer, 2016:850-865.
[8]	LI B, YAN J J, WU W, et al. High performance visual tracking with Siamese region proposal network[C]// Proceedings of the 2018 IEEE Conference on Computer Vision and Pattern Recognition. Salt Lake City,UT,US: IEEE, 2018:8971-8980.
[9]	LI B, WU W, WANG Q, et al. SiamRPN++:evolution of siamese visual tracking with very deep networks[C]// Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. Long Beach,CA,US: IEEE, 2019:4277-4286.

序号	搜索区域	原始特征	实例增强特征	类别聚合特征
1
2
3
4

序号	搜索区域	原始特征	实例增强特征	类别聚合特征
1
2
3
4

视频序列	不同算法的跟踪结果
自行车
鸟
盒子
女孩
液体

视频序列	不同算法的跟踪结果
自行车
鸟
盒子
女孩
液体

模型编号	稀疏内嵌注意力编码器	类间特征聚合编码器	运动特征嵌入	成功率	精度
MFET(0)	√			0.671	0.887
MFET(1)	√	√		0.694	0.900
MFET(2)	√		√	0.692	0.897
MFET(3)	√	√	√	0.714	0.926