基于空间偏振混叠特性的红外偏振视频目标跟踪

doi:10.12382/bgxb.2023.0600

摘要/Abstract

摘要：

基于分焦平面的红外偏振摄像仪体积小、实时性好、精确度高,并能有效应对复杂天候与红外伪装等干扰,但输出的偏振马赛克视频需要去马赛克并计算偏振参数,这种耗时较长且存在误差的处理方式限制了该设备在实时目标跟踪领域的应用。为解决这一问题,分析了红外偏振马赛克图像中空间信息与偏振信息的混叠特性,提出基于空间偏振混叠梯度的马赛克图像空间偏振梯度方向直方图特征。特征利用偏振通道之间的空间相关性,设计空间梯度算子获取图像的空间信息,并利用偏振伪全色图像与偏振通道之间的关联性设计偏振梯度算子,结合图像空间信息与偏振信息构建目标特征。将该特征应用于AutoTrack目标跟踪方法,实现了基于空间-偏振混叠信息的实时红外偏振视频目标跟踪。在实拍的红外偏振马赛克视频数据上的实验结果表明,该特征提取方法可提高红外偏振图像的目标跟踪实时性和准确性,并有效利用马赛克图像的偏振信息。

关键词: 偏振图像, 特征提取, 目标跟踪, 空间偏振混叠梯度

Abstract:

The division-of-focal-plane (DoFP)-based Infrared polarization imager has a strong anti-interference against complex weather and infrared camouflage while having the advantages of small size, batter real-time and high accuracy as well. However, the polarization mosaic image obtained by the imager need to be demosaiced and the polarization parameters should be calculated, which are time-consuming and contains inevitable errors, thus limiting the application of the imager in real-time object tracking. To solve this problem, the correlation between the spatial and polarization field is studied, and an object tracking method based on the designed histograms of spatial-polarization mixture gradient (HSPMG) feature extraction method is proposed. A spatial gradient operator is designed to obtain the spatial information of an image based on the spatial correlation between the polarization channels, and a polarized gradient operator is designed based on the polarization pseudo-panchromatic image (PPPI) and the spatial correlation between the polarization channels. Then the HSPMG feature is constructed based on the spatial and polarization information. To implement tracking on the infrared polarization mosaic image based on the spatial-polarization aliasing information, the HSPMG feature is applied to the AutoTrack object tracking method. Experiences on the infrared polarization mosaic dataset demonstrate that the feature extraction method could enhance the accuracy and speed of object tracking result.

Key words: polarization image, feature extraction, object tracking, spatial-polarization mixture gradient

中图分类号:

TN219

乔新博, 赵永强, 张景程. 基于空间偏振混叠特性的红外偏振视频目标跟踪[J]. 兵工学报, 2024, 45(9): 3274-3287.

QIAO Xinbo, ZHAO Yongqiang, ZHANG Jingcheng. Object Tracking Based on Spatial-polarization Aliasing for Infrared Polarization Video[J]. Acta Armamentarii, 2024, 45(9): 3274-3287.

图/表 12

参考文献 42

[1]	李博, 王博, 韩京冶, 等. 基于车载计算机的红外图像移动目标检测[J]. 兵工学报, 2022, 43(增刊1): 66-73.
	LI B, WANG B, HAN J Y, et al. Infrared image moving object detection technology based on onboard computer[J]. Acta Armamentarii, 2022, 43(S1): 66-73. (in Chinese) doi: 10.12382/bgxb.2022.A012
[2]	罗海波, 史泽林. 红外成像制导技术发展现状与展望[J]. 红外与激光工程, 2009, 38(4): 565-573.
	LUO H B, SHI Z L. Status and prospect of infrared imaging guidance technology[J]. Infrared and Laser Engineering, 2009, 38(4): 565-573. (in Chinese)
[3]	LI X M, HUANG Q C. Target detection for infrared polarization image in the background of desert[C]//Proceedings of the 2017 IEEE 9th International Conference on Communication Software and Networks.Guangzhou, China: IEEE, 2017:1147-1151.
[4]	方浩, 李艾华, 潘玉龙, 等. 面向视觉相似的红外伪装仿真效果评估[J]. 兵工学报, 2017, 38(2):351-357. doi: 10.3969/j.issn.1000-1093.2017.02.020
	FANG H, LI A H, PAN Y L, et al. The effect evaluation of infrared camouflage simulation system based on visual similarity[J]. Acta Armamentarii, 2017, 38(2):351-357. (in Chinese) doi: 10.3969/j.issn.1000-1093.2017.02.020
[5]	王红培, 栾光琦, 田超, 等. 天气条件对车载红外辅助驾驶性能的影响[J]. 红外与激光工程, 2020, 49(7): 84-90.
	WANG H P, LUAN G Q, TIAN C, et al. Influences of weather conditions on vehicular infrared assistant driving performance[J]. Infrared and Laser Engineering, 2020, 49(7): 84-90. (in Chinese)
[6]	TYO J S, GOLDSTEIN D L, CHENAULT D B, et al. Review of passive imaging polarimetry for remote sensing applications[J]. Applied Optics, 2006, 45(22): 5453-5469. pmid: 16855644
[7]	FU Q, JIANG H L, DUAN J, et al. Target detection technology based on polarization imaging in the complex environment[C]//Proceedings of International Symposium on Photoelectronic Detection and Imaging 2013:Laser Sensing and Imaging and Applications. Beijing, China: SPIE, 2013, 8905: 218-226.
[8]	罗海波, 张俊超, 盖兴琴, 等. 偏振成像技术的发展现状与展望(特邀)[J]. 红外与激光工程, 2022, 51(1): 109-118.
	LUO H B, ZHANG J C, GAI X Q, et al. Development status and prospects of polarization imaging technology(Invited)[J]. Infrared and Laser Engineering, 2022, 51(1): 109-118. (in Chinese)
[9]	FARLOW C A, CHENAULT D B, PEZZANITI J L, et al. Imaging polarimeter development and applications[C]//Proceedings of Polarization Analysis and Measurement IV. San Diego, CA, US:SPIE, 2002, 4481: 118-125.
[10]	YIN J K, WANG Y, GUAN B W. Polarized images-based dehazing from the viewpoint of self-guided multi-image features fusion[C]//Proceedings of the 2022 IEEE 24th International Workshop on Multimedia Signal Processing. Shanghai, China:IEEE, 2022:1-6. https://ieeexplore.ieee.org/document/9949160.
[11]	EGAN W G, JOHNSON W R, WHITEHEAD V S. Terrestrial polarization imagery obtained from the space shuttle: characterization and interpretation[J]. Applied Optics, 1991, 30(4): 435-442. doi: 10.1364/AO.30.000435 pmid: 20582011
[12]	李欣锴. 分时偏振成像系统定标方法研究[D]. 长春: 中国科学院长春光学精密机械与物理研究所, 2021.
	LI X K. Calibration method research for time divided polarimetric imaging system[D]. Changchun: Changchun Institute of Optics, Fine Mechanics and Physics Chinese Academy of Sciences, 2021. (in Chinese)
[13]	PEZZANITI J L, CHENAULT D B. A division of aperture MWIR imaging polarimeter[J]. Proceedings of SPIE-The International Society for Optical Engineering, 2005, 44(3): 515-533.
[14]	赵永强, 李宁, 张鹏, 等. 红外偏振感知与智能处理[J]. 红外与激光工程, 2018, 47(11): 1102001-1-1102001-7.
	ZHAO Y Q, LI N, ZHANG P, et al. Infrared polarization perception and intelligent processing[J]. Infrared and Laser Engineering, 2018, 47(11):1102001-1-1102001-7. (in Chinese)
[15]	周建, 周易, 倪歆玥, 等. 偏振集成红外光电探测器研究进展与应用[J]. 光电工程, 2023, 50(5): 67-84.
	ZHOU J, ZHOU Y, NI X Y, et al. Research progress and applications of polarization integrated infrared photodetector[J]. Opto-Electron Eng, 2023, 50(5): 67-84. (in Chinese)
[16]	LI N, ZHAO Y Q, PAN Q, et al. Demosaicing DoFP images using Newton’s polynomial interpolation and polarization difference model[J]. Optics Express, 2019, 27(2):1376-1391.
[17]	GAO S K, GRUEY V. Bilinear and bicubic interpolation methods for division of focal plane polarimeters[J]. Optics Express, 2011, 19(27): 26161-26173. doi: 10.1364/OE.19.026161 pmid: 22274203
[18]	ZENG X L, LUO Y, ZHAO X J, et al. An end-to-end fully-convolutional neural network for division of focal plane sensors to reconstruct S0, DoLP, and AoP[J]. Optics Express, 2019, 27(6): 8566-8577.
[19]	王鑫, 宋永红, 张元林. 基于显著性特征提取的图像描述算法[J]. 自动化学报, 2022, 48(3): 735-746.
	WANG X, SONG Y H, ZHANG Y L. Salient feature extraction mechanism for image captioning[J]. Acta Automatica Sinica, 2022, 48(3): 735-746. (in Chinese)
[20]	孟琭, 杨旭. 目标跟踪算法综述[J]. 自动化学报, 2019, 45(7): 1244-1260.
	MENG L, YANG X. A survey of object tracking algorithms[J]. Acta Automatica Sinica, 2019, 45(7): 1244-1260. (in Chinese)
[21]	BERTINETTO L, VALMADRE J, GOLODETZ S, et al. Staple:complementary learners for real-time tracking[C]//Proceedings of 2016 IEEE Conference on Computer Vision and Pattern Recognition.Las Vegas, NV, US:IEEE, 2016: 1401-1409.
[22]	邓小玲, 倪江群, 李震, 等. 多特征融合的低景深图像前景提取算法[J]. 自动化学报, 2013, 39(6): 846-851.
	DENG X L, NI J Q, LI Z, et al. Foreground extraction from low depth-of-field images based on colour-texture and HOS features[J]. Acta Automatica Sinica, 2013, 39(6): 846-851. (in Chinese)
[23]	WANG M M, LIU Y, HUANG Z Y. Large margin object tracking with circulant feature maps[C]//Proceedings of 2017 IEEE Conference on Computer Vision and Pattern Recognition.Honolulu, HI, US:IEEE, 2017:4800-4808.
[24]	NAVNEET D, BILL T. Histograms of oriented gradients for human detection[C]//Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition.San Diego, CA, US: IEEE, 2005, 1:886-893.
[25]	LI Y M, FU C H, DING F Q, et al. AutoTrack: towards high-performance visual tracking for UAV with automatic spatio-temporal regularization[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Seattle,WA, US:IEEE, 2020:11920-11929.
[26]	GONZALEZ R C, WOODS R E, EDDINS S L. Digital image processing using matlab[M]. Beijing: Publishing House of Electronics Industry, 2004:378-387.
[27]	MIHOUBI S, LOSSON O, MATHON B, et al. Multispectral demosaicing using pseudo-panchromatic image[J]. IEEE Transactions on Computational Imaging, 2017, 3(4): 982-995.
[28]	赵永强, 李宁, 潘泉. 分焦平面红外偏振摄像技术[M]. 北京: 科学出版社, 2022.
	ZHAO Y Q, LI N, PAN Q. Division of focal plane infrared polarization photography[M]. Beijing: Science Press, 2022. (in Chinese)
[29]	QIAO X B, CHEN L L, ZHAO Y Q. Polarization gradient histogram for object tracking in infrared polarization imaging: a feature extraction method for polarization mosaic image[C]//Proceedings of the 2021 10th International Conference on Computing and Pattern Recognition. 2021:224-230. https://doi.org/10.1145/3497623.3497659.
[30]	WU Y, LIM J, YANG M H. Object tracking benchmark[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2015, 37(9): 1834-1848. doi: 10.1109/TPAMI.2014.2388226 pmid: 26353130
[31]	HUANG D, ZHU C, WANG Y H, et al. HSOG: a novel local image descriptor based on histograms of the second-order gradients[J]. IEEE Transactions on Image Processing, 2014, 23(11): 4680-4695. doi: 10.1109/TIP.2014.2353814 pmid: 25203990
[32]	XIONG F C, ZHOU J, QIAN Y T. Material based object tracking in hyperspectral videos[J]. IEEE Transactions on Image Processing, 2020, 29(1): 3719-3733.
[33]	QIANG H Q, QIAN C H, ZHONG B J. A real-time moving target tracking algorithm based on SIFT[C]//Proceedings of 2017 International Conference on Security, Pattern Analysis, and Cybernetics. Shenzhen, China:SPAC,2017:569-572.
[34]	YANG Y X, YANG J, LIU L Y, et al. High-speed target tracking system based on a hierarchical parallel vision processor and gray-level LBP algorithm[J]. IEEE Transactions on Systems, Man, and Cybernetics: Systems, 2017, 47(6): 950-964.
[35]	BI P S, WANG D L, CHEN W, et al. Image dehazing based on polarization information and deep prior learning[J]. Optik, 2022, 267:169746.
[36]	DANELLIAN M, BHAT G, KHAN F S, et al. ECO: efficient convolution operators for tracking[C]//Proceedings of 2017 IEEE Conference on Computer Vision and Pattern Recognition.Honolulu, HI, US:IEEE, 2017:6931-6939.
[37]	XU T Y, FENG Z H, WU X J, et al. Joint group feature selection and discriminative filter learning for robust visual object tracking[C]//Proceedings of 2019 IEEE/CVF International Conference on Computer Vision.Seoul, Korea:IEEE, 2019:7949-7959.
[38]	HUANG Z Y, FU C H, LI Y M, et al. Learning aberrance repressed correlation filters for real-time UAV tracking[C]//Proceedings of 2019 IEEE/CVF International Conference on Computer Vision.Seoul, Korea: IEEE, 2019:2891-2900.
[39]	DAI K N, WANG D, LU H C, et al. Visual tracking via adaptive spatially-regularized correlation filters[C]//Proceedings of 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition.Long Beach, CA, US: IEEE, 2019:4665-4674.
[40]	TANG Z Q, ARAKAWA K. Visual tracking via spatial-temporal regularized correlation filters with advanced state estimation[C]//Proceedings of 2020 Asia-Pacific Signal and Information Processing Association Annual Summit and Conference. Auckland, New Zealand: IEEE, 2020:1145-1149.
[41]	LUKEZIC A, VOJIR T, ZAIC L C, et al. Discriminative correlation filter with channel and spatial reliability[C]//Proceedings of 2017 IEEE Conference on Computer Vision and Pattern Recognition.Honolulu, HI, US:IEEE, 2017:4847-4856.
[42]	GALOOGAHI H K, FAGG A, LUCEY S. Learning background-aware correlation filters for visual tracking[C]//Proceedings of 2017 IEEE International Conference on Computer Vision.Venice, Italy:IEEE, 2017:1144-1152.

方法	原始马赛克图像			去马赛克强度图像			偏振、强度融合图像
方法	跟踪精度	成功率	帧率	跟踪精度	成功率	帧率	跟踪精度	成功率	帧率
HSPMG	0.9428	0.8435	36.64
HOG	0.8990	0.7998	26.18	0.944	0.8425	1.42	0.8363	0.7032	1.56
HSOG	0.8069	0.6854	1.89	0.7925	0.6833	0.74	0.5560	0.4360	0.79
SSHMG	0.8784	0.7707	15.43	0.9102	0.8022	1.54	0.7560	0.6112	1.51
SIFT	0.8910	0.7839	0.37	0.9162	0.8132	0.25	0.7376	0.6016	0.19
LBP	0.3810	0.2750	0.64	0.2137	0.1691	0.38	0.2849	0.2108	0.40
Grayscale	0.7629	0.6529	200.01	0.7668	0.6607	1.15	0.5815	0.4543	1.22
HOG+Grayscale	0.8970	0.7973	39.58	0.9289	0.8232	1.12	0.7101	0.6001	0.95

方法	原始马赛克图像			去马赛克强度图像			偏振、强度融合图像
方法	跟踪精度	成功率	帧率	跟踪精度	成功率	帧率	跟踪精度	成功率	帧率
HSPMG	0.9428	0.8435	36.64
HOG	0.8990	0.7998	26.18	0.944	0.8425	1.42	0.8363	0.7032	1.56
HSOG	0.8069	0.6854	1.89	0.7925	0.6833	0.74	0.5560	0.4360	0.79
SSHMG	0.8784	0.7707	15.43	0.9102	0.8022	1.54	0.7560	0.6112	1.51
SIFT	0.8910	0.7839	0.37	0.9162	0.8132	0.25	0.7376	0.6016	0.19
LBP	0.3810	0.2750	0.64	0.2137	0.1691	0.38	0.2849	0.2108	0.40
Grayscale	0.7629	0.6529	200.01	0.7668	0.6607	1.15	0.5815	0.4543	1.22
HOG+Grayscale	0.8970	0.7973	39.58	0.9289	0.8232	1.12	0.7101	0.6001	0.95

参数	天气
参数	晴天	雨天	雾天	夜晚	强干扰	全部
跟踪精度	0.9633	0.9773	0.9754	0.9866	0.8383	0.9428
成功率	0.8825	0.8540	0.8319	0.8843	0.7686	0.8435
帧率	28.49	37.33	35.56	41.37	29.96	36.64

参数	天气
参数	晴天	雨天	雾天	夜晚	强干扰	全部
跟踪精度	0.9633	0.9773	0.9754	0.9866	0.8383	0.9428
成功率	0.8825	0.8540	0.8319	0.8843	0.7686	0.8435
帧率	28.49	37.33	35.56	41.37	29.96	36.64

方法	原始马赛克图像			去马赛克强度图像			偏振、强度融合图像
方法	跟踪精度	成功率	帧率	跟踪精度	成功率	帧率	跟踪精度	成功率	帧率
HSPMG	0.9428	0.8435	36.64
AutoTrack	0.8970	0.7973	39.58	0.9289	0.8232	1.12	0.7101	0.6001	0.95
ECO	0.9358	0.8172	22.72	0.9434	0.8280	1.33	0.5833	0.463	1.49
GFS-DCF	0.9036	0.7771	29.40	0.8745	0.7547	1.42	0.6331	0.511	1.39
GFS-DCF deep	0.7909	0.6572	3.82	0.8295	0.6979	0.99	0.7610	0.6156	1.55
ARCF	0.9221	0.8146	53.17	0.9300	0.8184	1.44	0.7979	0.6623	1.29
ASRCF	0.7840	0.6634	12.16	0.7912	0.6642	1.05	0.7137	0.5858	1.27
STRCF	0.9220	0.8036	46.96	0.9013	0.7884	1.40	0.7629	0.6199	1.54
STRCF deep	0.8457	0.7267	5.3687	0.8747	0.7449	1.10	0.7671	0.6264	1.26
CSR-DCF	0.9133	0.7142	15.41	0.8526	0.6596	1.40	0.7224	0.5579	1.38
BACF	0.8871	0.7791	61.14	0.9473	0.8349	1.62	0.8012	0.6636	1.20