基于边缘智能终端的跨模态行人重识别方法及应用

doi:10.12382/bgxb.2022.1113

摘要/Abstract

摘要：

跨模态行人重识别技术旨在可见光、红外等不同模态图像中识别出同一个人,其在人机协同、万物互联、跨界融合、万物智能的智能系统与装备中具有重要应用。提出一种数据增强的跨模态行人重识别方法,在波长域进行数据增强的同时保留可见图像的结构信息,以弥合不同模态之间的差距。在此基础上,基于瑞芯微的RK3588芯片设计实现了一套边缘智能终端,并部署了跨模态行人重识别算法。在边缘计算部署的硬件设计、软件开发中,通过模块化的设计、层级配置的方案,实现了系统弹性可扩展,降低了数据集中处理的计算压力。实验结果表明,新方法在两个基准数据集SYSU-MM01和RegDB上取得了较好的性能,并能够在实际场景中进行应用部署。

关键词: 跨模态行人重识别, 数据增强, 边缘智能终端

Abstract:

Cross-modallity person re-identification technology (cm-ReID) aims to identify the same person in visible and infrared images. It has crucial applications in intelligent systems and equipment for human-machine cooperation, Internet of everything, cross-border integration, and intelligence of everything. In this paper, we propose a cross-modality person re-identification method based on data enhancement, which preserves the structural information of the visible image while performing data enhancement in the wavelength domain to bridge the gap between different modalities. On this basis, a set of edge intelligent terminals are designed and implemented based on the RK3588 chip and a cross-modality person re-identification algorithm is deployed. In the hardware design and software development of edge computing deployment, the system is flexible and scalable through modular design and hierarchical configuration, reducing the computing pressure of centralized data processing. The experimental results show that the proposed method has performed well on two benchmark data sets, SYSU-MM01 and RegDB, and can be deployed in practical scenarios.

Key words: cross-modality person re-identification, data enhancement, edge intelligent terminal

中图分类号:

卞紫阳, 许廷发, 马亮, 李佳男. 基于边缘智能终端的跨模态行人重识别方法及应用[J]. 兵工学报, 2023, 44(9): 2631-2638.

BIAN Ziyang, XU Tingfa, MA Liang, LI Jianan. Application of Cross-modality Person Re-identificaton based on Edge Intelligent Terminal[J]. Acta Armamentarii, 2023, 44(9): 2631-2638.

图/表 11

图1 Re-ID问题示意图

Fig.1 Schematic diagram of person re-identification

图2 跨模态Re-ID方法及数据集(左为SYSU-MM01数据集,右为RegDB数据集)

Fig.2 Dataset of cross-modality person re-identification (SYSU-MM01 on the left and RegDB on the right)

图3 红外可见光跨模态Re-ID方法

Fig.3 Cross-modality person re-identification algorithm

图4 跨模态Re-ID系统

Fig.4 Implementation of the cross-modality person re-identification

图5 基于RK3588处理器的边缘智能终端实物图(左为终端正面,右为终端反面)

Fig.5 Edge intelligent terminal based on the RK3588 (Left: front of the device; right: back of the device)

图6 边缘智能终端软件示意图

Fig.6 Diagram of edge intelligent terminal software

图7 边缘智能终端软件依赖关系图

Fig.7 Edge intelligent terminal software dependency diagram

图8 模型训练流程图

Fig.8 Flow chart of model training

表1 SYSU-MM01数据集上的运行结果对比分析

Table 1 Comparison of results in SYSU-MM01

方法	Rank-1	Rank-10	Rank-20	mAP
HCML^[19]	14.32	53.16	69.17	16.16
Zero-Pad^[16]	14.80	54.12	71.33	15.95
HSME^[20]	20.68	32.74	77.95	23.12
D²RL^[21]	28.90	70.60	82.40	29.20
MAC^[22]	33.26	79.04	90.09	36.22
MSR^[23]	37.35	83.40	93.34	38.11
HAT^[18]	55.29	92.14	97.36	53.89
本文方法	58.89	93.01	97.67	57.67

表2 RegDB数据集上的运行结果对比分析

Table 2 Comparison of results in RegDB

方法	Rank-1	Rank-10	Rank-20	mAP
HCML^[19]	24.44	47.53	56.78	20.08
Zero-Pad^[16]	17.75	34.21	44.35	18.90
HSME^[20]	50.85	73.36	81.66	47.00
D²RL ^[21]	43.40	66.10	76.30	44.10
MAC^[22]	36.43	62.36	71.63	37.03
MSR^[23]	48.43	70.32	79.95	48.67
HAT^[18]	71.83	87.16	92.16	67.56
本文方法	75.32	89.62	93.83	71.50

图9 边缘智能终端实际应用部署效果(上为可见光图像,下为红外图像)

Fig.9 Application and deployment of edge intelligent terminal (Visible image on the top and infrared image on the bottom)

参考文献 23

[1]	ZHENG L, YANG Y, HAUPTMANN A G. Person re-identification: past, present and future: arXiv:1610.02984[R]. Ithaca, NY, US: Cornell University, 2016: 1610.02984.
[2]	KARANAM S, GOU M R, WU Z Y, et al. A systematic evaluation and benchmark for person re-identification:features, metrics, and datasets[J]. IEEE Transactions on Pattern Analysis & Machine Intelligence, 2019, 41(3):523-536.
[3]	CHEN Y C, ZHU X, ZHENG W S, et al. Person re-identification by camera correlation aware feature augmentation[J]. IEEE Transactions on Pattern Analysis & Machine Intelligence, 2017, 40(2):392-408.
[4]	WU L, WANG Y, GAO J B, et al. Where-and-when to look: deep siamese attention networks for video-based person re-identification[J]. IEEE Transactions on Multimedia, 2018, 21: 1412-1424. doi: 10.1109/TMM.6046 URL
[5]	YANG X, ZHOU P C, WANG M. Person reidentification via structural deep metric learning[J]. IEEE Transactions on Neural Networks and Learning Systems, 2019, 30(10):2987-2998. doi: 10.1109/TNNLS.2018.2861991 pmid: 32175851
[6]	WEI L H, ZHANG S L, YAO H T, et al. GLAD:global-local-alignment descriptor for scalable person re-identification[J]. IEEE Transactions on Multimedia, 2019, 21(4):986-999. doi: 10.1109/TMM.6046 URL
[7]	郑爱华, 曾小强, 江波, 等. 基于局部异质协同双路网络的跨模态行人重识别[J]. 模式识别与人工智能, 2020, 33(10):867-878. doi: 10.16451/j.cnki.issn1003-6059.202010001
	ZHENG A H, ZENG X Q, JIANG B, et al. Cross-modal person re-identification based on local heterogeneous collaborative dual-path network[J]. Pattern Recognition and Artificial Intelligence, 2020, 33(10):867-878. (in Chinese) doi: 10.16451/j.cnki.issn1003-6059.202010001
[8]	刘玉林. 跨模态行人重识别研究综述[J]. 电视技术, 2022, 46(5):9-11.
	LIU Y L. A review of cross-modal person re-identification[J]. Video Engineering, 2022, 46(5):9-11. (in Chinese)
[9]	XIA D X, LIU H J, XU L L, et al. Visible-infrared person re-identification with data augmentation via cycle-consistent adversarial network[J]. Neurocomputing, 2021, 443:35-46. doi: 10.1016/j.neucom.2021.02.088 URL
[10]	崔令飞, 郭永红, 修全发, 等. 基于国产嵌入式智能计算平台的无人机检测方法[J]. 兵工学报, 2022, 43(增刊1): 146-154.
	CUI L F, GUO Y H, XIU Q F, et al. UAV detection method based on domestic embedded intelligent computing platform[J]. Acta Armamentarii, 2022, 43(S1): 146-154. (in Chinese) doi: 10.12382/bgxb.2022.A013
[11]	GOMES H, REDINHA N, LAVADO N, et al. Counting people and bicycles in real time using YOLO on jetson nano[J]. Energies, 2022, 15(23):8816. doi: 10.3390/en15238816 URL
[12]	OLIVEIRA V M, MOREIRA A H J. Edge AI system using a thermal camera for industrial anomaly detection[M]. Cham, Germany:Springer, 2022:172-178.
[13]	DINH D L, NGUYEN H N, THAI H T, et al. Towards AI-based traffic counting system with edge computing[J]. Journal of Advanced Transportation, 2021, 2021(2):1-15.
[14]	REN S, KIM J S, CHO W S, et al. Big data platform for intelligence industrial IoT sensor monitoring system based on edge computing and AI[C]//Proceedings of 2021 International Conference on Artificial Intelligence in Information and Communication.Jeju Island, Korea:IEEE, 2021.
[15]	LYU S, LI R, ZHAO Y W, et al. Green citrus detection and counting in orchards based on YOLOv5-CS and AI edge system[J]. Sensors, 2022, 22(2):576. doi: 10.3390/s22020576 URL
[16]	WU A C, ZHENG W S, YU H X, et al. RGB-infrared cross-modality person re-identification[C]//Proceedings of 2017 IEEE International Conference on Computer Vision. Venice,Italy:IEEE, 2017.
[17]	NGUYEN D T, HONG H G, KIM K W, et al. Person recognition system based on a combination of body images from visible light and thermal cameras[J]. Sensors, 2017, 17(3):605. doi: 10.3390/s17030605 URL
[18]	YE M, SHEN J B, SHAO L. Visible-infrared person re-identification via homogeneous augmented tri-modal learning[J]. IEEE Transactions on Information Forensics and Security, 2020, 16:728-739. doi: 10.1109/TIFS.10206 URL
[19]	YE M, LAN X, LI J W, et al. Hierarchical discriminative learning for visible thermal person re-identification[C]//Proceedings of National Conference on Artificial Intelligence. Reston, VA, US:AAAI, 2018.
[20]	HAO Y, WANG N N, LI J, et al. HSME: hypersphere manifold embedding for visible thermal person re-identification[C]//Proceedings of the AAAI Conference on Artificial Intelligence. HI, US:AAAI, 2019:8385-8392.
[21]	WANG Z X, WANG Z, ZHENG Y Q, et al. Learning to reduce dual-level discrepancy for infrared-visible person re-identification[C]//Proceedings of 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Washington,D.C.,US:IEEE, 2019.
[22]	YE M, LAN X, LENG Q M. Modality-aware collaborative learning for visible thermal person re-identification[C]//Proceedings of the 27th ACM International Conference. New York, NY, US:ACM, 2019.
[23]	FENG Z X, LAI J H, XIE X H. Learning modality-specific representations for visible-infrared person re-identification[J]. IEEE Transactions on Image Processing: a Ppublication of the IEEE Signal Processing Society, 2019, 29:579-590.