Application of Cross-modality Person Re-identificaton based on Edge Intelligent Terminal

doi:10.12382/bgxb.2022.1113

Abstract

Abstract:

Cross-modallity person re-identification technology (cm-ReID) aims to identify the same person in visible and infrared images. It has crucial applications in intelligent systems and equipment for human-machine cooperation, Internet of everything, cross-border integration, and intelligence of everything. In this paper, we propose a cross-modality person re-identification method based on data enhancement, which preserves the structural information of the visible image while performing data enhancement in the wavelength domain to bridge the gap between different modalities. On this basis, a set of edge intelligent terminals are designed and implemented based on the RK3588 chip and a cross-modality person re-identification algorithm is deployed. In the hardware design and software development of edge computing deployment, the system is flexible and scalable through modular design and hierarchical configuration, reducing the computing pressure of centralized data processing. The experimental results show that the proposed method has performed well on two benchmark data sets, SYSU-MM01 and RegDB, and can be deployed in practical scenarios.

Key words: cross-modality person re-identification, data enhancement, edge intelligent terminal

CLC Number:

BIAN Ziyang, XU Tingfa, MA Liang, LI Jianan. Application of Cross-modality Person Re-identificaton based on Edge Intelligent Terminal[J]. Acta Armamentarii, 2023, 44(9): 2631-2638.

Figures/Tables 11

Fig.1 Schematic diagram of person re-identification

Fig.2 Dataset of cross-modality person re-identification (SYSU-MM01 on the left and RegDB on the right)

Fig.3 Cross-modality person re-identification algorithm

Fig.4 Implementation of the cross-modality person re-identification

Fig.5 Edge intelligent terminal based on the RK3588 (Left: front of the device; right: back of the device)

Fig.6 Diagram of edge intelligent terminal software

Fig.7 Edge intelligent terminal software dependency diagram

Fig.8 Flow chart of model training

Table 1 Comparison of results in SYSU-MM01

方法	Rank-1	Rank-10	Rank-20	mAP
HCML^[19]	14.32	53.16	69.17	16.16
Zero-Pad^[16]	14.80	54.12	71.33	15.95
HSME^[20]	20.68	32.74	77.95	23.12
D²RL^[21]	28.90	70.60	82.40	29.20
MAC^[22]	33.26	79.04	90.09	36.22
MSR^[23]	37.35	83.40	93.34	38.11
HAT^[18]	55.29	92.14	97.36	53.89
本文方法	58.89	93.01	97.67	57.67

Table 2 Comparison of results in RegDB

方法	Rank-1	Rank-10	Rank-20	mAP
HCML^[19]	24.44	47.53	56.78	20.08
Zero-Pad^[16]	17.75	34.21	44.35	18.90
HSME^[20]	50.85	73.36	81.66	47.00
D²RL ^[21]	43.40	66.10	76.30	44.10
MAC^[22]	36.43	62.36	71.63	37.03
MSR^[23]	48.43	70.32	79.95	48.67
HAT^[18]	71.83	87.16	92.16	67.56
本文方法	75.32	89.62	93.83	71.50

Fig.9 Application and deployment of edge intelligent terminal (Visible image on the top and infrared image on the bottom)

References 23

[1]	ZHENG L, YANG Y, HAUPTMANN A G. Person re-identification: past, present and future: arXiv:1610.02984[R]. Ithaca, NY, US: Cornell University, 2016: 1610.02984.
[2]	KARANAM S, GOU M R, WU Z Y, et al. A systematic evaluation and benchmark for person re-identification:features, metrics, and datasets[J]. IEEE Transactions on Pattern Analysis & Machine Intelligence, 2019, 41(3):523-536.
[3]	CHEN Y C, ZHU X, ZHENG W S, et al. Person re-identification by camera correlation aware feature augmentation[J]. IEEE Transactions on Pattern Analysis & Machine Intelligence, 2017, 40(2):392-408.
[4]	WU L, WANG Y, GAO J B, et al. Where-and-when to look: deep siamese attention networks for video-based person re-identification[J]. IEEE Transactions on Multimedia, 2018, 21: 1412-1424. doi: 10.1109/TMM.6046 URL
[5]	YANG X, ZHOU P C, WANG M. Person reidentification via structural deep metric learning[J]. IEEE Transactions on Neural Networks and Learning Systems, 2019, 30(10):2987-2998. doi: 10.1109/TNNLS.2018.2861991 pmid: 32175851
[6]	WEI L H, ZHANG S L, YAO H T, et al. GLAD:global-local-alignment descriptor for scalable person re-identification[J]. IEEE Transactions on Multimedia, 2019, 21(4):986-999. doi: 10.1109/TMM.6046 URL
[7]	郑爱华, 曾小强, 江波, 等. 基于局部异质协同双路网络的跨模态行人重识别[J]. 模式识别与人工智能, 2020, 33(10):867-878. doi: 10.16451/j.cnki.issn1003-6059.202010001
	ZHENG A H, ZENG X Q, JIANG B, et al. Cross-modal person re-identification based on local heterogeneous collaborative dual-path network[J]. Pattern Recognition and Artificial Intelligence, 2020, 33(10):867-878. (in Chinese) doi: 10.16451/j.cnki.issn1003-6059.202010001
[8]	刘玉林. 跨模态行人重识别研究综述[J]. 电视技术, 2022, 46(5):9-11.
	LIU Y L. A review of cross-modal person re-identification[J]. Video Engineering, 2022, 46(5):9-11. (in Chinese)
[9]	XIA D X, LIU H J, XU L L, et al. Visible-infrared person re-identification with data augmentation via cycle-consistent adversarial network[J]. Neurocomputing, 2021, 443:35-46. doi: 10.1016/j.neucom.2021.02.088 URL
[10]	崔令飞, 郭永红, 修全发, 等. 基于国产嵌入式智能计算平台的无人机检测方法[J]. 兵工学报, 2022, 43(增刊1): 146-154.
	CUI L F, GUO Y H, XIU Q F, et al. UAV detection method based on domestic embedded intelligent computing platform[J]. Acta Armamentarii, 2022, 43(S1): 146-154. (in Chinese) doi: 10.12382/bgxb.2022.A013
[11]	GOMES H, REDINHA N, LAVADO N, et al. Counting people and bicycles in real time using YOLO on jetson nano[J]. Energies, 2022, 15(23):8816. doi: 10.3390/en15238816 URL
[12]	OLIVEIRA V M, MOREIRA A H J. Edge AI system using a thermal camera for industrial anomaly detection[M]. Cham, Germany:Springer, 2022:172-178.
[13]	DINH D L, NGUYEN H N, THAI H T, et al. Towards AI-based traffic counting system with edge computing[J]. Journal of Advanced Transportation, 2021, 2021(2):1-15.
[14]	REN S, KIM J S, CHO W S, et al. Big data platform for intelligence industrial IoT sensor monitoring system based on edge computing and AI[C]//Proceedings of 2021 International Conference on Artificial Intelligence in Information and Communication.Jeju Island, Korea:IEEE, 2021.
[15]	LYU S, LI R, ZHAO Y W, et al. Green citrus detection and counting in orchards based on YOLOv5-CS and AI edge system[J]. Sensors, 2022, 22(2):576. doi: 10.3390/s22020576 URL
[16]	WU A C, ZHENG W S, YU H X, et al. RGB-infrared cross-modality person re-identification[C]//Proceedings of 2017 IEEE International Conference on Computer Vision. Venice,Italy:IEEE, 2017.
[17]	NGUYEN D T, HONG H G, KIM K W, et al. Person recognition system based on a combination of body images from visible light and thermal cameras[J]. Sensors, 2017, 17(3):605. doi: 10.3390/s17030605 URL
[18]	YE M, SHEN J B, SHAO L. Visible-infrared person re-identification via homogeneous augmented tri-modal learning[J]. IEEE Transactions on Information Forensics and Security, 2020, 16:728-739. doi: 10.1109/TIFS.10206 URL
[19]	YE M, LAN X, LI J W, et al. Hierarchical discriminative learning for visible thermal person re-identification[C]//Proceedings of National Conference on Artificial Intelligence. Reston, VA, US:AAAI, 2018.
[20]	HAO Y, WANG N N, LI J, et al. HSME: hypersphere manifold embedding for visible thermal person re-identification[C]//Proceedings of the AAAI Conference on Artificial Intelligence. HI, US:AAAI, 2019:8385-8392.
[21]	WANG Z X, WANG Z, ZHENG Y Q, et al. Learning to reduce dual-level discrepancy for infrared-visible person re-identification[C]//Proceedings of 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Washington,D.C.,US:IEEE, 2019.
[22]	YE M, LAN X, LENG Q M. Modality-aware collaborative learning for visible thermal person re-identification[C]//Proceedings of the 27th ACM International Conference. New York, NY, US:ACM, 2019.
[23]	FENG Z X, LAI J H, XIE X H. Learning modality-specific representations for visible-infrared person re-identification[J]. IEEE Transactions on Image Processing: a Ppublication of the IEEE Signal Processing Society, 2019, 29:579-590.