1. 北京理工大学 光电成像技术与系统教育部重点实验室, 北京 100081
2. 华北光电技术研究所, 北京 100015
3. 北京理工大学重庆创新中心, 重庆 401120
*邮箱: ciom_xtf1@bit.edu.cn
收稿:2022-11-29,
网络出版:2023-09-25,
纸质出版:2023-09-20
移动端阅览
卞紫阳, 许廷发, 马亮, 等. 基于边缘智能终端的跨模态行人重识别方法及应用[J]. 兵工学报, 2023,44(9):2631-2638.
Ziyang BIAN, Tingfa XU, Liang MA, et al. Application of Cross-modality Person Re-identificaton based on Edge Intelligent Terminal[J]. Acta Armamentarii, 2023, 44(9): 2631-2638.
卞紫阳, 许廷发, 马亮, 等. 基于边缘智能终端的跨模态行人重识别方法及应用[J]. 兵工学报, 2023,44(9):2631-2638. DOI: 10.12382/bgxb.2022.1113.
Ziyang BIAN, Tingfa XU, Liang MA, et al. Application of Cross-modality Person Re-identificaton based on Edge Intelligent Terminal[J]. Acta Armamentarii, 2023, 44(9): 2631-2638. DOI: 10.12382/bgxb.2022.1113.
跨模态行人重识别技术旨在可见光、红外等不同模态图像中识别出同一个人
其在人机协同、万物互联、跨界融合、万物智能的智能系统与装备中具有重要应用。提出一种数据增强的跨模态行人重识别方法
在波长域进行数据增强的同时保留可见图像的结构信息
以弥合不同模态之间的差距。在此基础上
基于瑞芯微的RK3588芯片设计实现了一套边缘智能终端
并部署了跨模态行人重识别算法。在边缘计算部署的硬件设计、软件开发中
通过模块化的设计、层级配置的方案
实现了系统弹性可扩展
降低了数据集中处理的计算压力。实验结果表明
新方法在两个基准数据集SYSU-MM01和RegDB上取得了较好的性能
并能够在实际场景中进行应用部署。
Cross-modallity person re-identification technology (cm-ReID) aims to identify the same person in visible and infrared images. It has crucial applications in intelligent systems and equipment for human-machine cooperation
Internet of everything
cross-border integration
and intelligence of everything. In this paper
we propose a cross-modality person re-identification method based on data enhancement
which preserves the structural information of the visible image while performing data enhancement in the wavelength domain to bridge the gap between different modalities. On this basis
a set of edge intelligent terminals are designed and implemented based on the RK3588 chip and a cross-modality person re-identification algorithm is deployed. In the hardware design and software development of edge computing deployment
the system is flexible and scalable through modular design and hierarchical configuration
reducing the computing pressure of centralized data processing. The experimental results show that the proposed method has performed well on two benchmark data sets
SYSU-MM01 and RegDB
and can be deployed in practical scenarios.
ZHENG L , YANG Y , HAUPTMANN A G . Person re-identification: past, present and future: arXiv:1610.02984 [R ] . Ithaca, NY , US : Cornell University , 2016 : 1610.02984.
KARANAM S , GOU M R , WU Z Y , et al. A systematic evaluation and benchmark for person re-identification:features, metrics, and datasets [J ] . IEEE Transactions on Pattern Analysis & Machine Intelligence , 2019 , 41 ( 3 ): 523 - 536 .
CHEN Y C , ZHU X , ZHENG W S , et al. Person re-identification by camera correlation aware feature augmentation [J ] . IEEE Transactions on Pattern Analysis & Machine Intelligence , 2017 , 40 ( 2 ): 392 - 408 .
WU L , WANG Y , GAO J B , et al. Where-and-when to look: deep siamese attention networks for video-based person re-identification [J ] . IEEE Transactions on Multimedia , 2018 , 21 : 1412 - 1424 . DOI: 10.1109/TMM.6046 http://doi.org/10.1109/TMM.6046 https://ieeexplore.ieee.org/xpl/RecentIssue.jsp?punumber=6046 https://ieeexplore.ieee.org/xpl/RecentIssue.jsp?punumber=6046
YANG X , ZHOU P C , WANG M . Person reidentification via structural deep metric learning [J ] . IEEE Transactions on Neural Networks and Learning Systems , 2019 , 30 ( 10 ): 2987 - 2998 . DOI: 10.1109/TNNLS.2018.2861991 http://doi.org/10.1109/TNNLS.2018.2861991 Despite the promising progress made in recent years, person reidentification (re-ID) remains a challenging task due to the complex variations in human appearances from different camera views. This paper proposes to tackle this task by jointly learning feature representation and distance metric in an end-to-end manner. Existing deep metric learning-based re-ID methods usually encounter the following two weaknesses: 1) most works based on pairwise or triplet constraints often suffer from slow convergence and poor local optima, partially because they use very limited samples for each update and 2) hard negative sample mining has been widely applied in existing works. However, hard positive samples, which also contribute to the training of network, have not received enough attention. To alleviate these problems, we develop a novel structural metric learning objective for person re-ID, in which each positive pair is allowed to be compared against all negative pairs in a minibatch and each positive pair is adaptively assigned a hardness-aware weight to modulate its contribution. The introduced positive pair weighting strategy enables the algorithm to focus more on the hard positive samples. Furthermore, we propose to enhance the proposed loss function by adding a global loss term to reduce the variances of positive/negative pair distances, which is able to improve the generalization capability of the network model. By this approach, person images can be nonlinearly mapped into a low-dimensional embedding space where similar samples are kept closer and dissimilar samples are pushed farther apart. We implement the proposed algorithm using the inception architecture and evaluate it on three large-scale re-ID data sets. Experiment results demonstrate that our approach is able to outperform most state of the arts while using much lower dimensional deep features.
WEI L H , ZHANG S L , YAO H T , et al . GLAD:global-local-alignment descriptor for scalable person re-identification [J ] . IEEE Transactions on Multimedia , 2019 , 21 ( 4 ): 986 - 999 . DOI: 10.1109/TMM.6046 http://doi.org/10.1109/TMM.6046 https://ieeexplore.ieee.org/xpl/RecentIssue.jsp?punumber=6046 https://ieeexplore.ieee.org/xpl/RecentIssue.jsp?punumber=6046
郑爱华 , 曾小强 , 江波 , 等 . 基于局部异质协同双路网络的跨模态行人重识别 [J ] . 模式识别与人工智能 , 2020 , 33 ( 10 ): 867 - 878 . DOI: 10.16451/j.cnki.issn1003-6059.202010001 http://doi.org/10.16451/j.cnki.issn1003-6059.202010001 针对现有跨模态行人重识别方法忽略行人的局部特征及模态间的相互协同的问题,文中提出基于局部异质协同双路网络的跨模态行人重识别方法.首先,通过双路网络提取不同模态的全局特征进行局部精细化,挖掘行人的结构化局部信息.然后,通过标签和预测信息建立跨模态局部信息之间的关联,进行协同自适应的跨模态融合,使不同模态的特征之间相互补充,获得富有判别力的特征.在RegDB、SYSU-MM01跨模态行人重识别数据集上的实验验证文中方法的有效性.
ZHENG A H , ZENG X Q , JIANG B , et al . Cross-modal person re-identification based on local heterogeneous collaborative dual-path network [J ] . Pattern Recognition and Artificial Intelligence , 2020 , 33 ( 10 ): 867 - 878 . (in Chinese) DOI: 10.16451/j.cnki.issn1003-6059.202010001 http://doi.org/10.16451/j.cnki.issn1003-6059.202010001 The coordinating fusion between modalities is ignored in the existing cross-modal person re-identification methods in the learning process. In this paper, a strategy for cross-modal person re-identification(Re-ID) based on local heterogeneous collaborative dual-path network is proposed. Firstly, the global features of each modality are extracted by the dual-path network for local refinement, and the structured local information of pedestrians is mined. Then, the local information of different modalities is correlated with the label and prediction information to achieve cooperative adaptive fusion and learn more discriminative features. The effectiveness of the proposed method is demonstrated through comprehensive
刘玉林 . 跨模态行人重识别研究综述 [J ] . 电视技术 , 2022 , 46 ( 5 ): 9 - 11 .
LIU Y L . A review of cross-modal person re-identification [J ] . Video Engineering , 2022 , 46 ( 5 ): 9 - 11 . (in Chinese)
XIA D X , LIU H J , XU L L , et al. Visible-infrared person re-identification with data augmentation via cycle-consistent adversarial network [J ] . Neurocomputing , 2021 , 443 : 35 - 46 . DOI: 10.1016/j.neucom.2021.02.088 http://doi.org/10.1016/j.neucom.2021.02.088 https://linkinghub.elsevier.com/retrieve/pii/S0925231221003465 https://linkinghub.elsevier.com/retrieve/pii/S0925231221003465
崔令飞 , 郭永红 , 修全发 , 等 . 基于国产嵌入式智能计算平台的无人机检测方法 [J ] . 兵工学报 , 2022 , 43 ( 增刊1 ): 146 - 154 .
CUI L F , GUO Y H , XIU Q F , et al. UAV detection method based on domestic embedded intelligent computing platform [J ] . Acta Armamentarii , 2022 , 43 ( S1 ): 146 - 154 . (in Chinese) DOI: 10.12382/bgxb.2022.A013 http://doi.org/10.12382/bgxb.2022.A013 A UAV detection method based on a domestic embedded intelligent computing platform is proposed to meet the actual requirements of anti-unmanned aerial vehicle(UAV) reconnaissance on the land battlefield. For the problem that UAV is small in size and not easy to be detected in the battlefield environment,the detection method is to use infrared and visible light images and video streams inputs for target detection. For the limited computing power and storage capacity of embedded platform,a lightweight deep neural network is built,and the feature extraction network in single shot multi-box detector(SSD) is replaced with MobileNet for model compression. The embedded platform Bitmain SE5 intelligent computing box is selected for verification, and the model conversion and transplantation are achieved. The experimental result shows that the proposed UAV detection method based on the lightweight deep neural network MobileNet-SSD can accurately determine the type of targets on the embedded intelligent computing platform, and the mean recognition accuracy and frame rate are basically same with those running in the development environment. It fully shows that the detection method can meet the requirements of the real-time and accuracy of UAV detection algorithm in the application environment in terms of speed and accuracy after being transplanted on the embedded intelligent computing platform.
GOMES H , REDINHA N , LAVADO N , et al. Counting people and bicycles in real time using YOLO on jetson nano [J ] . Energies , 2022 , 15 ( 23 ): 8816 . DOI: 10.3390/en15238816 http://doi.org/10.3390/en15238816 https://www.mdpi.com/1996-1073/15/23/8816 https://www.mdpi.com/1996-1073/15/23/8816 Counting objects in video images has been an active area of computer vision for decades. For precise counting, it is necessary to detect objects and follow them through consecutive frames. Deep neural networks have allowed great improvements in this area. Nonetheless, this task is still a challenge for edge computing, especially when low-power edge AI devices must be used. The present work describes an application where an edge device is used to run a YOLO network and V-IOU tracker to count people and bicycles in real time. A selective frame-downsampling algorithm is used to allow a larger frame rate when necessary while optimizing memory usage and energy consumption. In the experiments, the system was able to detect and count the objects with 18 counting errors in 525 objects and a mean inference time of 112.82 ms per frame. With the selective downsampling algorithm, it was also capable of recovering and reduce memory usage while maintaining its precision.
OLIVEIRA V M , MOREIRA A H J . Edge AI system using a thermal camera for industrial anomaly detection [M ] . Cham, Germany:Springer , 2022 : 172 - 178 .
DINH D L , NGUYEN H N , THAI H T , et al . Towards AI-based traffic counting system with edge computing [J ] . Journal of Advanced Transportation , 2021 , 2021 ( 2 ): 1 - 15 .
REN S , KIM J S , CHO W S , et al . Big data platform for intelligence industrial IoT sensor monitoring system based on edge computing and AI [C ] //Proceedings of 2021 International Conference on Artificial Intelligence in Information and Communication.Jeju Island, Korea:IEEE , 2021 .
LYU S , LI R , ZHAO Y W , et al . Green citrus detection and counting in orchards based on YOLOv5-CS and AI edge system [J ] . Sensors , 2022 , 22 ( 2 ): 576 . DOI: 10.3390/s22020576 http://doi.org/10.3390/s22020576 https://www.mdpi.com/1424-8220/22/2/576 https://www.mdpi.com/1424-8220/22/2/576 Green citrus detection in citrus orchards provides reliable support for production management chains, such as fruit thinning, sunburn prevention and yield estimation. In this paper, we proposed a lightweight object detection YOLOv5-CS (Citrus Sort) model to realize object detection and the accurate counting of green citrus in the natural environment. First, we employ image rotation codes to improve the generalization ability of the model. Second, in the backbone, a convolutional layer is replaced by a convolutional block attention module, and a detection layer is embedded to improve the detection accuracy of the little citrus. Third, both the loss function CIoU (Complete Intersection over Union) and cosine annealing algorithm are used to get the better training effect of the model. Finally, our model is migrated and deployed to the AI (Artificial Intelligence) edge system. Furthermore, we apply the scene segmentation method using the “virtual region” to achieve accurate counting of the green citrus, thereby forming an embedded system of green citrus counting by edge computing. The results show that the mAP@.5 of the YOLOv5-CS model for green citrus was 98.23%, and the recall is 97.66%. The inference speed of YOLOv5-CS detecting a picture on the server is 0.017 s, and the inference speed on Nvidia Jetson Xavier NX is 0.037 s. The detection and counting frame rate of the AI edge system-side counting system is 28 FPS, which meets the counting requirements of green citrus.
WU A C , ZHENG W S , YU H X , et al . RGB-infrared cross-modality person re-identification [C ] //Proceedings of 2017 IEEE International Conference on Computer Vision. Venice,Italy:IEEE , 2017 .
NGUYEN D T , HONG H G , KIM K W , et al. Person recognition system based on a combination of body images from visible light and thermal cameras [J ] . Sensors , 2017 , 17 ( 3 ): 605 . DOI: 10.3390/s17030605 http://doi.org/10.3390/s17030605 http://www.mdpi.com/1424-8220/17/3/605 http://www.mdpi.com/1424-8220/17/3/605
YE M , SHEN J B , SHAO L . Visible-infrared person re-identification via homogeneous augmented tri-modal learning [J ] . IEEE Transactions on Information Forensics and Security , 2020 , 16 : 728 - 739 . DOI: 10.1109/TIFS.10206 http://doi.org/10.1109/TIFS.10206 https://ieeexplore.ieee.org/xpl/RecentIssue.jsp?punumber=10206 https://ieeexplore.ieee.org/xpl/RecentIssue.jsp?punumber=10206
YE M , LAN X , LI J W , et al. Hierarchical discriminative learning for visible thermal person re-identification [C ] //Proceedings of National Conference on Artificial Intelligence. Reston, VA, US:AAAI , 2018 .
HAO Y , WANG N N , LI J , et al. HSME: hypersphere manifold embedding for visible thermal person re-identification [C ] //Proceedings of the AAAI Conference on Artificial Intelligence. HI, US:AAAI , 2019 : 8385 - 8392 .
WANG Z X , WANG Z , ZHENG Y Q , et al . Learning to reduce dual-level discrepancy for infrared-visible person re-identification [C ] //Proceedings of 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Washington,D.C.,US:IEEE , 2019 .
YE M , LAN X , LENG Q M . Modality-aware collaborative learning for visible thermal person re-identification [C ] //Proceedings of the 27th ACM International Conference. New York, NY, US:ACM , 2019 .
FENG Z X , LAI J H , XIE X H . Learning modality-specific representations for visible-infrared person re-identification [J ] . IEEE Transactions on Image Processing: a Ppublication of the IEEE Signal Processing Society , 2019 , 29 : 579 - 590 .
0
浏览量
533
下载量
0
CNKI被引量
关联资源
相关文章
相关作者
相关机构
京公网安备11010802024360号