基于旋转等变卷积的航拍红外图像目标识别算法

doi:10.12382/bgxb.2023.0503

摘要/Abstract

摘要：

为提高传统无人机红外目标识别算法对输入图像的旋转鲁棒性,提出一种具有旋转等变性的红外图像目标识别算法。参照可见光三通道结构,将红外图像扩张为三通道以丰富输入图像的细节及边缘信息;以旋转等变卷积为基础,设计并实现能够高度保留图像旋转特征的标准旋转等变卷积模块和旋转残差模块,使得所设计模型FC-YOLOv5对图像及图像中目标旋转具有鲁棒性;加入压缩和激励注意力机制自适应地学习到每个通道的重要性,并且根据任务的需要加权调整特征图中的通道贡献,提取重要的特征信息并抑制不重要的特征信息。在航拍行人车辆数据集和海上船舶数据集上验证模型的性能,以基准模型YOLOv5s及常见轻量级目标识别任务所用模型YOLOv8s、NanoDet作为对照组模型。实验结果表明,所提算法的平均精度均值相较于基准模型能够提升2%~4%,且当输入图像具有不同角度的旋转时,能够比对照组模型识别到更多旋转目标,且识别错误更少。

关键词: 低空航拍, 红外图像, 多角度目标识别, 旋转等变卷积

Abstract:

In order to improve the rotation robustness of the traditional UAV infrared target recognition algorithm to the input image, an infrared image target recognition algorithm with rotation equivariant is proposed. An infrared image is expanded into a three-channel image to enrich the details and edge information of the input image by referring to the visible-light three-channel structure. The standard rotation equivariant convolution FBL and rotational residual FSP modules are designed and implemented based on the rotation equivariant convolution, which can highly retain the rotation characteristics of the image, so that the FC-YOLOv5 model is robust to the rotation of the image and the target in the image; The SE attention mechanism is added to learn the importance of each channel adaptively, and the channel contribution in the feature map is weighted according to the needs of the task, so as to extract the important feature information and suppress the unimportant feature information. The performance of the model is verified on APOPV data set and SAS data set, and the benchmark model YOLOv5s and the models YOLOv8s and NanoDet used in common lightweight target recognition tasks are used as the control models. The experimental results show that the mean average precision of the proposed algorithm can be improved by 2%-4% compared to the benchmark model, and when the input image has different rotation angles, the FC-YOLOv5 can recognize more rotating targets with fewer recognition errors than those of the control model.

Key words: low-altitude aerial photography, infrared image, multi-angle target recognition, rotation equivariant convolution

中图分类号:

TP391.41

肖锋, 卢浩, 张文娟, 黄姝娟, 焦雨林, 卢昭廷, 李照山. 基于旋转等变卷积的航拍红外图像目标识别算法[J]. 兵工学报, 2024, 45(8): 2817-2827.

XIAO Feng, LU Hao, ZHANG Wenjuan, HUANG Shujuan, JIAO Yulin, LU Zhaoting, LI Zhaoshan. Aerial Infrared Image Target Recognition Algorithm Based on Rotation Equivariant Convolution[J]. Acta Armamentarii, 2024, 45(8): 2817-2827.

图/表 10

图1 FC-YOLOv5网络结构示意图

Fig.1 Schematic diagram of FC-YOLOv5 network structure

图2 FBL与FSP结构

Fig.2 Structure diagram of FBL and FSP

图3 通道扩展示意图

Fig.3 Schematic diagram of channel expansion

图4 通道扩展评价指标对比

Fig.4 Comparison of channel expansion evaluation indicators

表1 通道扩展效果图

Table 1 Channel expansion rendering

表2 APOPV数据集识别效果

Table 2 APOPV recognition rendering

表3 SAS数据集识别效果

Table 3 SAS recognition rendering

表4 旋转图像识别效果

Table 4 Rotation image recognition rendering

表5 F-Conv效果验证(mAP@0.5)

Table 5 F-Conv effect verification (mAP@0.5)

模型	APOPV数据集	SAS数据集
YOLOv5s	64.28	83.62
FC-YOLOv5	67.04	88.35
NanoDet	64.65	85.39
YOLOv8s	66.81	86.27

表6 消融实验结果

Table 6 Results of ablation experiment

SE注意力机制	通道扩展方法	F-Conv方法	mAP@0.5
			64.28
	√		66.53
		√	67.04
	√	√	66.65
√	√	√	67.86

参考文献 22

[1]	王靖宇, 王霰禹, 张科, 等. 基于深度神经网络的低空弱小无人机目标检测研究[J]. 西北工业大学学报, 2018, 36(2):258-263.
	WANG J Y, WANG X Y, ZHANG K, et al. Research on low altitude weak small unmanned aerial vehicle target detection based on deep neural network[J]. Journal of Northwestern Polytechnical University, 2018, 36(2):258-263. (in Chinese)
[2]	WEILER M, CESA G. General E(2) -equivariant steerable CNNs[C]//Advances in Neural Information Processing Systems 32(NeurIPS 2019). Red Hook, NY, US: Curran Associates, Inc., 2019.
[3]	殷飞, 焦李成. 基于旋转扩展和稀疏表示的鲁棒遥感图像目标识别[J]. 模式识别与人工智能, 2012, 25(1):89-95.
	YIN F, JIAO L C. Robust remote sensing image target recognition based on rotation extension and sparse representation[J]. Pattern Recognition and Artificial Intelligence, 2012, 25(1):89-95. (in Chinese)
[4]	JIANG Y Y, ZHU X Y, WANG X B, et al. R2CNN: rotational region CNN for orientation robust scene text detection: arXiv:1706.09579[R/OL]. Ithaca, NY, US: Cornell University, 2017(2017-06-30). https://arxiv.org/abs/1706.09579.
[5]	XIA G S, DING J, QIAN M, et al. LUAI Challenge 2021 on learning to understand aerial images[C]//Proceedings of 2021 IEEE/CVF International Conference on Computer Vision. Montreal, BC, Canada:IEEE,2021: 762-768.
[6]	NABATI R, QI H R. RRPN: radar region proposal network for object detection in autonomous vehicles[C]//Proceedings of 2019 IEEE International Conference on Image Processing. Taipei, China:IEEE,2019: 3093-3097.
[7]	XIN Y, CHEN D L, ZENG C Y, et al. High throughput hardware/software heterogeneous system for RRPN-based scene text detection[J]. IEEE Transactions on Computers, 2022, 71(7):1507-1521.
[8]	YANG X, YAN J C, FENG Z M, et al. R3det: refined single-stage detector with feature refinement for rotating object[C]// Proceedings of the AAAI conference on artificial intelligence. Washington, DC, US:AAAI,2021: 3163-3171.
[9]	HAN J M, DING J, XUE N, et al. Redet: a rotation-equivariant detector for aerial object detection[C]// Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. Nashville, TN, US:IEEE,2021: 2786-2795.
[10]	XIE X X, CHENG G, WANG J B, et al. Oriented R-CNN for object detection[C]//Proceedings of the IEEE/CVF international conference on computer vision. Montreal, QC, Canada:IEEE,2021: 3500-3509.
[11]	COHEN T S, GEIGER M, WEILER M. A general theory of equivariant CNNs on homogeneous spaces[C]// Proceedings of the 33rd International Conference on Neural Information Processing Systems. Vancouver, Canada: Neural Information Processing Systems Foundation, Inc.,2019: 9145-9156.
[12]	WEILER M, HAMPRECHT F A, STORATH M. Learning steerable filters for rotation equivariant CNNs[C]// Proceedings of 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Salt Lake City, UT, US: Neural Information Processing Systems Foundation, Inc., 2018: 849-858.
[13]	SHEN Z Y, HE L S, LIN Z C, et al. PDO-eConvs: partial differential operator based equivariant convolutions[C]// Proceedings of the 37th International Conference on Machine Learning. Vienna, Austria:ICML,2020:8697-8706.
[14]	XIE Q, ZHAO Q, XU Z B, et al. Fourier series expansion based filter parametrization for equivariant convolutions[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2022, 45(4): 4537-4551.
[15]	于博文, 吕明. 改进的YOLOv3算法及其在军事目标检测中的应用[J]. 兵工学报, 2022, 43(2): 345-354. doi: 10.3969/j.issn.1000-1093.2022.02.012
	YU B W, LÜ M. Improved YOLOv3 algorithm and its application in military target detection[J]. Acta Armamentarii, 2022, 43(2): 345-354. (in Chinese) doi: 10.3969/j.issn.1000-1093.2022.02.012
[16]	张博尧, 冷雁冰. 基于YOLOv4网络模型的金属表面划痕检测[J]. 兵工学报, 2022, 43(增刊1): 214-221.
	ZHANG B Y, LENG Y B. Detection of metal surface scratch based on YOLOv4 network model[J]. Acta Armamentarii, 2022, 43(S1): 214-221. (in Chinese) doi: 10.12382/bgxb.2022.A011
[17]	陈旭, 彭冬亮, 谷雨. 基于改进 YOLOv5s 的无人机图像实时目标检测[J]. 光电工程, 2022, 49(3): 210372.
	CHEN X, PENG D L, GU Y. Real time object detection of unmanned aerial vehicle images based on improved YOLOv5s[J]. Opto-Electronic Engineering, 2022, 49(3): 210372. (in Chinese)
[18]	HU J, SHEN L, SUN G. Squeeze-and-excitation networks[C]//Proceedings of the 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Salt Lake City, UT, US:IEEE, 2018: 7132-7141.
[19]	张汝榛, 张建林, 祁小平, 等. 复杂场景下的红外目标检测[J]. 光电工程, 2020, 47(10):128-137.
	ZHANG R Z, ZHANG J L, QI X P, et al. Infrared target detection in complex scenes[J]. Opto-Electronic Engineering, 2020, 47(10):128-137. (in Chinese)
[20]	GAO W S, ZHANG X G, YANG L, et al. An improved Sobel edge detection[C]//Proceedings of the 2010 3rd International conference on computer science and information technology. Chengdu, China:IEEE, 2010: 67-71.
[21]	WANG Z, GUO J X, ZHANG C L, et al. Multiscale feature enhancement network for salient object detection in optical remote sensing images[J]. IEEE Transactions on Geoscience and Remote Sensing, 2022, 60: 5634819.
[22]	王亮, 陈建华, 李烨. 一种基于深度学习的无人艇海上目标识别技术[J]. 兵工学报, 2022, 43(增刊2): 13-19.
	WANG L, CHEN J H, LI Y. A target identification technique for unmanned surface vessel based on deep learning[J]. Acta Armamentarii, 2022, 43(S2): 13-19. (in Chinese) doi: 10.12382/bgxb.2022.B021