Aerial Infrared Image Target Recognition Algorithm Based on Rotation Equivariant Convolution

doi:10.12382/bgxb.2023.0503

Abstract

Abstract:

In order to improve the rotation robustness of the traditional UAV infrared target recognition algorithm to the input image, an infrared image target recognition algorithm with rotation equivariant is proposed. An infrared image is expanded into a three-channel image to enrich the details and edge information of the input image by referring to the visible-light three-channel structure. The standard rotation equivariant convolution FBL and rotational residual FSP modules are designed and implemented based on the rotation equivariant convolution, which can highly retain the rotation characteristics of the image, so that the FC-YOLOv5 model is robust to the rotation of the image and the target in the image; The SE attention mechanism is added to learn the importance of each channel adaptively, and the channel contribution in the feature map is weighted according to the needs of the task, so as to extract the important feature information and suppress the unimportant feature information. The performance of the model is verified on APOPV data set and SAS data set, and the benchmark model YOLOv5s and the models YOLOv8s and NanoDet used in common lightweight target recognition tasks are used as the control models. The experimental results show that the mean average precision of the proposed algorithm can be improved by 2%-4% compared to the benchmark model, and when the input image has different rotation angles, the FC-YOLOv5 can recognize more rotating targets with fewer recognition errors than those of the control model.

Key words: low-altitude aerial photography, infrared image, multi-angle target recognition, rotation equivariant convolution

CLC Number:

TP391.41

XIAO Feng, LU Hao, ZHANG Wenjuan, HUANG Shujuan, JIAO Yulin, LU Zhaoting, LI Zhaoshan. Aerial Infrared Image Target Recognition Algorithm Based on Rotation Equivariant Convolution[J]. Acta Armamentarii, 2024, 45(8): 2817-2827.

Figures/Tables 10

Fig.1 Schematic diagram of FC-YOLOv5 network structure

Fig.2 Structure diagram of FBL and FSP

Fig.3 Schematic diagram of channel expansion

Fig.4 Comparison of channel expansion evaluation indicators

Table 1 Channel expansion rendering

Table 2 APOPV recognition rendering

Table 3 SAS recognition rendering

Table 4 Rotation image recognition rendering

Table 5 F-Conv effect verification (mAP@0.5)

模型	APOPV数据集	SAS数据集
YOLOv5s	64.28	83.62
FC-YOLOv5	67.04	88.35
NanoDet	64.65	85.39
YOLOv8s	66.81	86.27

Table 6 Results of ablation experiment

SE注意力机制	通道扩展方法	F-Conv方法	mAP@0.5
			64.28
	√		66.53
		√	67.04
	√	√	66.65
√	√	√	67.86

References 22

[1]	王靖宇, 王霰禹, 张科, 等. 基于深度神经网络的低空弱小无人机目标检测研究[J]. 西北工业大学学报, 2018, 36(2):258-263.
	WANG J Y, WANG X Y, ZHANG K, et al. Research on low altitude weak small unmanned aerial vehicle target detection based on deep neural network[J]. Journal of Northwestern Polytechnical University, 2018, 36(2):258-263. (in Chinese)
[2]	WEILER M, CESA G. General E(2) -equivariant steerable CNNs[C]//Advances in Neural Information Processing Systems 32(NeurIPS 2019). Red Hook, NY, US: Curran Associates, Inc., 2019.
[3]	殷飞, 焦李成. 基于旋转扩展和稀疏表示的鲁棒遥感图像目标识别[J]. 模式识别与人工智能, 2012, 25(1):89-95.
	YIN F, JIAO L C. Robust remote sensing image target recognition based on rotation extension and sparse representation[J]. Pattern Recognition and Artificial Intelligence, 2012, 25(1):89-95. (in Chinese)
[4]	JIANG Y Y, ZHU X Y, WANG X B, et al. R2CNN: rotational region CNN for orientation robust scene text detection: arXiv:1706.09579[R/OL]. Ithaca, NY, US: Cornell University, 2017(2017-06-30). https://arxiv.org/abs/1706.09579.
[5]	XIA G S, DING J, QIAN M, et al. LUAI Challenge 2021 on learning to understand aerial images[C]//Proceedings of 2021 IEEE/CVF International Conference on Computer Vision. Montreal, BC, Canada:IEEE,2021: 762-768.
[6]	NABATI R, QI H R. RRPN: radar region proposal network for object detection in autonomous vehicles[C]//Proceedings of 2019 IEEE International Conference on Image Processing. Taipei, China:IEEE,2019: 3093-3097.
[7]	XIN Y, CHEN D L, ZENG C Y, et al. High throughput hardware/software heterogeneous system for RRPN-based scene text detection[J]. IEEE Transactions on Computers, 2022, 71(7):1507-1521.
[8]	YANG X, YAN J C, FENG Z M, et al. R3det: refined single-stage detector with feature refinement for rotating object[C]// Proceedings of the AAAI conference on artificial intelligence. Washington, DC, US:AAAI,2021: 3163-3171.
[9]	HAN J M, DING J, XUE N, et al. Redet: a rotation-equivariant detector for aerial object detection[C]// Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. Nashville, TN, US:IEEE,2021: 2786-2795.
[10]	XIE X X, CHENG G, WANG J B, et al. Oriented R-CNN for object detection[C]//Proceedings of the IEEE/CVF international conference on computer vision. Montreal, QC, Canada:IEEE,2021: 3500-3509.
[11]	COHEN T S, GEIGER M, WEILER M. A general theory of equivariant CNNs on homogeneous spaces[C]// Proceedings of the 33rd International Conference on Neural Information Processing Systems. Vancouver, Canada: Neural Information Processing Systems Foundation, Inc.,2019: 9145-9156.
[12]	WEILER M, HAMPRECHT F A, STORATH M. Learning steerable filters for rotation equivariant CNNs[C]// Proceedings of 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Salt Lake City, UT, US: Neural Information Processing Systems Foundation, Inc., 2018: 849-858.
[13]	SHEN Z Y, HE L S, LIN Z C, et al. PDO-eConvs: partial differential operator based equivariant convolutions[C]// Proceedings of the 37th International Conference on Machine Learning. Vienna, Austria:ICML,2020:8697-8706.
[14]	XIE Q, ZHAO Q, XU Z B, et al. Fourier series expansion based filter parametrization for equivariant convolutions[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2022, 45(4): 4537-4551.
[15]	于博文, 吕明. 改进的YOLOv3算法及其在军事目标检测中的应用[J]. 兵工学报, 2022, 43(2): 345-354. doi: 10.3969/j.issn.1000-1093.2022.02.012
	YU B W, LÜ M. Improved YOLOv3 algorithm and its application in military target detection[J]. Acta Armamentarii, 2022, 43(2): 345-354. (in Chinese) doi: 10.3969/j.issn.1000-1093.2022.02.012
[16]	张博尧, 冷雁冰. 基于YOLOv4网络模型的金属表面划痕检测[J]. 兵工学报, 2022, 43(增刊1): 214-221.
	ZHANG B Y, LENG Y B. Detection of metal surface scratch based on YOLOv4 network model[J]. Acta Armamentarii, 2022, 43(S1): 214-221. (in Chinese) doi: 10.12382/bgxb.2022.A011
[17]	陈旭, 彭冬亮, 谷雨. 基于改进 YOLOv5s 的无人机图像实时目标检测[J]. 光电工程, 2022, 49(3): 210372.
	CHEN X, PENG D L, GU Y. Real time object detection of unmanned aerial vehicle images based on improved YOLOv5s[J]. Opto-Electronic Engineering, 2022, 49(3): 210372. (in Chinese)
[18]	HU J, SHEN L, SUN G. Squeeze-and-excitation networks[C]//Proceedings of the 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Salt Lake City, UT, US:IEEE, 2018: 7132-7141.
[19]	张汝榛, 张建林, 祁小平, 等. 复杂场景下的红外目标检测[J]. 光电工程, 2020, 47(10):128-137.
	ZHANG R Z, ZHANG J L, QI X P, et al. Infrared target detection in complex scenes[J]. Opto-Electronic Engineering, 2020, 47(10):128-137. (in Chinese)
[20]	GAO W S, ZHANG X G, YANG L, et al. An improved Sobel edge detection[C]//Proceedings of the 2010 3rd International conference on computer science and information technology. Chengdu, China:IEEE, 2010: 67-71.
[21]	WANG Z, GUO J X, ZHANG C L, et al. Multiscale feature enhancement network for salient object detection in optical remote sensing images[J]. IEEE Transactions on Geoscience and Remote Sensing, 2022, 60: 5634819.
[22]	王亮, 陈建华, 李烨. 一种基于深度学习的无人艇海上目标识别技术[J]. 兵工学报, 2022, 43(增刊2): 13-19.
	WANG L, CHEN J H, LI Y. A target identification technique for unmanned surface vessel based on deep learning[J]. Acta Armamentarii, 2022, 43(S2): 13-19. (in Chinese) doi: 10.12382/bgxb.2022.B021