一种坐标通道注意力深度学习网络的军用飞机识别方法

doi:10.12382/bgxb.2023.0526

摘要/Abstract

摘要：

战场态势瞬息万变,利用可见光图像对敌方用于军事行动的飞机类型进行有效区分,对提供军事作战信息具有重要意义。针对现有军用飞机识别方法存在小目标飞机和环境背景复杂导致的模型特征提取困难、数据样本数量不足导致的模型训练不充分的问题,提出一种坐标通道注意力(ConvNeXt-Coordinate Attention,ConvNeXt-CA)深度学习网络军用飞机目标识别方法。该方法在ConvNeXt网络可以保留小目标飞机特征的基础上,引入CA机制设计CA-Stage模块,提升网络对于背景和前景的区分能力;采用数据增强的方式扩充数据集,以及使用迁移学习的策略提高模型的泛化能力,训练得到具备最优超参数的ConvNeXt-CA网络。实验结果表明,与传统的军用飞机识别方法和其他深度学习模型相比,基于迁移学习的ConvNeXt-CA网络在预测准确率上有明显的提升,且具备较强的泛化能力。

关键词: 军用飞机识别, 深度卷积神经网络, 坐标注意力机制, 迁移学习

Abstract:

The military combat information can be provided by using the visible images to effectively distinguish the types of enemy aircrafts used for military operations in a rapidly changing battlefield environment. To address the challenges associated with extracting the model features from small aircraft targets and complex environmental backgrounds, as well as the limited training data available in existing military aircraft recognition methods, this paper proposes a ConvNeXt with coordinate attention(ConvNeXt-CA)-based recognition method for military aircraft targets. Based on the fact that the ConvNeXt-CA network can retain the characteristics of small aircrafttargets, the proposed method introduces the CA mechanism to design a CA-Stage module, which improves the ability of the network to distinguish between background and foreground.It uses data augmentation to expand the data set and the migration learning strategy to improve the generalization capability of the model, and trains the ConvNeXt-CA network with optimal hyperparameters.The experimental results show that, compared with the traditional military aircraft identification methods and other deep learning models, the migration learning-based ConvNeXt-CA network has a significantly improved prediction accuracy and a strong generalization capability.

Key words: military aircraft recognition, deep convolutional neural network, coordinate attention mechanism, migration learning

中图分类号:

TJ85

杨环宇, 王军, 吴祥, 薄煜明, 马立丰, 陆金磊. 一种坐标通道注意力深度学习网络的军用飞机识别方法[J]. 兵工学报, 2024, 45(7): 2128-2143.

YANG Huanyu, WANG Jun, WU Xiang, BO Yuming, MA Lifeng, LU Jinlei. A Method for Military Aircraft Recognition Using a Coordinate Attention-based Deep Learning Network[J]. Acta Armamentarii, 2024, 45(7): 2128-2143.

图/表 21

图1 ConvNeXt网络结构图

Fig.1 Structure diagram of ConvNeXt network

图2 改进后的ConvNeXt-CA网络结构图

Fig.2 Structure diagram of the improved ConvNeXt-CA network

图3 坐标注意力模块示意图

Fig.3 Schematic diagram of coordinate attention module

图4 采用迁移学习策略的ConvNeXt-CA网络流程结构图

Fig.4 Flow chart of ConvNeXt-CA network using migration learning strategy

图5 模型训练及预测整体流程图

Fig.5 Overall flow chart of model training and prediction

图6 不同型号军用飞机的光学图像

Fig.6 Images of different types of military aircraft

图7 存在遮挡的军用飞机图像

Fig.7 Obscured images of military aircrafts

图8 小目标军用飞机图像

Fig.8 Images of small target military aircrafts

图9 额外添加的军用飞机图像

Fig.9 Additional images of military aircrafts

表1 不同模型的最佳准确率

Table 1 Best accuracies of different models

算法模型	预测准确度/%	epoch
HOG+SVM	68.6	1
ResNet50	90.4	20
ResNet50-CBAM	89.8	20
ResNet50-CA	91.5	20
Swin-T	92.0	20
VAN-B2	94.3	20
ConvNeXt	93.5	20
ConvNeXt-CBAM	94.7	20
ConvNeXt-CA-a	94.8	20
ConvNeXt-CA	96.0	20

图10 不同模型的损失值、准确率变化曲线

Fig.10 Loss value and accuracy curves of different models

表2 混淆矩阵基础指标关系

Table 2 Confusion matrix underlying index relation

混淆矩阵		真实值
混淆矩阵		正样本	负样本
预测值	正样本	TP	FP
	负样本	FN	TN

表3 不同模型的评价指标数值

Table 3 The evaluation index values of different models

算法模型	精确率/%	召回率/%	特异度/%
ResNet50	91.04	90.13	99.70
ResNet50-CBAM	90.97	89.19	99.69
ResNet50-CA	91.87	91.01	99.73
Swin-T	92.76	91.31	99.76
ConvNeXt	93.71	92.75	99.80
ConvNeXt-CBAM	95.73	93.86	99.83
ConvNeXt-CA-a	95.15	94.34	99.84
ConvNeXt-CA	96.12	95.33	99.88

图11 ConvNeXt混淆矩阵可视图

Fig.11 ConvNeXt confusion matrix

图12 ConvNeXt-CA混淆矩阵可视图

Fig.12 ConvNeXt-CA confusion matrix

图13 不同模型的热力图(左为不同模型识别F35飞机时的注意力热力图,中为C17,右为EF2000)

Fig.13 Heat maps of different models(left: the attention heatmaps of different models when recognizing F35 aircraft, middle: C17, and right: EF2000)

表4 不同方法的准确率和运行时间

Table 4 Recognition accuraciesandrun times of different methods

算法模型	准确度/%	运行时间/s
ConvNeXt	93.5	4.60
ConvNeXt+CA	96	4.62

表5 不同方法的准确率

Table 5 The accuracies of different methods

算法模型	训练准确度/%	epoch
ResNet50+迁移学习	86.1	20
ResNet50随机初始化	70.4	20
ConvNeXt+迁移学习	94.4	20
ConvNeXt随机初始化	83.1	20

图14 不同方法的训练识别准确率示意图

Fig.14 Schematic diagram of training recognition accuracies of different methods

表6 不同改进方法对军用飞机图像识别性能的影响

Table 6 Effects of different improved methods on military aircraft image recognition results

算法模型	迁移学习	CA	Epoch	预测准确度/%
			20	82.6
ConvNeXt	P		20	93.5
		P	20	84.2
	P	P	20	96.0

图15 部分识别结果展示

Fig.15 Display of partial recognition results

参考文献 28

[1]	张永梅, 赖裕平, 马健喆, 等. 基于视频的装甲车和飞机检测跟踪及轨迹预测算法[J]. 兵工学报, 2021, 42(3):545-554. doi: 10.3969/j.issn.1000-1093.2021.03.010
	ZHANG Y M, LAI Y P, MA J Z, et al. A video-based prediction algorithm for armored vehicle and aircraft detection/tracking and trajectory[J]. Acta Armamentarii, 2021, 42(3):545-554. (in Chinese) doi: 10.3969/j.issn.1000-1093.2021.03.010
[2]	ZHAO B J, TANG W, PAN Y, et al. Aircraft type recognition in remote sensing images:bilinear discriminative extreme learning machine framework[J]. Electronics, 2021, 10(17):2046.
[3]	王铁虎, 焦爱泉, 冯连仲, 等. 精确打击作战与装甲装备未来发展[J]. 兵工学报, 2010, 31(增刊2):61-67.
	WANG T H, JIAO A Q, FENG L Z, et al. Futuredevelopment of armored equipment and precise attack operation[J]. Acta Armamentarii, 2010, 31(S2):61-67. (in Chinese)
[4]	李明悦, 何乐生, 雷晨, 等. 基于注意力特征融合的SqueezeNet细粒度图像分类模型[J]. 云南大学学报(自然科学版), 2021, 43(5):868-876.
	LI M Y, HE L S, LEI C, et al. Fine-grained image classification model based onattention feature fusion with SqueezeNet[J]. Journal of Yunnan University(Natural Sciences Edition), 2021, 43(5):868-876. (in Chinese)
[5]	龙怡灿, 雷蓉, 董杨, 等. 基于YOLOv5算法的飞机类型光学遥感识别[J]. 地球信息科学学报, 2022, 24(3):572-582. doi: 10.12082/dqxxkx.2022.210369
	LONG Y C, LEI R, DONG Y, et al. YOLOv5 based on aircraft type detection from remotely sensed optical images[J]. Journal of Geo-information Science, 2022, 24(3):572-582. (in Chinese)
[6]	王娜, 胡超芳. 基于客观聚类的手写数字识别方法[J]. 复杂系统与复杂性科学, 2019, 16(2):77-84,94.
	WANG N, HU C F. A Handwriting digital recognition method based on enhanced objective cluster analysis[J]. Complex Systems and Complexity Science, 2019, 16(2):77-84,94. (in Chinese)
[7]	DALAL N, TRIGGS B. Histograms of oriented gradients for human detection[C]// Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. San Diego,CA, US:IEEE, 2005: 886-893.
[8]	LOWE D G. Distinctive image features from scale-invariantkeypoints[J]. International Journal of Computer Vision, 2004, 60(2):91-110.
[9]	ELENE F O, GABRIEL M B, JOAO V S D C, et al. Automatic detection of COVID-19 infection using chest X-Ray images through transfer learning[J]. IEEE/CAA Journal of Automatica Sinica, 2021, 8(1):239-248.
[10]	张宪法, 郝矿荣, 陈磊. 免疫多域特征融合的多核学习SVM运动想象脑电信号分类[J]. 自动化学报, 2020, 46(11):2417-2426.
	ZHANG X F, HAO K R, CHEN L. Motor imagery EEG classification based on immune multi-domain-featurefusion and multiple kernel learning SVM[J]. Acta Automatica Sinica, 2020, 46(11):2417-2426. (in Chinese)
[11]	GARCIA-LUCAS D, CEBRIAN-MARQUEZ G, DIAZ-HONRUBIA A J, et al. A fast full partitioning algorithm for HEVC-to-VVC video transcoding using Bayesian classifiers[J]. Journal of Visual Communication and Image Representation, 2023, 94,103829.DOI:10.1016/J.JVCIR.2023.103829.
[12]	XU Y Z, YU G Z, WANG Y P, et al. A hybrid vehicle detection method based on viola-jones and HOG+SVM from UAV images[J]. Sensors, 2016, 16(8):1325.
[13]	李泽东, 李志农, 陶俊勇, 等. 基于特征融合的注意力增强卷积神经网络的航空发动机滚动轴承故障诊断方法[J]. 兵工学报, 2022, 43(12):3228-3239.
	LI Z D, LI Z N, TAO J Y, et al. Fault diagnosis for aero-engine rolling bearings based on an attention augmented convolutional neural network with feature fusion[J]. Acta Armamentarii, 2022, 43(12):3228-3239. (in Chinese)
[14]	张良安, 陈洋, 谢胜龙, 等. 基于机器视觉与深度学习的飞机防护栅裂纹检测系统[J]. 兵工学报, 2023, 44(2):507-516. doi: 10.12382/bgxb.2021.0674
	ZHANG L A, CHEN Y, XIE S L, et al. Crack detection system for aircraft protective grill based on machine vision and deep learning[J]. Acta Armamentarii, 2023, 44(2):507-516. (in Chinese) doi: 10.12382/bgxb.2021.0674
[15]	刘秋, 孙晋伟, 张华, 等. 基于卷积神经网络的路面识别及半主动悬架控制[J]. 兵工学报, 2020, 41(8):1483-1493. doi: 10.3969/j.issn.1000-1093.2020.08.002
	LIU Q, SUN J W, ZHANG H, et al. Road identification and semi-active suspension control based on convolutional neural network[J]. Acta Armamentarii, 2020, 41(8):1483-1493. (in Chinese) doi: 10.3969/j.issn.1000-1093.2020.08.002
[16]	兰天, 李博, 杨敬宝. 基于小样本多背景下的飞机图像识别研究[J]. 电脑编程技巧与维护, 2020(4):142-144.
	LAN T, LI B, YANG J B. Research on aircraft image recognition based on small sample multi-background[J]. Computer Programming Skills & Maintenance, 2020(4):142-144. (in Chinese)
[17]	张文达, 许悦雷, 倪嘉成, 等. 基于多尺度分块卷积神经网络的图像目标识别算法[J]. 计算机应用, 2016, 36(4):1033-1038. doi: 10.11772/j.issn.1001-9081.2016.04.1033
	ZHANG W D, XU Y L, NI J C, et al. Image target recognition method based on multi-scale block convolutional neural network[J]. Journal of Computer Applications, 2016, 36(4):1033-1038. (in Chinese) doi: 10.11772/j.issn.1001-9081.2016.04.1033
[18]	HE K M, ZHANG X Y, REN S Q, et al. Deep residual learning for image recognition[C]//Proceedings of IEEE Conference on Computer Vision and Pattern Recognition. Washington, D.C., US: IEEE Computer Society, 2016: 770-778.
[19]	HUANG G, LIU Z, VAN D M L, et al. Densely connected convolutional networks[C]//Proceedings of IEEE Conference on Computer Vision and Pattern Recognition. Honolulu, HI, US: IEEE Computer Society, 2017:2261-2269.
[20]	LIU Z, MAO H, WU C Y, et al. A convnet for the 2020s[C]// Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. New Orleans, LA, US:IEEE, 2022:11966-11976.
[21]	XIONG R B, YANG Y C, HE D, et al. On layer normalization in the transformer architecture[C]// Proceedings of International Conference on Machine Learning. Online:IMLS, 2020:10524-10533.
[22]	LI Z, WU C, HAN Q, et al. CASI-net: a novel and effect steel surface defect classification method based on coordinate attention and self-interaction mechanism[J]. Mathematics, 2022, 10(6):963.
[23]	CHE C C, WANG H W, XIONG M L, et al. Few-shot fatigue damage evaluation of aircraft structure using neural augmentation and deep transfer learning[J]. Engineering Failure Analysis, 2023, 148:107185.
[24]	ZHANG W L, WANG N, CHEN K Z, et al. A pruning method for deep convolutional network based on heat map generation metrics[J]. Sensors, 2022, 22(5):2022.DOI:10.3390/S22052022.
[25]	LIU Z, LIN Y T, CAO Y, et al. Swin transformer:hierarchical vision transformer using shifted windows[C]// Proceedings of the IEEE/CVF International Conference on Computer Vision.Washington,D.C., US:IEEE, 2021:10012-10022.
[26]	HU J, SHEN L, SUN G. Squeeze-and-excitation networks[C]// Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Salt Lake City, UT, US:IEEE, 2018:7132-7141.
[27]	CHEN Y T, ZHANG X Z, CHEN W N, et al. Research on recognition of fly species based on improved retinaNet and CBAM[J]. IEEE Access, 2020, 8,102907-102919.DOI:10.1109/access.2020.2997466.
[28]	GUO M H, LU C Z, LIU Z N, et al. Visual attention network:arXiv:2202.09741[R]. Ithaca,NY, US: Cornell University, 2022:2202.09741.