基于集成迁移学习的新装备装甲车辆分类

doi:10.12382/bgxb.2022.0412

摘要/Abstract

摘要：

在复杂的陆战环境中,图像分类技术是快速区分装甲车辆目标的一种重要手段。针对现有基于卷积神经网络(CNN)的主流分类算法对于训练样本的数量及质量有较高要求,在新装备装甲车辆图像分类任务中精度不足的问题,提出一种集成了两个基于不同学习策略的CNN的迁移学习方法。一个CNN在图像样本较易获取、数量充足的老式装甲车辆图像数据集上进行预训练,学习局部细节特征;另一个CNN在图像质量较低的新装备装甲车辆的虚拟图像数据集上进行预训练,学习全局特征。对预训练好的CNN均利用数量有限的新装备装甲车辆真实样本按照不同策略微调,提升表征能力。设计基于Optuna超参数优化框架的自学习模型集成机制,可对两个CNN的输出进行自主加权优化,进一步提高算法的分类准确率。实验结果表明,与随机初始化训练的同一模型相比,所提方法在新装备装甲车辆图像分类任务中准确率提高7%,有效缓解了训练样本偏少的问题。

关键词: 迁移学习, 卷积神经网络, 装甲车辆分类, 特征提取, 模型集成机制

Abstract:

In complicated land warfare environments, image classification techniques is an important tool to quickly distinguish armored vehicle targets. To address the problem that the existing mainstream classification algorithms based on Convolutional Neural Network (CNN) have high requirements for the number and quality of training samples and perform with insufficient accuracy in the image classification task of newly equipped armored vehicles, a transfer learning method that integrates two CNNs based on different learning strategies is proposed. Specifically, one CNN is pre-trained on an old-fashioned armored vehicle image dataset whose samples can be easily obtained and have sufficient quantity to learn local detail features. The other CNN is pre-trained on the dataset of virtual images of the newly equipped armored vehicles with a low image quality to learn the global features. The pre-trained CNNs are all fine-tuned according to different strategies using a limited number of real samples of newly equipped armored vehicles to improve the characterization capability. A self-learning model integration mechanism based on the Optuna hyperparametric optimization framework is designed, which can autonomously weight the outputs of the two CNNs for optimization and further improve the classification accuracy of the algorithm. The experimental results show that the accuracy of the proposed algorithm is improved by 7% in the image classification task of newly equipped armored vehicles compared with the same model trained from scratch, which effectively alleviates the problem of insufficient training samples.

Key words: transfer learning, convolutional neural network, armored vehicles, feature extraction, model integration mechanism

中图分类号:

TP181

刘懿, 任济寰, 吴祥, 薄煜明. 基于集成迁移学习的新装备装甲车辆分类[J]. 兵工学报, 2023, 44(8): 2319-2328.

LIU Yi, REN Jihuan, WU Xiang, BO Yuming. Newly Equipped Armored Vehicle Classification Based on Integrated Transfer Learning[J]. Acta Armamentarii, 2023, 44(8): 2319-2328.

图/表 8

图1 基于集成迁移学习的图像分类方法流程图

Fig.1 Flow chart of the proposed method

图2 ResNet-18网络结构的示意图

Fig.2 Schematic diagram of the ResNet-18 structure

表1 ResNet-18网络结构

Table 1 Network structure of the ResNet-18

模块名	输出尺寸	构成
卷积模块1	56×56	7×7×64,3×3 池化
残差模块1	56×56	3×3×64、3×3×64堆叠2次
残差模块2	28×28	3×3×128、3×3×128堆叠2次
残差模块3	14×14	3×3×256、3×3×256堆叠2次
残差模块4	7×7	3×3×512、3×3×512堆叠2次
输出层	1×1	均值池化, 全连接层, SoftMax激活函数

图3 数据集类别组成

Fig.3 Schematic diagram of category composition of datasets

表2 数据集的介绍

Table 2 Introduction to the datasets

数据集	总样本数	训练样本数	测试样本数	每类样本数
D₁	1000	800	200	200
D₂	500	300	200	100
$D ¯ 2$	250	40	10	50

表2 数据集的介绍

Table 2 Introduction to the datasets

数据集	总样本数	训练样本数	测试样本数	每类样本数
D₁	1000	800	200	200
D₂	500	300	200	100
$D ¯ 2$	250	40	10	50

表3 实验结果

Table 3 Experimental results

分类方法	测试数据集正确率
分类方法	30张D₂学习样本	60张D₂学习样本
HOG+SVM	54	62
深度学习	60	74
迁移学习 (D₁)	63	74
迁移学习 ( $D ¯ 2$ )	64	74
本文方法	67	76.5

表3 实验结果

Table 3 Experimental results

分类方法	测试数据集正确率
分类方法	30张D₂学习样本	60张D₂学习样本
HOG+SVM	54	62
深度学习	60	74
迁移学习 (D₁)	63	74
迁移学习 ( $D ¯ 2$ )	64	74
本文方法	67	76.5

图4 两种方法的分类准确率均值和标准差示意图

Fig.4 Schematic diagram of classification accuracy and standard deviation of accuracy for the two methods

表4 全局结构提取器和局部特征提取器网络注意力结果

Table 4 Attention results of global structure extractor and local feature extractor

参考文献 35

[1]	孙皓泽, 常天庆, 王全东, 等. 一种基于分层多尺度卷积特征提取的坦克装甲目标图像检测方法[J]. 兵工学报, 2017, 38(9): 1681-1691. doi: 10.3969/j.issn.1000-1093.2017.09.003
	SUN H Z, CHANG T Q, WANG Q D, et al. Image detection method for tank and armored targets based on hierarchical multi-scale convolution feature extraction[J]. Acta Armamentarii, 2017, 38(9): 1681-1691. (in Chinese)
[2]	LOWE D G. Distinctive image features from scale-invariant keypoints[J]. International Journal of Computer Vision, 2004, 60(2): 91-110. doi: 10.1023/B:VISI.0000029664.99615.94 URL
[3]	BAY H, TUYTELAARS T, GOOL L V. SURF: speeded up robust features[C]//Proceedings of the European Conference on Computer Vision. Graz, Austria: Springer, 2006: 404-417.
[4]	DALAL N, TRIGGS B. Histograms of oriented gradients for human detection[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. San Diego, CA, US: IEEE, 2005: 886-893.
[5]	VAPNIK V N, LERNER A Y. Recognition of patterns with help of generalized portraits[J]. Avtomatika i Telemekhanika, 1963, 24(6): 774-780. (in Russian)
[6]	张珂, 冯晓晗, 郭玉荣, 等. 图像分类的深度卷积神经网络模型综述[J]. 中国图象图形学报, 2021, 26(10): 2305-2325.
	ZHANG K, FENG X H, GUO Y R, et al. Overview of deep convolutional neural networks for image classification[J]. Journal of Image and Graphics, 2021, 26(10): 2305-2325. (in Chinese)
[7]	普运伟, 刘涛涛, 郭江, 等. 基于卷积神经网络和模糊函数主脊坐标变换的雷达辐射源信号识别[J]. 兵工学报, 2021, 42(8): 1680-1689. doi: 10.3969/j.issn.1000-1093.2021.08.012
	PU Y W, LIU T T, GUO J, et al. Radar emitter signal recognition based on convolutional neural network and coordinate transformation of ambiguity function main ridge[J]. Acta Armamentarii, 2021, 42(8): 1680-1689. (in Chinese)
[8]	呙鹏程, 吴礼洋. 融合卷积特征与判别字典学习的低截获概率雷达信号识别[J]. 兵工学报, 2019, 40(9): 1881-1889. doi: 10.3969/j.issn.1000-1093.2019.09.013
	GUO P C, WU L Y. LPI radar signal recognition with convolution feature and discrimination dictionary learning[J]. Acta Armamentarii, 2019, 40(9): 1881-1889. (in Chinese) doi: 10.3969/j.issn.1000-1093.2019.09.013
[9]	刘秋, 孙晋伟, 张华, 等. 基于卷积神经网络的路面识别及半主动悬架控制[J]. 兵工学报, 2020, 41(8): 1483-1493. doi: 10.3969/j.issn.1000-1093.2020.08.002
	LIU Q, SUN J W, ZHANG H, et al. Road identification and semi-active suspension control based on convolutional neural network[J]. Acta Armamentarii, 2020, 41(8): 1483-1493. (in Chinese) doi: 10.3969/j.issn.1000-1093.2020.08.002
[10]	KRIZHEVSKY A, SUTSKEVER I, HINTON G E. ImageNet classification with deep convolutional neural networks[J]. Communications of the ACM, 2017, 60(6):84-90 doi: 10.1145/3065386 URL
[11]	KOLESNIKOV A, BEYER L, ZHAI X, et al. Big transfer (bit): General visual representation learning[C]//Proceedings of the European Conference on Computer Vision. Edinburgh, England, UK: Springer, 2020: 491-507.
[12]	穆思奇, 林进健, 汪海泉, 等. 基于改进YOLOv4的X射线图像违禁品检测算法[J]. 兵工学报, 2021, 42(12): 2675-2683. doi: 10.3969/j.issn.1000-1093.2021.12.015
	MU S Q, LIN J J, WANG H Q, et al. An algorithm for detection of prohibited items in X-ray images based on improved YOLOv4[J]. Acta Armamentarii, 2021, 42(12): 2675-2683. (in Chinese) doi: 10.3969/j.issn.1000-1093.2021.12.015
[13]	吴鸿昊, 王立国, 石瑶. 高光谱图像小样本分类的卷积神经网络方法[J]. 中国图象图形学报, 2021, 26(8): 2009-2020.
	WU H H, WANG L G, SHI Y. Convolution neural network method for small-sample classification of hyperspectral images[J]. Journal of Image and Graphics, 2021, 26(8): 2009-2020. (in Chinese)
[14]	疏颖, 毛龙彪, 陈思, 等. 结合自监督学习和生成对抗网络的小样本人脸属性识别[J]. 中国图象图形学报, 2020, 25(11): 2391-2403.
	SHU Y, MAO L B, CHEN S, et al. Self-supervised learning and generative adversarial network-based facial attribute recognition with small sample size training[J]. Journal of Image and Graphics, 2020, 25(11): 2391-2403. (in Chinese)
[15]	戴宏, 郝轩廷, 盛立杰, 等. 面向小样本约束的域适应分类算法[J]. 计算机学报, 2022, 45(5):935-950
	DAI H, HAO X T, SHENG L J, et al. Domain adaptation algorithm for few-shot classification task[J]. Chinese Journal of Computers, 2022, 45(5): 935-950. (in Chinese)
[16]	GOODFELLOW I, POUGET-ABADIE J, MIRZA M, et al. Generative adversarial networks[J]. Communications of the ACM, 2020, 63(11): 139-144. doi: 10.1145/3422622 URL
[17]	黎英, 宋佩华. 迁移学习在医学图像分类中的研究进展[J]. 中国图象图形学报, 2022, 27(3): 672-686.
	LI Y, SONG P H. Review of transfer learning in medical image classification[J]. Journal of Image and Graphics, 2022, 27(3): 672-686. (in Chinese)
[18]	TAJBAKHSH N, SHIN J Y, GURUDU S R, et al. Convolutional neural networks for medical image analysis: Full training or fine tuning?[J]. IEEE Transactions on Medical Imaging, 2016, 35(5): 1299-1312. doi: 10.1109/TMI.2016.2535302 pmid: 26978662
[19]	ZEBARI R, ABDULAZEEZ A, ZEEBAREE D, et al. A comprehensive review of dimensionality reduction techniques for feature selection and feature extraction[J]. Journal of Applied Science and Technology Trends, 2020, 1(2): 56-70. doi: 10.38094/jastt1224 URL
[20]	GUO Y, SHI H, KUMAR A, et al. Spottune: transfer learning through adaptive fine-tuning[C]//Proceedings of the IEEE conference on computer vision and pattern recognition. Long Beach, CA, US: IEEE, 2019: 4805-4814.
[21]	NIXON M, AGUADO A. Feature extraction and image processing for computer vision[M]. Kidlington, UK: Academic press, 2019.
[22]	GUYON I, GUNN S, NIKRAVESH M, et al. Feature extraction: foundations and applications[M]. Heidelberg, Germany: Springer, 2008.
[23]	BATZELIS E, BLANES J M, TOLEDO F J, et al. Noise-scaled euclidean distance: a metric for maximum likelihood estimation of the PV model parameters[J]. IEEE Journal of Photovoltaics, 2022, 12(3): 815-826. doi: 10.1109/JPHOTOV.2022.3159390 URL
[24]	徐志航, 陈博, 张辉, 等. 小数据下的音素级别说话人嵌入的语音合成自适应方法[J]. 计算机学报, 2022, 45(5): 1003-1017.
	XU Z H, CHEN B, ZHANG H, et al. Speech synthesis adaption method based on phoneme-level speaker embedding under small data[J]. Chinese Journal of Computers, 2022, 45(5): 1003-1017. (in Chinese)
[25]	SCHICK T, SCHÜTZE H. It’s not just size that matters: small language models are also few-shot learners[C]// Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics:Human Language Technologies. Online: ACL, 2021: 2339-2352.
[26]	LIU Y, REN J H, WU X, et al. Armored vehicle image classification based on transfer learning[C]//Proceedings of 2021 International Conference on Cyber-Physical Social Intelligence. Beijing, China: IEEE, 2021: 1-5.
[27]	张雪松, 庄严, 闫飞, 等. 基于迁移学习的类别级物体识别与检测研究与进展[J]. 自动化学报, 2019, 45(7): 1224-1243.
	ZHANG X S, ZHUANG Y, YAN F, et al. Status and Development of transfer learning based category-level object recognition and detection[J]. Acta Automatica Sinica, 2019, 45(7): 1224-1243. (in Chinese)
[28]	李策, 张栋, 杜少毅, 等. 一种迁移学习和可变形卷积深度学习的蝴蝶检测算法[J]. 自动化学报, 2019, 45(9): 1772-1782.
	LI C, ZHANG D, DU S Y, et al. A butterfly detection algorithm based on transfer learning and deformable convolution deep learning[J]. Acta Automatica Sinica, 2019, 45(9): 1772-1782. (in Chinese)
[29]	LOGESWARAN L, CHANG M W, LEE K, et al. Zero-shot entity linking by reading entity descriptions[C]// Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics. Florence, Italy: ACL, 2019: 3449-3460.
[30]	MEHRI S, RAZUMOVSKAIA E, ZHAO T, et al. Pretraining methods for dialog context representation learning[C]// Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics. Florence, Italy: ACL, 2019: 3836-3845.
[31]	ZHANG W J, WANG J C, LAN F P. Dynamic hand gesture recognition based on short-term sampling neural networks[J]. IEEE Journal of Automatica Sinica, 2020, 8(1): 110-120.
[32]	ZHOU B, KHOSLA A, LAPEDRIZA A, et al. Learning deep features for discriminative localization[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Las Vegas, NV, US: IEEE, 2016: 2921-2929.
[33]	AKIBA T, SANO S, YANASE T, et al. Optuna: a next-generation hyperparameter optimization framework[C]//Proceedings of the International Conference on Knowledge Discovery and Data Mining. Anchorage, AK, US: ACM, 2019: 2623-2631.
[34]	HE K M, ZHANG X Y, REN S Q, et al. Deep residual learning for image recognition[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Las Vegas, NV, US: IEEE, 2016: 770-778.
[35]	SZEGEDY C, LIU W, JIA Y Q, et al. Going deeper with convolutions[C]//Proceedings of 2015 IEEE Conference on Computer Vision and Pattern Recognition. Boston, MA, US: IEEE, 2015: 1-9.