基于序列生成对抗网络的无人机集群任务分配方法

doi:10.12382/bgxb.2022.0931

摘要/Abstract

摘要：

针对现有无人机集群任务分配算法在进行较大规模的任务分配时求解效率降低、求解时间大幅增加的问题,提出一种基于序列生成对抗网络的任务分配方法。通过构建包含战场信息特征提取网络和序列生成网络的序列生成模型,解决战场信息到任务分配序列的生成问题;构建基于多核多层卷积网络的判别模型,提出收益-评价双指导式策略梯度更新对模型进行训练,解决任务分配序列离散的问题,保证任务分配序列的质量。仿真结果表明,新方法在保证分配序列质量的情况下,能够高效地生成与战场信息对应的任务分配序列。

关键词: 无人机集群, 任务分配, 生成对抗网络, 序列生成模型, 策略梯度

Abstract:

In response to the problem that the existing UAV cluster task allocation algorithm decreases the solution efficiency and increases the solution time significantly when performing larger scale task allocations, a task allocation method based on sequence generative adversarial network is proposed. A sequence generation model containing a battlefield information feature extraction network and a sequence generation network are constructed to solve the problem of generating a sequence from battlefield information to task allocation. A discriminative model based on a multicore-multilayer convolutional network is constructed, and a gain-evaluation dual-guided policy gradient update is proposed for model training, which solves the problem of discrete task allocation sequences and ensures the quality of task allocation sequences. Simulation results show that the proposed method can efficiently generate task allocation sequences corresponding to battlefield information while guaranteeing the quality.

Key words: UAV cluster, task allocation, generative adversarial network, sequence generation model, policy gradient

中图分类号:

E919

严钰文, 毕文豪, 张安, 张百川. 基于序列生成对抗网络的无人机集群任务分配方法[J]. 兵工学报, 2023, 44(9): 2672-2684.

YAN Yuwen, BI Wenhao, ZHANG An, ZHANG Baichuan. Task Allocation Method of UAV Clusters Based on Sequence Generative Adversarial Network[J]. Acta Armamentarii, 2023, 44(9): 2672-2684.

图/表 16

图1 GAN模型结构

Fig.1 Model structure of GAN

图2 卷积操作与输入战场信息矩阵

Fig.2 Convolution operation and battlefield information matrix

图3 战场信息特征提取网络结构

Fig.3 Structure of the battlefield information feature extraction network

图4 GRU单元

Fig.4 GRU unit

图5 序列生成网络结构

Fig.5 Structure of the sequence generation network

图6 判别模型结构

Fig.6 Structure of the discriminative model

图7 TA-SeqGAN网络训练

Fig.7 TA-SeqGAN network training

图8 TA-SeqGAN模型训练算法

Fig.8 Training algorithm of TA-SeqGAN

表1 TA-SeqGAN网络结构及相关超参数

Table 1 TA-SeqGAN network structure and related hyperparameters

操作	隐藏层大小	核大小	步长	特征图大小	标准化	非线性处理
G_θ(Z)-3×N_U×N_T输入
全连接(1)	N/A	N/A	N/A	256	×	Leaky ReLU
全连接(2)	N/A	N/A	N/A	128	×	NONE
GRU	128	N/A	N/A	N/A	×	NONE
全连接	N/A	N/A	N/A	1	×	NONE
D_ϕ(G_θ(Z),R)-3×N_T输入
卷积(1,2,…,N_conv)	N/A	(1,2,…,N_conv)×N	1×1	32	√	Leaky ReLU
全连接	N/A	N/A	N/A	2	×	NONE
优化器	Adam(α=10^-3,β₁=0.9,β₂=0.999)
随机失活率(Dropout)	P=0.4
批量大小(Batch size)	16
Leaky ReLU slope	0.2

图9 不同超参数下TA-SeqGAN网络训练误差

Fig.9 Training errors of the TA-SeqGAN network with different hyperparameters

表2 任务场景设置

Table 2 Task scene settings

编号	N_U	N_T
1	15	8
2	30	18
3	60	50
4	100	80

图10 训练数据和测试数据上的任务分配效果

Fig.10 Task allocation effect on Training Data and Test Data

图11 4种场景下各算法任务分配序列的收益函数指标值

Fig.11 Task allocation reward of different methods under 4 conditions

图12 各算法收益指标均值

Fig.12 Mean value of indicators by each method

表3 不同场景下不同算法运行时间对比

Table 3 Comparison of running time of different algorithms with the different scenes

场景	GA	PSO算法	TA-SeqGAN
1	3.75	0.17	0.12
2	8.14	4.57	0.13
3	23.83	51.08	0.15
4	47.96	102.61	0.21

图13 不同分配规模下各算法运行时间对比

Fig.13 Comparison of running time of each algorithm with different allocation sizes

参考文献 42

[1]	贾永楠, 田似营, 李擎. 无人机集群研究进展综述[J]. 航空学报, 2020, 41(增刊1): 4-14.
	JIA Y N, TIAN S Y, LI Q. Recent development of unmanned aerial vehicle swarms[J]. Acta Aeronautica et Astronautica Sinica, 2020, 41(S1): 4-14. (in Chinese)
[2]	贾高伟, 王建峰. 无人机集群任务规划方法研究综述[J]. 系统工程与电子技术, 2021, 43(1): 99-111.
	JIA G W, WANG J F. Research review of UAV swarm mission planning method[J]. Systems Engineering and Electronics, 2021, 43(1): 99-111. (in Chinese)
[3]	AFONS R, MAXIMO M, GALVAO R. Task allocation and trajectory planning for multiple agents in the presence of obstacle and connectivity constraints with mixed-integer linear programming[J]. International Journal of Robust and Nonlinear Control, 2020, 30(14): 5464-5491. doi: 10.1002/rnc.v30.14 URL
[4]	ZHU Q, ZENG H B, ZHENG W, et al. Optimization of task allocation and priority assignment in hard real-time distributed systems[J]. Acm Transactions on Embedded Computing Systems, 2013, 11(4): 1-30.
[5]	ZHU D, HUANG H, YANG S X. Dynamic task assignment and path planning of multi-AUV system based on an improved self-organizing map and velocity synthesis method in three-dimensional underwater workspace[J]. IEEE Transactions on Cybernetics, 2013, 43(2): 504-514. doi: 10.1109/TSMCB.2012.2210212 pmid: 22949070
[6]	杜永浩, 邢立宁, 姚锋, 等. 航天器任务调度模型、算法与通用求解技术综述[J]. 自动化学报, 2021, 47(12):2715-2741.
	DU Y H, XING L N, YAO F, et al. Survey on models, algorithms and general techniques for spacecraft mission scheduling[J]. Acta Automatica Sinica, 2021, 47(12):2715-2741. (in Chinese)
[7]	ZHAO B, ZHANG Y W, LIU D R. Adaptive dynamic programming-based cooperative motion/force control for modular reconfigurable manipulators: a joint task assignment approach[J/OL]. IEEE Transactions on Neural Networks and Learning Systems, 2022(2022-05-11)[2022-10-21]. https://doi.org/10.1109/TNNLS.2022.3171828.
[8]	沈林成, 陈璟, 王楠. 飞行器任务规划技术综述[J]. 航空学报, 2014, 35(3): 593-606. doi: 10.7527/S1000-6893.2013.0500
	SHEN L C, CHEN J, WANG N. Overview of air vehicle mission planning techniques[J]. Acta Aeronautica et Astronautica Sinica, 2014, 35(3): 593-606. (in Chinese) doi: 10.7527/S1000-6893.2013.0500
[9]	LIU J, ANAVATTI S, GARRATT M, et al. Modified continuous ant colony optimisation for multiple unmanned ground vehicle path planning[J]. Expert Systems with Applications, 2022, 196: 116605. doi: 10.1016/j.eswa.2022.116605 URL
[10]	彭鹏菲, 于钱, 李启元. 基于优先排序与粒子群优化的装备保障任务规划方法[J]. 兵工学报, 2016, 37(6): 1082-1088. doi: 10.3969/j.issn.1000-1093.2016.06.016
	PENG P F, YU Q, LI Q Y. A planning method of equipment support task based on priority ordering and particle swarm optimization algorithm[J]. Acta Armamentarii, 2016, 37(6): 1082-1088. (in Chinese)
[11]	WU G H, PEDRYCZ W, LI H F, et al. Coordinated planning of heterogeneous earth observation resources[J]. IEEE Transactions on Systems, Man, and Cybernetics: Systems, 2015, 46(1): 109-125. doi: 10.1109/TSMC.2015.2431643 URL
[12]	苏菲, 陈岩, 沈林成. 基于蚁群算法的无人机协同多任务分配[J]. 航空学报, 2008, 29(增刊1): 184-191.
	SU F, CHEN Y, SHEN L C. UAV Cooperative multi-task assignment based on ant colony algorithm[J]. Acta Aeronautica et Astronautica Sinica, 2008, 29(S1): 184-191. (in Chinese)
[13]	范博洋, 赵高鹏, 薄煜明, 等. 多目标空地异构无人系统协同任务分配方法[J]. 兵工学报, 2023, 44(6):1564-1575. doi: 10.12382/bgxb.2022.0095
	FAN B Y, ZHAO G P, BO Y M, et al. Collaborative task allocation method for multi-target aerial-ground heterogeneous unmanned system[J]. Acta Armamentarii, 2023, 44(6):1564-1575. (in Chinese)
[14]	KIM D Y, LEE J W. Joint mission assignment and topology management in the mission critical FANET[J]. IEEE Internet of Things Journal, 2019, 7(3): 2368-2385. doi: 10.1109/JIoT.6488907 URL
[15]	张祥银, 夏爽, 张天. 基于自适应遗传学习粒子群算法的多无人机协同任务分配[J/OL]. 控制与决策, 2022(2022-07-12)[2022-10-21]. https://doi.org/10.13195/j.kzyjc.2022.0240.
	ZHANG X Y, XIA S, ZHANG T. Adaptive genetic learning particle swarm optimization based cooperative task allocation for multi-UAV[J]. Control and Decision, 2022(2022-07-12)[2022-10-21]. https://doi.org/10.13195/j.kzyjc.2022.0240. (in Chinese)
[16]	彭雅兰, 段海滨, 张岱峰, 等. 仿灰狼合作捕食行为的无人机集群动态任务分配[J]. 控制理论与应用, 2021, 38(11): 1855-1862.
	PENG Y L, DUAN H B, ZHANG D F, et al. Unmanned aerial vehicle swarm dynamic mission planning inspired by cooperative predation of wolf-pack[J]. Control Theory & Applications, 2021, 38(11): 1855-1862. (in Chinese)
[17]	龙涛, 沈林成, 朱华勇, 等. 面向协同任务的多UCAV分布式任务分配与协调技术[J]. 自动化学报, 2007, 33(7): 731-737.
	LONG T, SHEN L C, ZHU H Y, et al. Distributed task allocation & coordination technique of multiple UCAVs for cooperative tasks[J]. Acta Automatica Sinica, 2007, 33(7): 731-737. (in Chinese)
[18]	张梦颖, 王蒙一, 王晓东, 等. 基于改进合同网的无人机群协同实时任务分配问题研究[J]. 航空兵器, 2019, 26(4): 38-46.
	ZHANG M Y, WANG M Y, WANG X D, et al. Cooperative real-time task assignment of uAV group based on improved contract net[J]. Aero Weaponry, 2019, 26(4):38-46. (in Chinese)
[19]	李鑫滨, 郭力争, 韩松. 一种分布式异构多AUV任务分配鲁棒拍卖算法[J]. 北京航空航天大学学报, 2022, 48(5): 736-746.
	LI X B, GUO L Z, HAN S. robust auction algorithm for distributed heterogeneous multi-AUV task assignment[J]. Journal of Beijing University of Aeronautics and Astronautics, 2022, 48(5): 736-746. (in Chinese)
[20]	DENG L, YU D. Deep learning: methods and applications[J]. Foundations and Trends in Signal Processing, 2014, 7(3/4): 197-387. doi: 10.1561/2000000039 URL
[21]	竺殊荣. 动态环境下多无人机协同任务规划方法[D]. 南京: 南京邮电大学, 2019: 28-40.
	ZHU S R. Cooperative task planning method of multi-UAV in dynamic environment[D]. Nanjing: Nanjing University of Posts and Telecommunications, 2019: 28-40. (in Chinese)
[22]	LIU J Y, WANG G, FU Q, et al. Task assignment in ground-to-air confrontation based on multiagent deep reinforcement learning[J]. Defence Technology, 2023, 19(1): 210-219. doi: 10.1016/j.dt.2022.04.001
[23]	李凯文, 张涛, 王锐, 等. 基于深度强化学习的组合优化研究进展[J]. 自动化学报, 2021, 47(11): 2521-2537.
	LI K W, ZHANG T, WANG R, et al. Research reviews of combinatorial optimization methods based on deep reinforcement learning[J]. Acta Automatica Sinica, 2021, 47(11): 2521-2537. (in Chinese)
[24]	GOODFELLOW I J, POUGET-ABADIE J, MIRZA M, et al. Generative adversarial nets[C]//Proceedings of International Conference on Neural Information Processing Systems. Cambridge, MA, US: MIT Press, 2014: 2672-2680.
[25]	ZHAO Y R, DENG B, HUANG J Q, et al. Stylized adversarial autoencoder for image generation[C]// Proceedings of the 25th ACM international Conference on Multimedia. New York, NY, US: Association for Computing Machinery, 2017: 244-251.
[26]	FRID-ADAR M, DIAMANT I, KLANG E, et al. GAN-based synthetic medical image augmentation for increased CNN performance in liver lesion classification[J]. Neurocomputing, 2018, 321: 321-331. doi: 10.1016/j.neucom.2018.09.013 URL
[27]	JING Y C, YANG Y Z, FENG Z L, et al. Neural style transfer:a review[J]. IEEE Transactions on Visualization and Computer Graphics, 2019, 26(11): 3365-3385. doi: 10.1109/TVCG.2945 URL
[28]	林懿伦, 戴星原, 李力, 等. 人工智能研究的新前线:生成式对抗网络[J]. 自动化学报, 2018, 44(5): 775-792.
	LIN Y L, DAI X Y, LI L, et al. The new frontier of AI research: generative adversarial networks[J]. Acta Automatica Sinica, 2018, 44(5): 775-792. (in Chinese)
[29]	MIRZA M, OSINDERO S. Conditional generative adversarial nets:arXiv:1411.1784[R]. Ithaca, NY, US: Cornell University, 2014:1411.1784.
[30]	RADFORD A, METZ L, CHINTALA S. Unsupervised representation learning with deep convolutional generative adversarial networks:arXiv:1511.06434[R]. Ithaca, NY, US: Cornell University, 2015:1511.06434.
[31]	ARJOVSKY M, BOTTOU L. Towards principled methods for training generative adversarial networks: arXiv:1701.04862[R]. Ithaca, NY, US: Cornell University, 2017:1701.04862.
[32]	YU L T, ZHANG W N, WANG J, et al. SeqGAN: sequence generative adversarial nets with policy gradient: arXiv:1609.05473[R]//Ithaca, NY, US: Cornell University, 2016:1609.05473.
[33]	孙秋野, 胡旌伟, 杨凌霄, 等. 基于GAN技术的自能源混合建模与参数辨识方法[J]. 自动化学报, 2018, 44(5): 901-914.
	SUN Q Y, HU J W, YANG L X, et al. We-energy hybrid modeling and parameter identification with GAN technology[J]. Acta Automatica Sinica, 2018, 44(5): 901-914. (in Chinese)
[34]	薛灵芝, 曾向阳, 杨爽. 基于生成对抗网络的水声目标识别算法[J]. 兵工学报, 2021, 42(11): 2444-2452. doi: 10.3969/j.issn.1000-1093.2021.11.018
	XUE L Z, ZENG X Y, YANG S. Underwater acoustic target recognition algorithm based on generative adversarial networks[J]. Acta Armamentarii, 2021, 42(11): 2444-2452. (in Chinese) doi: 10.3969/j.issn.1000-1093.2021.11.018
[35]	郑念祖, 丁进良. 基于Regression GAN的原油总氢物性预测方法[J]. 自动化学报, 2018, 44(5): 915-921.
	ZHENG N Z, DING J L. Regression GAN based prediction for physical properties of total hydrogen in crude oil[J]. Acta Automatica Sinica, 2018, 44(5): 915-921. (in Chinese)
[36]	殷小静, 胡晓峰, 杨镜宇. 面向作战体系的生成式对抗网络应用研究[J]. 火力与指挥控制, 2019, 44(11): 11-15.
	YIN X J, HU X F, YANG J Y. Research of operation SoS oriented application of generative adversarial networks[J]. Fire Control & Command Control, 2019, 44(11): 11-15. (in Chinese)
[37]	GU J X, WANG Z H, KUEN J, et al. Recent advances in convolutional neural networks[J]. Pattern Recognition, 2018, 77: 354-377. doi: 10.1016/j.patcog.2017.10.013 URL
[38]	CHUNG J Y, GUICEHRE C, CHO K H, et al. Empirical evaluation of gated recurrent neural networks on sequence modeling: arXiv:1412.3555[R]. Ithaca, NY, US: Cornell University, 2014:1412.3555.
[39]	ZAREMBA W, SUTSKEVER I, VINYALS O. Recurrent neural network regularization: arXiv:1409.2329[R]. Ithaca, NY, US: Cornell University, 2014:1409.2329.
[40]	MIKOLV T, CHEN K, CORRADO G, et al. Efficient estimation of word representations in vector space: arXiv:1301.3781[R]. Ithaca, NY, US: Cornell University, 2013:1301.3781.
[41]	MA X B, DRIGGS-CAMPBELL K, ZHANG Z Z, et al. Monte-carlo tree search for policy optimization:arXiv. 1912.10648[R]. Ithaca, NY, US: Cornell University, 2019: 1912.10648.
[42]	严飞, 祝小平, 周洲, 等. 考虑同时攻击约束的多异构无人机实时任务分配[J]. 中国科学:信息科学, 2019, 49(5): 555-569.
	YAN F, ZHU X P, ZHOU Z, et al. Real-time task allocation for a heterogeneous multi-UAV simultaneous attack[J]. Scientia Sinica(Informationis), 2019, 49(5): 555-569. (in Chinese)