Task Allocation Method of UAV Clusters Based on Sequence Generative Adversarial Network

doi:10.12382/bgxb.2022.0931

Abstract

Abstract:

In response to the problem that the existing UAV cluster task allocation algorithm decreases the solution efficiency and increases the solution time significantly when performing larger scale task allocations, a task allocation method based on sequence generative adversarial network is proposed. A sequence generation model containing a battlefield information feature extraction network and a sequence generation network are constructed to solve the problem of generating a sequence from battlefield information to task allocation. A discriminative model based on a multicore-multilayer convolutional network is constructed, and a gain-evaluation dual-guided policy gradient update is proposed for model training, which solves the problem of discrete task allocation sequences and ensures the quality of task allocation sequences. Simulation results show that the proposed method can efficiently generate task allocation sequences corresponding to battlefield information while guaranteeing the quality.

Key words: UAV cluster, task allocation, generative adversarial network, sequence generation model, policy gradient

CLC Number:

E919

YAN Yuwen, BI Wenhao, ZHANG An, ZHANG Baichuan. Task Allocation Method of UAV Clusters Based on Sequence Generative Adversarial Network[J]. Acta Armamentarii, 2023, 44(9): 2672-2684.

Figures/Tables 16

Fig.1 Model structure of GAN

Fig.2 Convolution operation and battlefield information matrix

Fig.3 Structure of the battlefield information feature extraction network

Fig.4 GRU unit

Fig.5 Structure of the sequence generation network

Fig.6 Structure of the discriminative model

Fig.7 TA-SeqGAN network training

Fig.8 Training algorithm of TA-SeqGAN

Table 1 TA-SeqGAN network structure and related hyperparameters

操作	隐藏层大小	核大小	步长	特征图大小	标准化	非线性处理
G_θ(Z)-3×N_U×N_T输入
全连接(1)	N/A	N/A	N/A	256	×	Leaky ReLU
全连接(2)	N/A	N/A	N/A	128	×	NONE
GRU	128	N/A	N/A	N/A	×	NONE
全连接	N/A	N/A	N/A	1	×	NONE
D_ϕ(G_θ(Z),R)-3×N_T输入
卷积(1,2,…,N_conv)	N/A	(1,2,…,N_conv)×N	1×1	32	√	Leaky ReLU
全连接	N/A	N/A	N/A	2	×	NONE
优化器	Adam(α=10^-3,β₁=0.9,β₂=0.999)
随机失活率(Dropout)	P=0.4
批量大小(Batch size)	16
Leaky ReLU slope	0.2

Fig.9 Training errors of the TA-SeqGAN network with different hyperparameters

Table 2 Task scene settings

编号	N_U	N_T
1	15	8
2	30	18
3	60	50
4	100	80

Fig.10 Task allocation effect on Training Data and Test Data

Fig.11 Task allocation reward of different methods under 4 conditions

Fig.12 Mean value of indicators by each method

Table 3 Comparison of running time of different algorithms with the different scenes

场景	GA	PSO算法	TA-SeqGAN
1	3.75	0.17	0.12
2	8.14	4.57	0.13
3	23.83	51.08	0.15
4	47.96	102.61	0.21

Fig.13 Comparison of running time of each algorithm with different allocation sizes

References 42

[1]	贾永楠, 田似营, 李擎. 无人机集群研究进展综述[J]. 航空学报, 2020, 41(增刊1): 4-14.
	JIA Y N, TIAN S Y, LI Q. Recent development of unmanned aerial vehicle swarms[J]. Acta Aeronautica et Astronautica Sinica, 2020, 41(S1): 4-14. (in Chinese)
[2]	贾高伟, 王建峰. 无人机集群任务规划方法研究综述[J]. 系统工程与电子技术, 2021, 43(1): 99-111.
	JIA G W, WANG J F. Research review of UAV swarm mission planning method[J]. Systems Engineering and Electronics, 2021, 43(1): 99-111. (in Chinese)
[3]	AFONS R, MAXIMO M, GALVAO R. Task allocation and trajectory planning for multiple agents in the presence of obstacle and connectivity constraints with mixed-integer linear programming[J]. International Journal of Robust and Nonlinear Control, 2020, 30(14): 5464-5491. doi: 10.1002/rnc.v30.14 URL
[4]	ZHU Q, ZENG H B, ZHENG W, et al. Optimization of task allocation and priority assignment in hard real-time distributed systems[J]. Acm Transactions on Embedded Computing Systems, 2013, 11(4): 1-30.
[5]	ZHU D, HUANG H, YANG S X. Dynamic task assignment and path planning of multi-AUV system based on an improved self-organizing map and velocity synthesis method in three-dimensional underwater workspace[J]. IEEE Transactions on Cybernetics, 2013, 43(2): 504-514. doi: 10.1109/TSMCB.2012.2210212 pmid: 22949070
[6]	杜永浩, 邢立宁, 姚锋, 等. 航天器任务调度模型、算法与通用求解技术综述[J]. 自动化学报, 2021, 47(12):2715-2741.
	DU Y H, XING L N, YAO F, et al. Survey on models, algorithms and general techniques for spacecraft mission scheduling[J]. Acta Automatica Sinica, 2021, 47(12):2715-2741. (in Chinese)
[7]	ZHAO B, ZHANG Y W, LIU D R. Adaptive dynamic programming-based cooperative motion/force control for modular reconfigurable manipulators: a joint task assignment approach[J/OL]. IEEE Transactions on Neural Networks and Learning Systems, 2022(2022-05-11)[2022-10-21]. https://doi.org/10.1109/TNNLS.2022.3171828.
[8]	沈林成, 陈璟, 王楠. 飞行器任务规划技术综述[J]. 航空学报, 2014, 35(3): 593-606. doi: 10.7527/S1000-6893.2013.0500
	SHEN L C, CHEN J, WANG N. Overview of air vehicle mission planning techniques[J]. Acta Aeronautica et Astronautica Sinica, 2014, 35(3): 593-606. (in Chinese) doi: 10.7527/S1000-6893.2013.0500
[9]	LIU J, ANAVATTI S, GARRATT M, et al. Modified continuous ant colony optimisation for multiple unmanned ground vehicle path planning[J]. Expert Systems with Applications, 2022, 196: 116605. doi: 10.1016/j.eswa.2022.116605 URL
[10]	彭鹏菲, 于钱, 李启元. 基于优先排序与粒子群优化的装备保障任务规划方法[J]. 兵工学报, 2016, 37(6): 1082-1088. doi: 10.3969/j.issn.1000-1093.2016.06.016
	PENG P F, YU Q, LI Q Y. A planning method of equipment support task based on priority ordering and particle swarm optimization algorithm[J]. Acta Armamentarii, 2016, 37(6): 1082-1088. (in Chinese)
[11]	WU G H, PEDRYCZ W, LI H F, et al. Coordinated planning of heterogeneous earth observation resources[J]. IEEE Transactions on Systems, Man, and Cybernetics: Systems, 2015, 46(1): 109-125. doi: 10.1109/TSMC.2015.2431643 URL
[12]	苏菲, 陈岩, 沈林成. 基于蚁群算法的无人机协同多任务分配[J]. 航空学报, 2008, 29(增刊1): 184-191.
	SU F, CHEN Y, SHEN L C. UAV Cooperative multi-task assignment based on ant colony algorithm[J]. Acta Aeronautica et Astronautica Sinica, 2008, 29(S1): 184-191. (in Chinese)
[13]	范博洋, 赵高鹏, 薄煜明, 等. 多目标空地异构无人系统协同任务分配方法[J]. 兵工学报, 2023, 44(6):1564-1575. doi: 10.12382/bgxb.2022.0095
	FAN B Y, ZHAO G P, BO Y M, et al. Collaborative task allocation method for multi-target aerial-ground heterogeneous unmanned system[J]. Acta Armamentarii, 2023, 44(6):1564-1575. (in Chinese)
[14]	KIM D Y, LEE J W. Joint mission assignment and topology management in the mission critical FANET[J]. IEEE Internet of Things Journal, 2019, 7(3): 2368-2385. doi: 10.1109/JIoT.6488907 URL
[15]	张祥银, 夏爽, 张天. 基于自适应遗传学习粒子群算法的多无人机协同任务分配[J/OL]. 控制与决策, 2022(2022-07-12)[2022-10-21]. https://doi.org/10.13195/j.kzyjc.2022.0240.
	ZHANG X Y, XIA S, ZHANG T. Adaptive genetic learning particle swarm optimization based cooperative task allocation for multi-UAV[J]. Control and Decision, 2022(2022-07-12)[2022-10-21]. https://doi.org/10.13195/j.kzyjc.2022.0240. (in Chinese)
[16]	彭雅兰, 段海滨, 张岱峰, 等. 仿灰狼合作捕食行为的无人机集群动态任务分配[J]. 控制理论与应用, 2021, 38(11): 1855-1862.
	PENG Y L, DUAN H B, ZHANG D F, et al. Unmanned aerial vehicle swarm dynamic mission planning inspired by cooperative predation of wolf-pack[J]. Control Theory & Applications, 2021, 38(11): 1855-1862. (in Chinese)
[17]	龙涛, 沈林成, 朱华勇, 等. 面向协同任务的多UCAV分布式任务分配与协调技术[J]. 自动化学报, 2007, 33(7): 731-737.
	LONG T, SHEN L C, ZHU H Y, et al. Distributed task allocation & coordination technique of multiple UCAVs for cooperative tasks[J]. Acta Automatica Sinica, 2007, 33(7): 731-737. (in Chinese)
[18]	张梦颖, 王蒙一, 王晓东, 等. 基于改进合同网的无人机群协同实时任务分配问题研究[J]. 航空兵器, 2019, 26(4): 38-46.
	ZHANG M Y, WANG M Y, WANG X D, et al. Cooperative real-time task assignment of uAV group based on improved contract net[J]. Aero Weaponry, 2019, 26(4):38-46. (in Chinese)
[19]	李鑫滨, 郭力争, 韩松. 一种分布式异构多AUV任务分配鲁棒拍卖算法[J]. 北京航空航天大学学报, 2022, 48(5): 736-746.
	LI X B, GUO L Z, HAN S. robust auction algorithm for distributed heterogeneous multi-AUV task assignment[J]. Journal of Beijing University of Aeronautics and Astronautics, 2022, 48(5): 736-746. (in Chinese)
[20]	DENG L, YU D. Deep learning: methods and applications[J]. Foundations and Trends in Signal Processing, 2014, 7(3/4): 197-387. doi: 10.1561/2000000039 URL
[21]	竺殊荣. 动态环境下多无人机协同任务规划方法[D]. 南京: 南京邮电大学, 2019: 28-40.
	ZHU S R. Cooperative task planning method of multi-UAV in dynamic environment[D]. Nanjing: Nanjing University of Posts and Telecommunications, 2019: 28-40. (in Chinese)
[22]	LIU J Y, WANG G, FU Q, et al. Task assignment in ground-to-air confrontation based on multiagent deep reinforcement learning[J]. Defence Technology, 2023, 19(1): 210-219. doi: 10.1016/j.dt.2022.04.001
[23]	李凯文, 张涛, 王锐, 等. 基于深度强化学习的组合优化研究进展[J]. 自动化学报, 2021, 47(11): 2521-2537.
	LI K W, ZHANG T, WANG R, et al. Research reviews of combinatorial optimization methods based on deep reinforcement learning[J]. Acta Automatica Sinica, 2021, 47(11): 2521-2537. (in Chinese)
[24]	GOODFELLOW I J, POUGET-ABADIE J, MIRZA M, et al. Generative adversarial nets[C]//Proceedings of International Conference on Neural Information Processing Systems. Cambridge, MA, US: MIT Press, 2014: 2672-2680.
[25]	ZHAO Y R, DENG B, HUANG J Q, et al. Stylized adversarial autoencoder for image generation[C]// Proceedings of the 25th ACM international Conference on Multimedia. New York, NY, US: Association for Computing Machinery, 2017: 244-251.
[26]	FRID-ADAR M, DIAMANT I, KLANG E, et al. GAN-based synthetic medical image augmentation for increased CNN performance in liver lesion classification[J]. Neurocomputing, 2018, 321: 321-331. doi: 10.1016/j.neucom.2018.09.013 URL
[27]	JING Y C, YANG Y Z, FENG Z L, et al. Neural style transfer:a review[J]. IEEE Transactions on Visualization and Computer Graphics, 2019, 26(11): 3365-3385. doi: 10.1109/TVCG.2945 URL
[28]	林懿伦, 戴星原, 李力, 等. 人工智能研究的新前线:生成式对抗网络[J]. 自动化学报, 2018, 44(5): 775-792.
	LIN Y L, DAI X Y, LI L, et al. The new frontier of AI research: generative adversarial networks[J]. Acta Automatica Sinica, 2018, 44(5): 775-792. (in Chinese)
[29]	MIRZA M, OSINDERO S. Conditional generative adversarial nets:arXiv:1411.1784[R]. Ithaca, NY, US: Cornell University, 2014:1411.1784.
[30]	RADFORD A, METZ L, CHINTALA S. Unsupervised representation learning with deep convolutional generative adversarial networks:arXiv:1511.06434[R]. Ithaca, NY, US: Cornell University, 2015:1511.06434.
[31]	ARJOVSKY M, BOTTOU L. Towards principled methods for training generative adversarial networks: arXiv:1701.04862[R]. Ithaca, NY, US: Cornell University, 2017:1701.04862.
[32]	YU L T, ZHANG W N, WANG J, et al. SeqGAN: sequence generative adversarial nets with policy gradient: arXiv:1609.05473[R]//Ithaca, NY, US: Cornell University, 2016:1609.05473.
[33]	孙秋野, 胡旌伟, 杨凌霄, 等. 基于GAN技术的自能源混合建模与参数辨识方法[J]. 自动化学报, 2018, 44(5): 901-914.
	SUN Q Y, HU J W, YANG L X, et al. We-energy hybrid modeling and parameter identification with GAN technology[J]. Acta Automatica Sinica, 2018, 44(5): 901-914. (in Chinese)
[34]	薛灵芝, 曾向阳, 杨爽. 基于生成对抗网络的水声目标识别算法[J]. 兵工学报, 2021, 42(11): 2444-2452. doi: 10.3969/j.issn.1000-1093.2021.11.018
	XUE L Z, ZENG X Y, YANG S. Underwater acoustic target recognition algorithm based on generative adversarial networks[J]. Acta Armamentarii, 2021, 42(11): 2444-2452. (in Chinese) doi: 10.3969/j.issn.1000-1093.2021.11.018
[35]	郑念祖, 丁进良. 基于Regression GAN的原油总氢物性预测方法[J]. 自动化学报, 2018, 44(5): 915-921.
	ZHENG N Z, DING J L. Regression GAN based prediction for physical properties of total hydrogen in crude oil[J]. Acta Automatica Sinica, 2018, 44(5): 915-921. (in Chinese)
[36]	殷小静, 胡晓峰, 杨镜宇. 面向作战体系的生成式对抗网络应用研究[J]. 火力与指挥控制, 2019, 44(11): 11-15.
	YIN X J, HU X F, YANG J Y. Research of operation SoS oriented application of generative adversarial networks[J]. Fire Control & Command Control, 2019, 44(11): 11-15. (in Chinese)
[37]	GU J X, WANG Z H, KUEN J, et al. Recent advances in convolutional neural networks[J]. Pattern Recognition, 2018, 77: 354-377. doi: 10.1016/j.patcog.2017.10.013 URL
[38]	CHUNG J Y, GUICEHRE C, CHO K H, et al. Empirical evaluation of gated recurrent neural networks on sequence modeling: arXiv:1412.3555[R]. Ithaca, NY, US: Cornell University, 2014:1412.3555.
[39]	ZAREMBA W, SUTSKEVER I, VINYALS O. Recurrent neural network regularization: arXiv:1409.2329[R]. Ithaca, NY, US: Cornell University, 2014:1409.2329.
[40]	MIKOLV T, CHEN K, CORRADO G, et al. Efficient estimation of word representations in vector space: arXiv:1301.3781[R]. Ithaca, NY, US: Cornell University, 2013:1301.3781.
[41]	MA X B, DRIGGS-CAMPBELL K, ZHANG Z Z, et al. Monte-carlo tree search for policy optimization:arXiv. 1912.10648[R]. Ithaca, NY, US: Cornell University, 2019: 1912.10648.
[42]	严飞, 祝小平, 周洲, 等. 考虑同时攻击约束的多异构无人机实时任务分配[J]. 中国科学:信息科学, 2019, 49(5): 555-569.
	YAN F, ZHU X P, ZHOU Z, et al. Real-time task allocation for a heterogeneous multi-UAV simultaneous attack[J]. Scientia Sinica(Informationis), 2019, 49(5): 555-569. (in Chinese)