基于智能算法的无人集群防御作战方案优化方法

doi:10.12382/bgxb.2021.0339

摘要/Abstract

摘要： 兵力部署与任务分配是无人集群防御作战的重要过程，有效利用集群中有限的兵力并发挥出最高的作战效能对提高无人集群的作战胜率至关重要，进行高效的作战任务分配能够协调集群一致性并更好完成作战任务。针对无人集群防御作战中的关键作战方案，研究无人集群防御作战的兵力部署及协同任务分配优化问题。构建基于智能体技术的无人集群防御作战模型，量化无人集群兵力部署所需的关键参数，对作战区域与兵力进行规划，设计目标函数。提出一种自适应遗传算法，解决无人集群的兵力部署问题。算法可根据实时运行情况动态调整目标函数、交叉率和变异率，保证适应度值较高个体的传承并避免算法出现局部最优。进行防御作战仿真，为验证无人集群兵力部署的效果，提出一种基于深度Q网络的深度强化学习改进算法，解决无人集群任务分配问题，对部署好的无人集群进行任务分配并作战。该算法能够自适应调整Q值，避免算法因过度估计造成无法收敛至最优解。防御作战实验结果表明，所提出的无人集群兵力部署及协同任务分配方法可有效提高防御作战的成功率，实现无人集群的自主协同及智能对抗。

关键词: 无人集群防御作战, 自适应遗传算法, 智能体技术, 深度强化学习

Abstract: Troop deployment and task allocation are important processes of unmanned swarm defense operations, it's very important to make effective use of the limited forces in the swarm and wield the highest operational efficiency to improve the battle victory rate of unmanned swarm. At the same time, efficient combat task allocation can coordinate the consistency of swarm and better complete combat tasks. Aiming at the key combat plan in unmanned swarm defense operations, the optimization of troop deployment and coordinated task allocation in unmanned swarm defense operations is studied. A multi-agent-based unmanned swarm defensive combat model is built to quantify the key parameters required for the deployment of unmanned swarm forces, the model plans the combat area and troops, and designs the objective function. An adaptive genetic algorithm is proposed to solve the deployment problem of unmanned swarms. The proposed algorithm could dynamically adjust the objective function，crossover rate and mutation rate according to the real-time operating conditions，ensuring the inheritance of individuals with higher fitness values and avoiding the local optimization of the algorithm. A defensive operation is simulated to verify the deployment effectiveness of unmanned swarm forces. The improved deep reinforcement learning algorithm based on deep Q network is proposed to find a solution to the task allocationfor the deployed unmanned swarms. The proposed algorithm could adjust the Q value through self-adaption to avoid non-convergence caused by the algorithm's overestimation and find the optimal solution.The experimental results of defensive operations show that the proposed unmanned swarm force deployment and coordinated task allocation method could effectively improve the success rate of defensive operations，and realize the autonomous coordination and intelligent confrontation of unmanned swarms.

Key words: unmannedswarmdefenseoperations, adaptivegeneticalgorithm, agenttechnology, deepreinforcementlearning

中图分类号:

TP399

马也，范文慧，常天庆. 基于智能算法的无人集群防御作战方案优化方法[J]. 兵工学报, 2022, 43(6): 1415-1425.

MA Ye， FAN Wenhui， CHANG Tianqing. Optimization Method of Unmanned Swarm Defensive Combat Scheme Based on Intelligent Algorithm[J]. Acta Armamentarii, 2022, 43(6): 1415-1425.

参考文献

［1］ NAQVI S A R，HASSAN S A，PERVAIZ H，et al. Drone-aided communication as a key enabler for 5G and resilient public safety networks[J].IEEE Communications Magazine，2018，56(1):36-42.
[2］ ZENG Y，LYU J B，ZHANG R.Cellular-connected UAV: potentials，challenges and promising technologies[J].IEEE Wireless Communications，2018，26(1):120-127.
[3］ GUVENE I，F KOOHIFAR，SINGH S，et al.Detection，tracking，and interdiction for amateur drones[J].IEEE Communications Magazine，2018，56(4):75-81.
[4］ WANG X M，XU Y H，CHEN C H，et al.Machine learning empowered spectrum sharing in intelligent unmanned swarm communication systems:challenges，requirements and solutions[J].IEEE Access，2020，8:89839-89849.
[5］ KIM H，MOKDAD L，BEN-OTHMAN J.Designing UAV surveillance frameworks for smart city and extensive ocean with differential perspectives[J].IEEE Communications Magazine，2018，56(4):98-104.
[6］ GU J J，TAO S，WANG Q H，et al.Multiple moving targets surveillance based on a cooperative network for multi-UAV[J].IEEE Communications Magazine，2018，56(4):82-89.
[7］ WANG J J，JIANG C X，HAN Z，et al.Taking drones to the next level:cooperative distributed unmanned-aerial-vehicular networks for small and mini drones[J].IEEE Vehicular Technology Magazine，2017，12(3):73-82.
[8］刘献伟，陈虎林，李飞，等.一网打尽——无人集群时代的新攻防[J].航空兵器，2019，26(1):74-79.
LIU X W，CHEN H L，LI F，et al.The new attack and defense in the era of unmanned group[J].Aero Weaponry，2019，26(1):74-79. (in Chinese)
[9］刘振，徐学文，李静.考虑协同制导的编队一体化防空问题分析与求解[J].指挥与控制学报，2018，4(3):213-219.
LIU Z，XU W X，LI J.Analysis and solving for formation integrated air defense considering cooperative guidance[J]. Journal of Command and Control，2018，4(3):213-219.(in Chinese)
[10］ WEN B Q，WANG T，CHENG K，et al.End-defense force optimization deployment method based on PSO-GA hybrid algorithm[J].Journal of Ordnance Equipment Engineering，2019.(in Chinese)
[11］李凌昊，邱兴业，李凌，等.面向舰载航空兵作战的不间断指挥能力研究[J].指挥与控制学报，2018，4(4):74-78.
LI L H，QIU X Y，LI L，et al.Continuous command capability for carrier-borne aviation force[J].Journal of Command and Control，2018，4(4):74-78.(in Chinese)
[12］赵鹏蛟，李建国，李红霞，等.基于 Memetic 算法的要地防空兵力机动部署优化方法[J].火力与指挥控制，2018，43(9):25-29.
ZHAO P J，LI J G，LI H X，et al.Optimization of troops maneuver deployment for key- point air defense based on Memetic algorithm[J].Fire Control & Command Control，2018，43(9):25-29.(in Chinese)
[13］ HUANG C N，CHAN C T.ZigBee-based indoor location system by K-nearest neighbor algorithm with weighted RSSI[J].Procedia Computer Science，2011，5:58-65.
[14］王玥，张克，孙鑫．无人飞行器任务规划技术[M].北京:国防工业出版社，2015.
WANG Y，ZHANG K，SUN X.Mission planning technology for UAV[M].Beijing:National Defense Industry Press，2015.(in Chinese)
[15］ FARUQ F，PARKER D，LACCRDA B，et al. Simultaneous task allocation and planning under uncertainty[C]∥Proceedings of 2018 IEEE/RSJ International Conference on Intelligent Robots and Systems.Madrid，Spain:IEEE，2018.
[16］ ASKA， RZANEH A，REZAZADEH A.A new heuristic optimization algorithm for modeling of proton exchange membrane fuel cell: bird mating optimizer[J].International Journal of Energy Research，2015，37(10):1196-1204.
[17］ LITTMAN M L.Markov games as a framework for multi-agent reinforcement learning[C]∥Proceedings of the 7th International Conference on Machine Learning. New Brunswick，NJ，US:Morgan Kaufmann Publishers Inc.，1994.
[18］ LI Y，AN W，WANG Y Q.Deep reinforcement learning with application to air confrontation intelligent decision-making of manned/unmanned aerial vehicle cooperative system[J］.IEEE Access，2020，8:67887-67898.
[19］ LITTMAN M L. Value-function reinforcement learning in Markov games[J].Cognitive Systems Research，2001，2(1):55-66.
[20］ LAUER M，RIEDMILLER M. An algorithm for distributed reinforcement learning in cooperative multi-agent systems[C]∥Proceedings of the 17th International Conference on Machine Learning. San Francisco，CA，US:Morgan Kaufmann Publishers Inc.，2000: 535-542.
[21］杨瑞平，高国华，张立勤.计算机生成兵力[M].北京：国防工业出版社，2013.
YANG R P，GAO G H，ZHANG L Q. omputer generated forces[M].Beijing:National Defense Industry Press，2013.(in Chinese)
[22］ OAINGA. Science，strategy and war: the strategic theory of John Boyd[M].the Netherlands: Eburon Academic Publishers，2005.
[23］ CHEN J，SUN D. Resource constrained multirobot task allocation based on leader-follower coalition methodology[J]. The International Journal of Robotics Research，2011，30(10):1423-1434.
[24］ CRUZ J，CHEN G，LI D，et al. Particle swarm optimization for resource allocation in UAV coperative control[C]∥Proceedings of the AIAA Guidance，Navigation，and Control Conference and Exhibit. Providence，RI，US: AIAA，2004.
[25］熊瑜.改进蚁群算法在武器目标分配中的应用研究[J]. 计算机与数字工程，2014，42(3): 399-402.
XIONG Y.Application of improved ACO in WTA of air defense[J].Computer & Digital Engineering，2014，42(3):399-402.(in Chinese)

[1]	张晚晴，余文斌，李静琳，陈万春. 基于纵程解析解的飞行器智能横程机动再入协同制导[J]. 兵工学报, 2021, 42(7): 1400-1411.
[2]	高昂，董志明，叶红兵，宋敬华，郭齐胜. 基于深度强化学习的巡飞弹突防控制决策[J]. 兵工学报, 2021, 42(5): 1101-1110.
[3]	张浩为，谢军伟，张昭建，宗彬锋，陈唐军. 基于混合自适应遗传算法的相控阵雷达任务调度[J]. 兵工学报, 2017, 38(9): 1761-1770.