李超 , 王瑞星 , 黄建忠 , 等 . 稀疏奖励下基于强化学习的无人集群自主决策与智能协同 [J ] . 兵工学报 , 2023 , 44 ( 6 ): 1537 - 1546 . DOI: 10.12382/bgxb.2022.0177 http://doi.org/10.12382/bgxb.2022.0177 无人集群将深刻地塑造战争样式,为提升无人集群自主决策算法能力,对异构无人集群攻防对抗自主决策方法进行研究。对无人集群对抗模型设计进行总体概述,并对无人集群攻防对抗场景进行模型设计;针对无人集群自主决策采用强化学习技术广泛存在的稀疏奖励问题,提出基于局部回报重塑的奖励机制设定方法;在此基础上叠加优先经验回放,有效地改善稀疏奖励问题;通过程序仿真和演示系统设计,验证该方法的优越性。该方法的研究将加速基于强化学习技术的无人集群自主决策算法网络收敛过程,对无人集群自主决策算法研究具有重要意义。

LI C , WANG R X , HUANG J Z , et al . Autonomous decision-making and intelligent collaboration of UAV swarms based on reinforcement learning with sparse rewards [J ] . Acta Armamentarii , 2023 , 44 ( 6 ): 1537 - 1546 . (in Chinese) DOI: 10.12382/bgxb.2022.0177 http://doi.org/10.12382/bgxb.2022.0177 UAV swarms will profoundly shape the pattern of warfare. In order to improve the autonomous decision-making algorithm capability of UAV swarms, the autonomous decision-making method for heterogeneous UAV swarm attack-defense confrontation scenarios is studied. An overview of the design of the UAV swarm confrontation model and the model design of the UAV swarm attack-defense confrontation scenario are carried out. To solve the sparse reward problem which widely exists in the reinforcement learning technology in the autonomous decision-making of the UAV swarm, a reward mechanism setting method based on local reward reshaping is proposed. And then, the prioritized experience replay is superimposed, which effectively improves the sparse reward problem. Finally, the superiority of this method is verified by simulation and demonstration system design. This study will accelerate the network convergence process of the autonomous decision-making algorithm for UAV swarms based on reinforcement learning technology, which is of great significance to the research on autonomous decision-making algorithms of UAV swarms.

张哲 , 吴剑 , 代冀阳 , 等 . 基于改进A * 算法的多无人机协同战术规划 [J ] . 兵工学报 , 2020 , 41 ( 12 ): 2530 - 2539 . DOI: 10.3969/j.issn.1000-1093.2020.12.019 http://doi.org/10.3969/j.issn.1000-1093.2020.12.019 多无人机协同作战是未来无人机作战方式的重要发展趋势。为增强多无人机系统的任务执行能力，提高系统整体作战效能并实现高效资源分配和调度，提出一种基于改进A * 算法的多无人机协同战术规划方法。按照离线规划和重规划两方面，设计战役层和战术层的作战目标迭代优化方案；建立编队协同作战的数学模型，以编队成员间的时间协同和碰撞协同代价为变量，得到多约束条件下的综合编队目标函数；结合多层变步长搜索策略和单步扩展的搜索方式，基于改进A * 算法，用于求解复杂战场环境下的多无人机编队协同作战航路。分别利用改进A * 算法和传统A * 算法进行对比仿真实验。仿真结果表明，多无人机协同战术规划方法能够较好地完成作战任务，改进A * 算法能够获得更优的航路，从而验证了所提算法的有效性。

ZHANG Z , WU J , DAI J Y , et al . Cooperative tactical planning for multi-UAVs based on improved A * algorithm [J ] . Acta Armamentarii , 2020 , 41 ( 12 ): 2530 - 2539 . (in Chinese)

袁静妮 , 杨林 , 唐晓峰 , 等 . 基于改进RRT*与行驶轨迹优化的智能汽车运动规划 [J ] . 自动化学报 , 2022 , 48 ( 12 ): 2941 - 2950 .

YUAN J N , YANG L , TANG X F , et al . Autonomous vehicle motion planning based on improved RRT* algorithm and trajectory optimization [J ] . Acta Automatica Sinica , 2022 , 48 ( 12 ): 2941 - 2950 . (in Chinese)

孙鹏耀 , 黄炎焱 , 潘尧 . 基于改进势场法的移动机器人路径规划 [J ] . 兵工学报 , 2020 , 41 ( 10 ): 2106 - 2121 . DOI: 10.3969/j.issn.1000-1093.2020.10.021 http://doi.org/10.3969/j.issn.1000-1093.2020.10.021 针对传统势场法存在的路径不被识别、局部极小陷阱、振荡问题，提出适用于复杂障碍环境情况下机器人路径规划、结合多行为策略与可变影响范围的势场法。通过对障碍物影响范围做可变处理，消除问题共同必要条件，提前规避路径不识别、多障碍区导致的振荡、多障碍区导致的局部极小陷阱3个问题；采取新的步进与振荡分类方式，设计多行为行动策略，并给出各行为的准确起止条件，通过预判问题的共同表现形式以及起止条件的衔接进行行为切换，提前规避单障碍区导致的局部极小陷阱和单障碍区导致的振荡2个问题；基于数学仿真软件MATLAB平台的仿真结果验证所提方法在战场复杂障碍环境下的有效性与稳定性，与传统势场法、动态窗口法、A星算法、快速随机树算法的对比结果更突出了该方法的可行性与优越性。

SUN P Y , HUANG Y Y , PAN Y . Path planning of mobile robots based on improved potential field algorithm [J ] . Acta Armamentarii , 2020 , 41 ( 10 ): 2106 - 2121 . (in Chinese) DOI: 10.3969/j.issn.1000-1093.2020.10.021 http://doi.org/10.3969/j.issn.1000-1093.2020.10.021 In view of the problems existing in the traditional potential field algorithm (PFA), such as unrecognized path, local minimal trap and oscillation, a potential field algorithm combining multi-behavior strategy and obstacles variable affected range, and the path planning of robots suitable for the complex obstacle environment is proposed. The obstacles affected range can be changed to eliminate the common necessary conditions of the above problems, so as to avoid unrecognized path, oscillation caused by multiple obstacles and local minimal trap caused by multiple obstacles in advance. Based on the new classification method for step-by-step and oscillation, the multi-behavior strategy is designed with the exact starting and ending conditions. The behaviors are switched by predicting the common expression of problems and the connection of starting and ending conditions, thus avoiding the local minimum trap caused by single obstacle and the oscillation caused by single obstacle in advance. The simulated results based on MATLAB verify the effectiveness and stability of the proposed method in the complex battlefield obstacle environment, and the proposed method has the feasibility for path planning compared with potential field algorithm, dynamic window approach, A-star algorithm and rapid-exploration random tree algorithm.

赵鹏程 , 宋保维 , 毛昭勇 , 等 . 基于改进的复合自适应遗传算法的UUV水下回收路径规划 [J ] . 兵工学报 , 2022 , 43 ( 10 ): 2598 - 2608 .

ZHAO P C , SONG B W , MAO Z Y , et al . Path planning for UUV underwater recovery based on improved composite adaptive genetic algorithm [J ] . Acta Armamentarii , 2022 , 43 ( 10 ): 2598 - 2608 . (in Chinese) DOI: 10.12382/bgxb.2021.0474 http://doi.org/10.12382/bgxb.2021.0474 Mutations of traditional genetic algorithms generate new paths in a simple and random manner, which negatively influence the evolutionary performance of the algorithms and makes it easy for them to fall into the trap of local optimality. Moreover, genetic algorithms are usually used together with the grid method for path planning, and the optimal path obtained is not always the shortest path for UUV recovery path planning, and the UUV mobility performance might conflict with the optimal path.An improved genetic algorithm with UUV mobility constraints is thus proposed. The concept of environment complexity is proposed to analyze the specific value of mobility constraints, so that path planning can be adapted to UUV mobility, and the algorithm results can be more practical. The compound adaptive mutation strategy is proposed to control the adaptive evolution of the mutated individuals in the iterative process. When the population evolution stagnates after a certain number of iterations, the optimal individual is guided for a two-stage adaptive mutation so that the optimal path approaches the approximate global optimal solution, and the convergence rate of the algorithm is effectively improved. The algorithm comparison simulation results based on MATLAB software show that the optimal path generated by the improved compound adaptive genetic algorithm is smoother and shorter in length compared with the optimal path of genetic algorithm and adaptive genetic algorithm in generally complex water area and complex water area, which demonstrates that the improved compound adaptive genetic algorithm has better convergence performance and superiority seeking ability in path planning and is more feasible and superior.

张瑜 , 宋荆洲 , 张琪祁 . 基于改进动态窗口法的户外清扫机器人局部路径规划 [J ] . 机器人 , 2020 , 42 ( 5 ): 617 - 625 . DOI: 10.13973/j.cnki.robot.190649 http://doi.org/10.13973/j.cnki.robot.190649 针对清扫机器人在停车场结构化路面下存在的加速度过大、路径偏离全局路径过大等问题，提出了一种改进的动态窗口法（DWA）．首先，为了限制小车加速度的范围，对DWA速度空间的动力学约束进行优化，避免出现过大的加速度导致轮胎垂直载荷过小的状况．然后，基于激光里程计对轨迹推算环节进行实时误差补偿，解决停车场路面下路径偏离全局路径较大的问题．最后将改进的DWA应用于四轮独立驱动、独立转向的清扫机器人上进行对比实验．实验结果表明，在相同的全局路径、相同的路况下，改进DWA的路径平均误差较传统DWA减小了约60%，轮胎垂直载荷在不同路况下也大大提高，验证了本文方法的有效性和可靠性．

ZHANG Y , SONG J Z , ZHANG Q Q . Local path planning of outdoor cleaning robot based on an improved DWA [J ] . Robot , 2020 , 42 ( 5 ): 617 - 625 . (in Chinese) DOI: 10.13973/j.cnki.robot.190649 http://doi.org/10.13973/j.cnki.robot.190649 Aiming at the problems of cleaning robots on the structured road surface in the parking lot, such as excessive acceleration and large path deviation from the global path, an improved dynamic window approach (DWA) is proposed. In order to limit the acceleration range of the vehicle, the dynamic constraints of the DWA speed space are optimized firstly to avoid the situation that the excessive acceleration leads to very small vertical tire load. Then, error compensation for trajectory estimation is performed in real time based on the laser odometer, to solve the problem that the path deviates greatly from the global path on the parking lot road. Finally, the improved DWA is applied to a cleaning robot with four wheels of independent driving and independent steering to conduct comparative experiments. Experimental results show that the average path error of the improved DWA is reduced by about 60% compared with the traditional DWA on the basis of the same global path and road conditions, and the vertical load of tires is also greatly increased under different road conditions, which verify the effectiveness and reliability of the proposed method.

高明 , 唐洪 , 张鹏 . 机器人集群路径规划技术研究现状 [J ] . 国防科技大学学报 , 2021 , 43 ( 1 ): 127 - 138 .

GAO M , TANG H , ZHANG P . Survey of path planning technologies for robots swarm [J ] . Journal of National University of Defense Technology , 2021 , 43 ( 1 ): 127 - 138 . (in Chinese)

李波 , 杨志鹏 , 贾卓然 , 等 . 一种无监督学习型神经网络的无人机全区域侦察路径规划 [J ] . 西北工业大学学报 , 2021 , 39 ( 1 ): 77 - 84 .

LI B , YANG Z P , JIA Z R , et al . An unsupervised learning neural network for planning UAV full-area reconnaissance path [J ] . Journal of Northwestern Polytechnical University , 2021 , 39 ( 1 ): 77 - 84 . (in Chinese)

LUAN P G , THINH N T . Hybrid genetic algorithm based smooth global-path planning for a mobile robot [J ] . Mechanics Based Design of Structures and Machines , 2023 , 51 ( 3 ): 1758 - 1774 .

张震 , 方群 , 宋金丰 , 等 . 基于协同粒子群算法的航天器集群动态路径规划算法研究 [J ] . 西北工业大学学报 , 2021 , 39 ( 6 ): 1222 - 1232 .

ZHANG Z , FANG Q , SONG J F , et al . Research on dynamic path planning algorithm of spacecraft cluster based on cooperative particle swarm algorithm [J ] . Journal of Northwestern Polytechnical University , 2021 , 39 ( 6 ): 1222 - 1232 . (in Chinese)

WU P F , LI T , SONG G F . UCAV path planning based on improved chaotic particle swarm optimization [C ] // Proceedings of the 2020 Chinese Automation Congress. Shanghai , China : IEEE , 2020 : 1069 - 1073 .

TAN Y , ZHU Y C . Fireworks algorithm for optimization [C ] // Proceedings of the International Conference on Advances in Swarm Intelligence . Beijing, China : Springer , 2010 : 355 - 364 .

余冬华 , 郭茂祖 , 刘晓燕 , 等 . 改进选择策略的烟花算法 [J ] . 控制与决策 , 2020 , 35 ( 2 ): 389 - 395 .

YU D H , GUO M Z , LIU X Y , et al . An improved selection strategy of firework algorithm [J ] . Control and Decision , 2020 , 35 ( 2 ): 389 - 395 . (in Chinese)

滕志军 , 李哲 , 王幸幸 , 等 . 无线传感器网络中基于μ律爆炸算子的烟花虚拟力混合覆盖策略 [J ] . 控制理论与应用 , 2023 , 40 ( 5 ): 817 - 824 .

TENG Z J , LI Z , WANG X X , et al . Fireworks virtual force mixture covering strategy based on μ-law explosion operator in wireless sensor networks [J ] . Control Theory & Applications , 2023 , 40 ( 5 ): 817 - 824 . (in Chinese)

邹适宇 , 李复名 , 谢爱平 , 等 . 基于改进烟花算法的资源分配 [J ] . 航空学报 , 2021 , 42 ( 12 ): 324716 . DOI: 10.7527/S1000-6893.2020.24716 http://doi.org/10.7527/S1000-6893.2020.24716 资源分配问题作为一个NP-Hard问题，在云计算、无线电、卫星调度、多无人机协同作业等领域皆有研究需求，是一个共性的数学问题。烟花算法作为一种智能优化算法，具有求解大规模资源分配问题的能力，但也存在求解精度低等问题。为了提高传统烟花算法的计算效率和全局寻优能力，提出一种改进烟花算法，用遗传算法中的变异算子替代高斯变异操作，并增加模拟退火流程。最后在多无人机协同作业任务分配数学模型上进行仿真验证，实验结果表明在收敛速度以及计算精度方面，该算法均优于其余3种烟花算法。

ZOU S Y , LI F M , XIE A P , et al . Resource allocation based on improved fireworks algorithm [J ] . Acta Aeronautica et Astronautica Sinica , 2021 , 42 ( 12 ): 324716 . (in Chinese) DOI: 10.7527/S1000-6893.2020.24716 http://doi.org/10.7527/S1000-6893.2020.24716 As a NP-hard problem, the resource allocation problem is a common mathematical problem in cloud computing, radio, satellite scheduling, multi-UAV collaborative task allocation, and many other fields. As an intelligent optimization algorithm, the fireworks algorithm has the ability to solve large-scale resource allocation problems, but also has some problems, such as low solving precision and probability to fall into locally optimal solution. To improve the computational efficiency and global optimization ability of the traditional fireworks algorithm, this paper proposes an improved fireworks algorithm, which uses the mutation operator in genetic algorithm to replace Gaussian mutation operation in the traditional fireworks algorithm and adds the simulated annealing process in each iteration. Finally, the performance of the algorithm is verified by simulation on a mathematical model for multi-UAV cooperative task assignment, and the results show that the algorithm is superior to other three fireworks algorithms in terms of convergence speed and solving precision.

余修武 , 李佩 , 刘永 , 等 . 基于烟花优化的WSN数据融合算法 [J ] . 华中科技大学学报 , 2023 , 51 ( 5 ): 112 - 118 .

YU X W , LI P , LIU Y , et al . Data fusion of WSN based on fireworks algorithm optimization [J ] . Journal of Huazhong University of Science and Technology , 2023 , 51 ( 5 ): 112 - 118 . (in Chinese)

申晓宁 , 许笛 , 宋丽妍 , 等 . 采用离散烟花算法的移动群智感知异构任务分配 [J ] . 计算机工程与科学 , 2023 , 45 ( 2 ): 321 - 331 .

SHEN X N , XU D , SONG L Y , et al . Heterogeneous task assignment of mobile crowdsensing based on a discrete fireworks algorithm [J ] . Computer Engineering & Science , 2023 , 45 ( 2 ): 321 - 331 . (in Chinese)

ZHAO H T , ZHANG C S , NING J X . A core firework updating information guided dynamic fireworks algorithm for global optimization [J ] . Soft Computing , 2020 , 24 ( 2 ): 1185 - 1211 .

YU J Y , GUO J S , ZHANG X F , et al . A Novel tent-levy fireworks algorithm for the UAV task allocation problem under uncertain environment [J ] . IEEE Access , 2022 , 10 : 102373 - 102385 .

薛裕颖 , 张祥银 , 张国梁 , 等 . 基于量子行为烟花算法的移动机器人路径规划及平滑 [J ] . 控制理论与应用 , 2019 , 36 ( 9 ): 1398 - 1408 .

XUE Y Y , ZHANG X Y , ZHANG G L , et al . Path planning and smoothing based on quantum-behaved fireworks algorithm for mobile robot [J ] . Control Theory & Applications , 2019 , 36 ( 9 ): 1398 - 1408 . (in Chinese)

张玮 , 马焱 , 赵捍东 , 等 . 基于改进烟花-蚁群混合算法的智能移动体避障路径规划 [J ] . 控制与决策 , 2019 , 34 ( 2 ): 335 - 343 .

ZHANG W , MA Y , ZHAO H D , et al . Obstacle avoidance path planning of intelligent mobile based on improved fireworks-ant colony hybrid algorithm [J ] . Control and Decision , 2019 , 34 ( 2 ): 335 - 343 . (in Chinese)

ZHANG X Y , XIA S , ZHANG T , et al . Hybrid FWPS cooperation algorithm based unmanned aerial vehicle constrained path planning [J ] . Aerospace Science and Technology , 2021 , 118 : 107004 .