
浏览全部资源
扫码关注微信
1. 北京理工大学 宇航学院, 北京 100081
2. 飞行器动力学与控制教育部重点实验室, 北京 100081
3. 北京理工大学重庆创新中心, 重庆 401121
4. 陆空基信息感知与控制全国重点实验室, 北京 100081
5. 中国兵器科学研究院, 北京 100089
Received:24 December 2024,
Published Online:07 May 2025,
Published:31 May 2025
移动端阅览
Zhenlin ZHOU, Teng LONG, Dawei LIU, et al. Path Planning Method for Large-scale UAV Swarms Based on Reinforcement Learning Conflict Resolution[J]. Acta Armamentarii, 2025, 46(5): 241146.
Zhenlin ZHOU, Teng LONG, Dawei LIU, et al. Path Planning Method for Large-scale UAV Swarms Based on Reinforcement Learning Conflict Resolution[J]. Acta Armamentarii, 2025, 46(5): 241146. DOI: 10.12382/bgxb.2024.1146.
面向大规模无人机集群协同作业场景
针对航迹冲突频繁导致集群航迹规划高耗时的问题
开展基于强化学习冲突消解的大规模无人机集群航迹规划方法研究。构建“顶层冲突消解-底层航迹规划”的双层规划架构
降低航迹冲突的时空维度。在顶层冲突消解层
设计基于Rainbow DQN (Deep Q-Networks algorithm)训练框架的冲突消解策略网络
将每个航迹冲突的消解过程转换为二叉树拓展左、右树节点的动作选择过程
实现不同冲突消解顺序与冲突消解结果的映射
减少树节点的遍历
提高冲突消解效率;在底层航迹规划层
将时间维度引入空间避碰策略
提出基于节点重扩展机制的跳点搜索法(Re-planning Jump Point Search
ReJPS)
增加规划可行域
提升航迹冲突的消解能力。仿真结果表明:相比基于CBS (Conflict Based Search)+A
*
方法与CBS+ReJPS航迹规划方法
新方法在最优性相当的前提下
平均规划耗时分别降低了86.64%和19.65%。
In the context of large-scale unmanned aerial vehicle (UAV) swarm cooperative flight scenari
os
the high computational time consumption in swarm path planning is caused by frequent path conflicts. Aiming at the problem above
a large-scale UAV swarm path planning method based on reinforcement learning conflict resolution is developed. A dual-layer planning architecture
comprising a high-level layer of conflict resolution and a low-level layer of path planning
is constructed to reduce the spatial and temporal dimensions of path conflicts. At the high-level layer of conflict resolution
a conflict resolution strategy network based on the Rainbow deep Q-networks (DQN) algorithm training framework is designed. This network transforms the resolution process of each path conflict into the action selection process of left and right tree nodes of a binary tree. This approach maps different conflict resolution sequences to their outcomes
thereby reducing the traversal of tree nodes and improving the efficiency of conflict resolution. At the low-level layer of path planning
the time dimension is incorporated into the spatial collision avoidance strategy. A re-planning jump point search (ReJPS) method based on a node re-expansion mechanism is proposed
which increases the feasible planning domain and enhances the ability to resolve the path conflicts. Simulated results indicate that
compared to the path planning methods based on the conflict-based search (CBS)+A
*
and CBS+ReJPS
the proposed method reduces the average planning time by 86.64% and 19.65%
respectively
while maintaining comparable optimality.
GHOMMAM J , SAAD M , WRIGHT S , et al. Relay manoeuvre based fixed-time synchronized tracking control for UAV transport system [J ] . Aerospace Science and Technology , 2020 , 103 : 105887 .
SHAHI T B , XU C Y , NEUPANE A , et al. Machine learning methods for precision agriculture with UAV imagery: a review [J ] . Electronic Research Archive , 2022 , 30 ( 12 ): 4277 - 4317 .
KHAN A , GUPTA S , GUPTA S K . Emerging UAV technology for disaster detection, mitigation, response, and preparedness [J ] . Journal of Field Robotics , 2022 , 39 ( 6 ): 905 - 955 .
陈亚萍 , 王楠 , 洪华杰 , 等 . 面向多无人平台区域监视任务的信息素正向激励栅格方法 [J ] . 兵工学报 , 2023 , 44 ( 9 ): 2859 - 2870 . DOI: 10.12382/bgxb.2022.0537 http://doi.org/10.12382/bgxb.2022.0537 密集城市地区作战普遍存在区域监视问题,为使我方撤出最高危险区域以减轻损伤或减少人力消耗,采用无人系统执行侦察监视任务极具军事意义和应用价值。面向环境复杂多变且多无人平台初始位置邻近情况下的协同监控任务,针对现有的控制策略在遍历目标空间时多无人平台容易发生冲突的不足且缺少对多无人平台初始位置邻近情况的研究,在半启发式控制策略和栅格法的基础上通过引入信息素改进目标函数并制定冲突消解规则,构建出信息素正向激励栅格法。试验结果表明,信息素正向激励栅格法在冲突消解方面的表现优于现有控制策略,综合性能表现较好,特别在障碍物数量较多时全局平均空闲时间的表现更好,所提方法的有效性和合理性得到了验证。
CHEN Y P , WANG N , HONG H J , et al. Pheromone positive incentive grid method for multi-unmanned platform regional surveillance task [J ] . Acta Armamentarii , 2023 , 44 ( 9 ): 2859 - 2870 . (in Chinese) DOI: 10.12382/bgxb.2022.0537 http://doi.org/10.12382/bgxb.2022.0537 There is a common problem of regional surveillance in operations in densely populated urban areas. In order to evacuate our side from the highest risk area to reduce damage or manpower consumption, the use of unmanned systems to carry out reconnaissance and surveillance tasks is of great military significance and application value. Aiming at collaborative monitoring tasks with complex and ever-changing environments and multiple unmanned platforms with adjacent initial positions, in response to the shortcomings of existing control strategies that are prone to conflicts when traversing the target space and the lack of research on the proximity of initial positions of multiple unmanned platforms, based on semi heuristic control strategies and grid methods, the objective function is improved by introducing pheromones and developing conflict resolution rules, a pheromone positive incentive grid method is constructed. Experimental results showed that the proposed method performed better in conflict resolution than existing control strategies, its overall performance was better, and the global average idle time was better especially when there were many obstacles. The effectiveness and rationality of the proposed method had been verified.
CHANG G N , FU W X , ZHAO J M , et al. Overview of research on intelligent swarm munitions [J/OL ] . Defence Technology , 2024 , DOI : https://doi.org/10.1016/j.dt.2024.08.017. https://doi.org/10.1016/j.dt.2024.08.017. https://doi.org/10.1016/j.dt.2024.08.017. https://doi.org/10.1016/j.dt.2024.08.017.
李军 , 陈士超 . 无人机蜂群关键技术发展综述 [J ] . 兵工学报 , 2023 , 44 ( 9 ): 2533 - 2545 . DOI: 10.12382/bgxb.2023.0514 http://doi.org/10.12382/bgxb.2023.0514 现代战争模式催生了无人机蜂群新型作战样式。系统分析了无人机蜂群利用功能简单的个体,应用自然界蜂群组织算法模型形成高级群体智能行为的概念,介绍无人蜂群的分类和特点,凝练总结无人机蜂群协同组网、协同感知、协同决策、协同控制等关键领域技术及发展现状,针对复杂对抗环境下的实际需求提出集光电探测、通信组网、协同控制、任务规划等多学科于一体的无人机蜂群技术发展的重点和难点,给出无人机蜂群技术发展的途径和举措建议,期望达到牵引无人机蜂群技术发展方向、推动无人机蜂群从技术走向实用的目的。
LI J , CHEN S C . Overview of key technology and its development of drone swarm [J ] . Acta Armamentarii , 2023 , 44 ( 9 ): 2533 - 2545 . (in Chinese) DOI: 10.12382/bgxb.2023.0514 http://doi.org/10.12382/bgxb.2023.0514 Modern warfare modes have given birth to the drone swarms as a new type of operational pattern. The concept of forming the advanced group intelligent behavior obtained by using drones with simple functions, which applies the natural bee swarm organization algorithm,is systematically analyzed. The classification and characteristics of drone swarm are introduced. The key technologies and development status of collaborative networking, sensing, decision-making and control are summarized. Focusing on the practical requirements under complex countermeasure environments, the key points and difficulties in the development of unmanned drone swarm technology which integrates electro-optic detection, communication networking, collaborative control and task planning into one are presented. The development approaches and measures of the drone swarm technologies are also put forward, hoping to achieve the goal of leading the development direction of unmanned drone swarm technology and promoting the corresponding technologies to practical applications.
赵军民 , 何浩哲 , 王少奇 , 等 . 复杂环境下多无人机目标跟踪与避障联合航迹规划 [J ] . 兵工学报 , 2023 , 44 ( 9 ): 2685 - 2696 . DOI: 10.12382/bgxb.2022.0525 http://doi.org/10.12382/bgxb.2022.0525 针对多无人机在密集障碍环境中协同执行地面目标跟踪任务时存在避障能力不足的问题,提出一种基于零空间方法的多无人机目标跟踪与避障联合航迹规划算法。利用Lyapunov导引向量场得到协同对地面目标进行Standoff跟踪时的无人机目标跟踪速度指令,构建障碍物模型和避障人工势场函数,利用人工势场方法得到无人机避障速度指令;基于零空间方法,将避障任务设定为高优先级任务,通过将目标跟踪速度指令向避障任务零空间投影后再与避障速度指令相加的联合航迹规划方法,获取综合后的无人机速度指令。通过仿真分析验证联合航迹规划方法的有效性。仿真结果表明,联合航迹规划方法能够在密集障碍物存在的复杂环境中实时规划有效航迹,保证多无人机避开密集障碍物,持续跟踪目标,并且多无人机具有良好的协同性。
ZHAO J M , HE H Z , WANG S Q , et al. Joint trajectory planning for multiple UAVs target tracking and obstacle avoidance in a complicated environment [J ] . Acta Armamentarii , 2023 , 44 ( 9 ): 2685 - 2696 . (in Chinese) DOI: 10.12382/bgxb.2022.0525 http://doi.org/10.12382/bgxb.2022.0525 In scenarios where multiple UAVs need to collaborate in ground target tracking tasks within obstacle-dense environments, the obstacle avoidance ability may be insufficient. To address this challenge, we propose a joint trajectory planning algorithm for multiple UAVs, enabling them to simultaneously track targets and avoid obstacles using the null-space method. First, Lyapunov guidance vector field is used to obtain the target tracking velocity command for UAVs when they perform standoff tracking to the ground target coordinately. Obstacle model and artificial potential field function for obstacle avoidance are established, and the obstacle avoidance velocity command for UAVs is obtained by using artificial potential field method. Second, based on the null-space method, the obstacle avoidance task is set as a high-priority task, and the integrated UAV velocity command is obtained through the joint trajectory planning method, which projects the target tracking velocity command into the null-space of the obstacle avoidance task and then adds it to the obstacle avoidance velocity command. Through simulation analysis, the effectiveness of the proposed method is verified. Simulated results show that the proposed joint trajectory planning method can plan effective trajectories for multiple UAVs in real time in complex environments with dense obstacles, and ensure that UAVs avoid dense obstacles and maintain target tracking with good coordination between UAVs.
于连波 , 曹品钊 , 石亮 , 等 . 基于改进冲突搜索的多智能体路径规划算法 [J ] . 航空学报 , 2023 , 44 ( 增刊1 ): 727648 .
YU L B , CAO P Z , SHI L , et al. An improved conflict-based search algorithm for multi-agent path planning [J ] . Acta Aeronautica et Astronautica Sinica , 2023 , 44 ( S1 ): 727648 . (in Chinese)
HONG Y K , KIM Y D . Two-stage multicriteria decision-making framework for aircraft conflict resolution [J ] . Journal of Aerospace Information Systems , 2023 , 20 ( 10 ): 596 - 604 .
LIU Z X , CAI K Q , XIE J D , et al. A network-based conflict resolution approach for unmanned aerial vehicle operations in dense nonsegregated airspace [J ] . IEEE Intelligent Transportation Systems Magazine , 2022 , 14 ( 3 ): 212 - 232 .
VAN DEN BERG J , SNOEYINK J , LIN M C , et al. Centralized path planning for multiple robots: optimal decoupling into sequential plans [C ] // Proceedings of Robotics : Science and Systems . Seattle, WA, US : MIT Press , 2009 .
YANG J , YIN D , NIU Y F , et al. Distributed cooperative onboard planning for the conflict resolution of unmanned aerial vehicles [J ] . Journal of Guidance, Control, and Dynamics , 2019 , 42 ( 2 ): 272 - 283 .
徐广通 , 王祝 , 曹严 , 等 . 动态优先级解耦的无人机集群轨迹分布式序列凸规划 [J ] . 航空学报 , 2022 , 43 ( 2 ): 325059 . DOI: 10.7527/S1000-6893.2021.25059 http://doi.org/10.7527/S1000-6893.2021.25059 针对无人机集群轨迹规划高维强耦合特征导致计算复杂度高的难题,提出了动态优先级解耦的序列凸规划方法(DPD-SCP),将耦合的集群轨迹规划问题拆分为若干单机凸规划子问题,通过分布式求解提高集群轨迹规划的计算效率与可扩展性。设计飞行时间驱动的动态优先级解耦机制,降低飞行时间短无人机优先级,挖掘其轨迹调整潜力,消除集群相互规避导致的迭代振荡问题,提升集群轨迹迭代的收敛速度。定制时间一致约束更新准则,避免集群飞行时间非正常增长情况,并理论证明了DPD-SCP方法能够生成满足动力学、避碰与时间一致约束的集群轨迹。仿真结果表明:所提的DPD-SCP方法的求解效率显著优于耦合SCP、串行优先级解耦SCP以及并行解耦SCP方法。
XU G T , WANG Z , CAO Y , et al. Dynamic-priority-decoupled UAV swarm trajectory planning using distributed sequential convex programming [J ] . Acta Aeronautica et Astronautica Sinica , 2022 , 43 ( 2 ): 325059 . (in Chinese) DOI: 10.7527/S1000-6893.2021.25059 http://doi.org/10.7527/S1000-6893.2021.25059 In this paper, a Dynamic-Priority-Decoupled Sequential Convex Programming method (DPD-SCP) is proposed to alleviate the high-computational complexity burden for UAV swarm trajectory planning caused by high-dimensional and strong-coupling features. DPD-SCP splits a coupled swarm trajectory planning problem into several single-UAV convex programming subproblems, and the computational efficiency and scalability are enhanced by utilizing distributed computation. The flight-time-driven dynamic priority decoupled mechanism is designed to improve the convergence rate of swarm trajectory iterations. In this decoupled mechanism, the priority of UAVs with short flight time is lowered to explore the UAV's trajectory adjustment potential and eliminate the oscillation problem due to mutual avoidance of swarms. The time-consistency constraint update criterion is customized to avoid abnormal growth of swarm flight time. Furthermore, it is theoretically validated that DPD-SCP can generate the swarm trajectories that can satisfy the constraints of dynamics, collision avoidance, and time consistency. The simulation results show that the efficiency of DPD-SCP is significantly higher than that of the coupled SCP, serial-priority-decoupled SCP, and parallel-decoupled SCP methods.
REN Z Q , RATHINAM S , CHOSET H . CBSS:a new approach for multiagent combinatorial path finding [J ] . IEEE Transactions on Robotics , 2023 , 39 ( 4 ): 2669 - 2683 .
SHARON G , STERN R , FELNER A , et al. Conflict-based search for optimal multi-agent pathfinding [J ] . Artificial Intelligence , 2015 , 219 : 40 - 66 .
王子晗 , 童向荣 . 基于冲突搜索的多智能体路径规划研究进展 [J ] . 计算机科学 , 2023 , 50 ( 6 ): 358 - 368 . DOI: 10.11896/jsjkx.220800151 http://doi.org/10.11896/jsjkx.220800151 多智能体路径规划是人工智能领域一个经典的搜索问题,基于冲突的搜索算法是当前解决该问题的最优算法之一。文中讨论了多智能体路径规划的基础研究,对国内外近年来基于冲突搜索算法及其变体的研究成果进行了分类,根据改进方式将其变体分为4类,包括分割策略的改进、启发式算法、对典型冲突的处理和次优算法。同时介绍了基于冲突的搜索算法在多智能体路径规划的扩展问题中的应用。最后根据当前算法的优缺点,指出了目前面临的挑战,并针对这些挑战给出了未来可能的研究方向。
WANG Z H , TONG X R . Research progress of multi-agent path finding based on conflict-based search algorithms [J ] . Computer Science , 2023 , 50 ( 6 ): 358 - 368 . (in Chinese) DOI: 10.11896/jsjkx.220800151 http://doi.org/10.11896/jsjkx.220800151 Multi-agent path finding is a classic search problem in the field of artificial intelligence.Conflict-based search algorithm is one of the best algorithms to solve this problem.This paper discusses the basic research of multi-agent path finding,and classifies the research results based on conflict search algorithms and their variants in recent years.According to the improved ways,the variants are divided into four categories,including segmentation strategy improvement,heuristic algorithm,bounded suboptimal algorithm and typical conflict processing.It also introduces the application of the conflict-based search algorithm to the extended problem of multi-agent path finding.Finally,according to the advantages and disadvantages of the current algorithm,the existing challenges are pointed out.In view of these challenges,the possible research directions in the future are given.
REN Z Q , RATHINAM S , CHOSET H . A conflict-based search framework for multiobjective multiagent path finding [J ] . IEEE Transactions on Automation Science and Engineering , 2023 , 20 ( 2 ): 1262 - 1274 .
BOYARSKI E , FELNER A , STERN R , et al. ICBS:the improved conflict-based search algorithm for multi-agent pathfinding: extended abstract [C ] // Proceedings of the 8th Annual International Symposium on Combinatorial Search. Ein Gedi, the Dead Sea , Israel : Israel Science Foundation , 2015 : 223 - 225 .
SHARON G , STERN R , FELNER A , et al. Meta-agent conflict-based search for optimal multi-agent path finding [C ] // Proceedings of the 5th Annual Symposium on Combinatorial Search. Niagara Falls, Ontario , Canada : Association for the Advancement of Artificial Intelligence , 2012 : 97 - 104 .
HUANG T A , KOENIG S , DILKINA B . Learning to resolve conflicts for multi-agent path finding with conflict-based search [C ] //Proceedings of the AAAI Conference on Artificial Intelligence.Reston, VA, US:AAAI , 2021 : 11246 - 11253 .
周熙栋 , 张辉 , 陈波 . 非结构化场景下基于改进JPS算法的移动机器人路径规划 [J ] . 控制与决策 , 2024 , 39 ( 2 ): 474 - 482 .
ZHOU X D , ZHANG H , CHEN B . Mobile robot path planning based on improved JPS algorithm in unstructured scenarios [J ] . Control and Decision , 2024 , 39 ( 2 ): 474 - 482 . (in Chinese)
ZHANG J C , AN Y Q , CAO J N , et al. UAV trajectory planning for complex open storage environments based on an improved RRT algorithm [J ] . IEEE Access , 2023 , 11 : 23189 - 23204 .
MAO T X , DENG H . Path planning of slender tensegrities based on the artificial potential field method [J ] . AIAA Journal , 2023 , 61 ( 5 ): 2255 - 2265 .
HESSEL M , MODAYIL J , VAN HASSELT H , et al. Rainbow: combining improvements in deep reinforcement learning [C ] //Proceedings of the AAAI Conference on Artificial Intelligence.New Orleans, LA, US:AAAI , 2018 : 3215 - 3222 .
JI S W , XU W , YANG M , et al. 3D convolutional neural networks for human action recognition [J ] . IEEE Transactions on Pattern Analysis and Machine Intelligence , 2013 , 35 ( 1 ): 221 - 231 . We consider the automated recognition of human actions in surveillance videos. Most current methods build classifiers based on complex handcrafted features computed from the raw inputs. Convolutional neural networks (CNNs) are a type of deep model that can act directly on the raw inputs. However, such models are currently limited to handling 2D inputs. In this paper, we develop a novel 3D CNN model for action recognition. This model extracts features from both the spatial and the temporal dimensions by performing 3D convolutions, thereby capturing the motion information encoded in multiple adjacent frames. The developed model generates multiple channels of information from the input frames, and the final feature representation combines information from all channels. To further boost the performance, we propose regularizing the outputs with high-level features and combining the predictions of a variety of different models. We apply the developed models to recognize human actions in the real-world environment of airport surveillance videos, and they achieve superior performance in comparison to baseline methods.
XIANG J , CHEN J , LIU Y C . Hybrid multiscale search for dynamic planning of multi-agent drone traffic [J ] . Journal of Guidance, Control, and Dynamics , 2023 , 46 ( 10 ): 1963 - 1974 .
0
Views
236
下载量
0
CNKI被引量
Publicity Resources
Related Articles
Related Author
Related Institution
京公网安备11010802024360号