REYNOLDS C W . Flocks , herds and schools : a distributed behavior model [J ] . ACM SIGGRAPH Computer Graphics , 1987 , 87 : 27 - 31

COUZIN I D , KRAUSE J , JAMES R , et al . Collective memory and spatial sorting in animal groups [J ] . Journal of Theoretical Biology , 2002 , 218 ( 1 ): 1 - 11 . We present a self-organizing model of group formation in three-dimensional space, and use it to investigate the spatial dynamics of animal groups such as fish schools and bird flocks. We reveal the existence of major group-level behavioural transitions related to minor changes in individual-level interactions. Further, we present the first evidence for collective memory in such animal groups (where the previous history of group structure influences the collective behaviour exhibited as individual interactions change) during the transition of a group from one type of collective behaviour to another. The model is then used to show how differences among individuals influence group structure, and how individuals employing simple, local rules of thumb, can accurately change their spatial position within a group (e.g. to move to the centre, the front, or the periphery) in the absence of information on their current position within the group as a whole. These results are considered in the context of the evolution and ecological importance of animal groups.

ULRICH Y , SARAGOSTI J , TOKITA C K , et al . Fitness benefits and emergent division of labour at the onset of group living [J ] . Nature , 2018 , 560 ( 7720 ): 635 - 638 . DOI: 10.1038/s41586-018-0422-6 http://doi.org/10.1038/s41586-018-0422-6

CHEN H , DUAN H B . Multiple unmanned aerial vehicle autonomous formation via wolf packs mechanism [C ] //Proceedings of IEEE International Conference on Aircraft Utility Systems.Washington,D.C.,US:IEEE , 2016 .

CASSIDY K A , MACNULTY D R , STAHLER D R , et al . Group composition effects on aggressive interpack interactions of gray wolves in Yellowstone National Park [J ] . Behavioral Ecollgy , 2015 , 26 : 637 - 642 .

DUCATELLE F , DI CARO G A , PINCIROLI C , et al . Self-organized cooperation between robotic swarms [J ] . Swarm Intelligence , 2011 , 5 ( 2 ): 73 - 96 . DOI: 10.1007/s11721-011-0053-0 http://doi.org/10.1007/s11721-011-0053-0 http://link.springer.com/10.1007/s11721-011-0053-0 http://link.springer.com/10.1007/s11721-011-0053-0

武文亮 , 周兴社 , 沈博 , 等 . 集群机器人系统特性评价研究综述 [J ] . 自动化学报 , 2022 , 48 ( 5 ): 1153 - 1172 . DOI: 10.16383/j.aas.c200964 https://dx.doi.org/10.16383/j.aas.c200964 .

WU W L , ZHOU X S , SHEN B , et al . A review of swarm robotic systems property evaluation research [J ] . Acta Automatica Sinica , 2022 , 48 ( 5 ): 1153 - 1172 . DOI: 10.16383/j.aas.c200964 https://dx.doi.org/10.16383/j.aas.c200964 . (in Chinese)

RAIBERT M , BLANKESPOOR K , NELSON G , et al . BigDog, the rough-terrain quadruped robot [C ] //Proceedings of the 17th World Congress.Seoul,Korea:IFAC , 2008 , 41 ( 2 ): 10822 - 10825 .

SEMINI C , TSAGARAKIS N G , GUGLIELMINO E , et al . Design of HyQ-a hydraulically and electrically actuated quadruped robot [J ] . Proceedings of the Institution of Mechanical Engineers, Part I:Journal of Systems and Control Engineering , 2011 , 225 ( 6 ): 831 - 849 . DOI: 10.1177/0959651811402275 http://doi.org/10.1177/0959651811402275 http://journals.sagepub.com/doi/10.1177/0959651811402275 http://journals.sagepub.com/doi/10.1177/0959651811402275 A new versatile hydraulically powered quadruped robot ( HyQ) has been developed to serve as a platform to study not only highly dynamic motions, such as running and jumping, but also careful navigation over very rough terrain. HyQ stands 1 m tall, weighs roughly 90 kg, and features 12 torque-controlled joints powered by a combination of hydraulic and electric actuators. The hydraulic actuation permits the robot to perform powerful and dynamic motions that are hard to achieve with more traditional electrically actuated robots. This paper describes design and specifications of the robot and presents details on the hardware of the quadruped platform, such as the mechanical design of the four articulated legs and of the torso frame, and the configuration of the hydraulic power system. Results from the first walking experiments are presented, along with test studies using a previously built prototype leg.

HUTTER M , GEHRING C , JUD D , et al . Anymal-a highly mobile and dynamic quadrupedal robot [C ] //Proceedings of 2016 IEEE/RSJ International Conference on Intelligent Robots and Systems.Washington,D.C. , US : IEEE , 2016 : 38 - 44 .

NAGAKUBO A , HIROSE S . Walking and running of the quadruped wall-climbing robot [C ] //Proceedings of the 1994 IEEE International Conference on Robotics and Automation.Washington,D.C.,US:IEEE , 1994 , 2 : 1005 - 1012 .doi: 10.1109/Robot.1994.351225 https://dx.doi.org/10.1109/Robot.1994.351225 .

柴汇 , 孟健 , 荣学文 , 等 . 高性能液压驱动四足机器人SCalf的设计与实现 [J ] . 机器人 , 2014 , 36 ( 4 ): 385 - 391 .

CHAI H , MENG J , RONG X W , et al. Design and implementation of SCalf, an advanced hydraulic quadruped robot [J ] . Robot , 2014 , 36 ( 4 ): 385 - 391 . (in Chinese)

PAN Y , GAO F , QI C K , et al . Human-tracking strategies for a six-legged rescue robot based on distance and view [J ] . Chinese Journal of Mechanical Engineering , 2016 , 29 ( 2 ): 219 - 230 . DOI: 10.3901/CJME.2015.1212.146 http://doi.org/10.3901/CJME.2015.1212.146 http://link.springer.com/10.3901/CJME.2015.1212.146 http://link.springer.com/10.3901/CJME.2015.1212.146

FOLKERTSMA G A , KIM S , STRAMIGIOLI S . Parallel stiffness in a bounding quadruped with flexible spine [C ] //Proceedings of 2012 IEEE/RSJ International Conference on Intelligent Robots and Systems.Washington,D.C. , US : IEEE , 2012 : 2210 - 2215 .

BARASUOL V , VILLARREAL-MAGAÑA O A , SANGIAH D , et al . Highly-integrated hydraulic smart actuators and smart manifolds for high-bandwidth force control [J ] . Frontiers in Robotics and AI , 2018 , 5 : 51 . DOI: 10.3389/frobt.2018.00051 http://doi.org/10.3389/frobt.2018.00051 Hydraulic actuation is the most widely used alternative to electric motors for legged robots and manipulators. It is often selected for its high power density, robustness and high-bandwidth control performance that allows the implementation of force/impedance control. Force control is crucial for robots that are in contact with the environment, since it enables the implementation of active impedance and whole body control that can lead to a better performance in known and unknown environments. This paper presents the hydraulic developed by Moog in collaboration with IIT, as well as smart manifolds for rotary hydraulic actuators. The ISA consists of an additive-manufactured body containing a hydraulic cylinder, servo valve, pressure/position/load/temperature sensing, overload protection and electronics for control and communication. The ISA v2 and ISA v5 have been specifically designed to fit into the legs of IIT's hydraulic quadruped robots HyQ and HyQ-REAL, respectively. The key features of these components tackle 3 of today's main challenges of hydraulic actuation for legged robots through: (1) built-in controllers running inside integrated electronics for high-performance control, (2) low-leakage servo valves for reduced energy losses, and (3) compactness thanks to metal additive manufacturing. The main contributions of this paper are the derivation of the representative dynamic models of these highly integrated hydraulic servo actuators, a control architecture that allows for high-bandwidth force control and their experimental validation with application-specific trajectories and tests. We believe that this is the first work that presents additive-manufactured, highly integrated hydraulic smart actuators for robotics.Copyright © 2018 Barasuol, Villarreal-Magana, Sangiah, Frigerio, Baker, Morgan, Medrano-Cerda, Caldwell and Semini.

华子森 . 面向四足机器人的被动柔顺液压伺服作动器设计, 分析与应用 [D ] . 济南 : 山东大学 , 2020 .

HUA Z S . Design, analysis and application of passive compliant hydraulic servo actuator for quadruped robot [D ] . Jinan : Shandong University , 2020 . (in Chinese)

WENSING P M , WANG A , SEOK S , et al . Proprioceptive actuator design in the mit cheetah:impact mitigation and high-bandwidth physical interaction for dynamic legged robots [J ] . IEEE Transactions on Robotics , 2017 , 33 ( 3 ): 509 - 522 . DOI: 10.1109/TRO.2016.2640183 http://doi.org/10.1109/TRO.2016.2640183 http://ieeexplore.ieee.org/document/7827048/ http://ieeexplore.ieee.org/document/7827048/

BLEDT G , POWELL M J , KATZ B , et al . MIT Cheetah 3:Design and control of a robust, dynamic quadruped robot [C ] //Proceedings of 2018 IEEE/RSJ International Conference on Intelligent Robots and Systems.Washington,D.C. , US : IEEE , 2018 : 2245 - 2252 .

YANG C Y , YUAN K , ZHU Q G , et al . Multi-expert learning of adaptive legged locomotion [J ] . Science Robotics , 2020 , 5 ( 49 ): eabb 2174 .

LEE J , HWANGBO J , WELLHAUSEN L , et al. Learning quadrupedal locomotion over challenging terrain [J ] . Science Robotics , 2020 , 5 ( 47 ): eabc5986 . DOI: 10.1126/scirobotics.abc5986 http://doi.org/10.1126/scirobotics.abc5986 https://www.science.org/doi/10.1126/scirobotics.abc5986 https://www.science.org/doi/10.1126/scirobotics.abc5986 A learning-based locomotion controller enables a quadrupedal ANYmal robot to traverse challenging natural environments.

MIKI T , LEE J , HWANGBO J , et al. Learning robust perceptive locomotion for quadrupedal robots in the wild [J ] . Science Robotics , 2022 , 7 ( 62 ): eabk2822 . DOI: 10.1126/scirobotics.abk2822 http://doi.org/10.1126/scirobotics.abk2822 https://www.science.org/doi/10.1126/scirobotics.abk2822 https://www.science.org/doi/10.1126/scirobotics.abk2822 Legged robots that can operate autonomously in remote and hazardous environments will greatly increase opportunities for exploration into underexplored areas. Exteroceptive perception is crucial for fast and energy-efficient locomotion: Perceiving the terrain before making contact with it enables planning and adaptation of the gait ahead of time to maintain speed and stability. However, using exteroceptive perception robustly for locomotion has remained a grand challenge in robotics. Snow, vegetation, and water visually appear as obstacles on which the robot cannot step or are missing altogether due to high reflectance. In addition, depth perception can degrade due to difficult lighting, dust, fog, reflective or transparent surfaces, sensor occlusion, and more. For this reason, the most robust and general solutions to legged locomotion to date rely solely on proprioception. This severely limits locomotion speed because the robot has to physically feel out the terrain before adapting its gait accordingly. Here, we present a robust and general solution to integrating exteroceptive and proprioceptive perception for legged locomotion. We leverage an attention-based recurrent encoder that integrates proprioceptive and exteroceptive input. The encoder is trained end to end and learns to seamlessly combine the different perception modalities without resorting to heuristics. The result is a legged locomotion controller with high robustness and speed. The controller was tested in a variety of challenging natural and urban environments over multiple seasons and completed an hour-long hike in the Alps in the time recommended for human hikers.

WEI H X , CAI Y P , LI H Y , et al . Sambot: a self-assembly modular robot for swarm robot [C ] //Proceedings of IEEE International Conference on Robotics and Automation. Anchorage, AK, US:IEEE , 2010 .

DORIGO M , TUCI E , GROΒ R , et al. The swarm-bots project [C ] //Proceedings of International Workshop on Swarm Robotics. Berlin, Heidelberg , Germany : Springer , 2004 : 31 - 44 .

JDEED M , ZHEVZHYK S , STEINKELLNER F , et al . Spiderino-a low-cost robot for swarm research and educational purposes [C ] //Proceedings of the 2017 13rd Workshop on Intelligent Solutions in Embedded Systems. Washington,D.C.,US:IEEE , 2017 : 35 - 39 .

HOWARD A , PARKER L E , SUKHATME G S . Experiments with a large heterogeneous mobile robot team: exploration, mapping, deployment and detection [J ] . The International Journal of Robotics Research , 2006 , 25 ( 5/6 ): 431 - 447 . DOI: 10.1177/0278364906065378 http://doi.org/10.1177/0278364906065378 http://journals.sagepub.com/doi/10.1177/0278364906065378 http://journals.sagepub.com/doi/10.1177/0278364906065378 We describe the design and experimental validation of a large heterogeneous mobile robot team built for the DARPA Software for Distributed Robotics (SDR) program. The core challenge for the SDR program was to develop a multi-robot system capable of carrying out a specific mission: to deploy a large number of robots into an unexplored building, map the building interior, detect and track intruders, and transmit all of the above information to a remote operator. To satisfy these requirements, we developed a heterogeneous robot team consisting of approximately 80 robots. We sketch the key technical elements of this team, focusing on the novel aspects, and present selected results from supervised experiments conducted in a 600 m 2 indoor environment.

ZHU P X , REN W . Multi-Robot joint visual-inertial localization and 3-D moving object tracking [C ] //Proceedings of the IEEE/RSJ International Conference on Intelligent Robots and Systems. Washington,D.C. , US : IEEE , 2020 : 11573 - 11580 .

PAPAIOANNOU S , KOLIOS P , PANAYIOTOU C G , et al . Cooperative simultaneous tracking and jamming for disabling a rogue drone [C ] //Proceedings of IEEE/RSJ International Conference on Intelligent Robots and Systems. Washington,D.C. , US : IEEE , 2020 .

SUNG Y , BUDHIRAJA A K , WILLIAMS R K , et al . Distributed simultaneous action and target assignment for Multi-Robot Multi-Target tracking [C ] //Proceedings of IEEE International Conference on Robotics and Automation. Washington,D.C. , US : IEEE , 2018 .

FRANCHI A , PETITTI A , RIZZO A . Distributed estimation of the inertial parameters of an unknown load via multi-robot manipulation [C ] //Proceedings of IEEE Conference on Decision and Control. Washington,D.C. , US : IEEE , 2014 .

HABIBI G , KINGSTON Z , XIE W , et al. Distributed centroid estimation and motion controllers for collective transport by multi-robot systems [C ] //Proceedings of IEEE International Conference on Robotics and Automation. Washington,D.C. , US : IEEE , 2015 : 1282 - 1288 .

BANFI J , LI A Q , BASILICO N , et al. Multirobot online construction of communication maps [C ] //Proceedings of 2017 IEEE International Conference on Robotics and Automation. Washington,D.C. , US : IEEE , 2017 : 2577 - 2583 .

PALIOTTA C , BELLETER D J W , PETTERSEN K Y . Adaptive source seeking with leader-follower formation control [J ] . IFAC-PapersOnLine , 2015 , 48 ( 16 ): 85 - 290 .

SHORINWA O , YU J , HALSTED T , et al. Distributed Multi-Target tracking for autonomous vehicle fleets [C ] //Proceedings of IEEE International Conference on Robotics and Automation. Washington,D.C. , US : IEEE , 2020 : 3495 - 3501 .

HINOSTROZA M A , XU H T , SOARES C G . Cooperative operation of autonomous surface vehicles for maintaining formation in complex marine environment [J ] . Ocean Engineering , 2019 , 183 : 132 - 154 . DOI: 10.1016/j.oceaneng.2019.04.098 http://doi.org/10.1016/j.oceaneng.2019.04.098 https://linkinghub.elsevier.com/retrieve/pii/S0029801818310783 https://linkinghub.elsevier.com/retrieve/pii/S0029801818310783

XIAO X S , DUFEK J , WOODBURY T , et al. UAV assisted USV visual navigation for marine mass casualty incident response [C ] //Proceedings of 2017 IEEE/RSJ International Conference on Intelligent Robots and Systems. Washington,D.C. , US : IEEE , 2017 : 6105 - 6110 .

REYNOLDS C W . Flocks , herds and schools: a distributed behavioral model [C ] //Proceedings of the 14th Annual Conference on Computer Graphics and Interactive Techniques . 1987 : 25 - 34 . DOI: 10.1145/280811.281008 https://dx.doi.org/10.1145/280811.281008 .

COUZIN I D , KRAUSE J , JAMES R , et al. Collective memory and spatial sorting in animal groups [J ] . Journal of Theoretical Biol Ogy , 2002 , 218 ( 1 ): 1 - 11 .

VICSEK T , CZIRÓK A , BEN-JACOB E , et al. Novel type of phase transition in a system of self-driven particles [J ] . Physical Review Letters , 1995 , 75 ( 6 ): 1226 .

CUCKER F , SMALE S . Emergent behavior in flocks [J ] . IEEE Transactions on Automatic Control , 2007 , 52 ( 5 ): 852 - 862 . DOI: 10.1109/TAC.2007.895842 http://doi.org/10.1109/TAC.2007.895842 http://ieeexplore.ieee.org/document/4200853/ http://ieeexplore.ieee.org/document/4200853/

SPEARS W M , SPEARS D F , HAMANN J C , et al . Distributed, physics-based control of swarms of vehicles [J ] . Autonomous Robots , 2004 , 17 ( 2 ): 137 - 162 . DOI: 10.1023/B:AURO.0000033970.96785.f2 http://doi.org/10.1023/B:AURO.0000033970.96785.f2 http://link.springer.com/10.1023/B:AURO.0000033970.96785.f2 http://link.springer.com/10.1023/B:AURO.0000033970.96785.f2

SOYSAL O , SAHIN E . Probabilistic aggregation strategies in swarm robotic systems [C ] //Proceedings 2005 IEEE Swarm Intelligence Symposium.Washington,D.C. , US : IEEE , 2005 : 325 - 332 .

THERAULAZ G , GOSS A . Task differentiation in polistes wasp colonies: a model for self-organizing groups of robots [J ] . Proceedings Internatinal Confrens on Simulation of Adaptive Behavior , 1991 : 346 - 355 . DOI: 10.7551/mitpress/3115.003.0047 https://dx.doi.org/10.7551/mitpress/3115.003.0047 .

SUNG Y , BUDHIRAJA A K , WILLIAMS R K , et al . Distributed simultaneous action and target assignment for multi-robot multi-target tracking [C ] //Proceedings of 2018 IEEE International Conference on Robotics and Automation. Washington,D.C.,US:IEEE , 2018 : 3724 - 3729 .

VINYALS O , BABUSCHKIN I , CZARNECKI W M , et al . Grandmaster level in StarCraft II using multi-agent reinforcement learning [J ] . Nature , 2019 , 575 ( 7782 ): 350 - 354 . DOI: 10.1038/s41586-019-1724-z http://doi.org/10.1038/s41586-019-1724-z

WANG X J , SONG J X , QI P H , et al. SCC:an efficient deep reinforcement learning agent mastering the game of StarCraft II:arXiv : 2012 .13169[R ] . Ithaca,NY , US : Cornell University , 2020 :2012.13169.

LOWE R , WU Y , TAMAR A , et al . Multi-agent actor-critic for mixed cooperative competitive environments, in: advances:arXiv:1706.02275 [R ] . Ithaca,NY , US : Cornell University , 2017 :1706.02275.

FOERSTER J N , FARQUHAR G , AFOURAS T , et al . Counterfactual multi-agent policy gradients [C ] //Proceedings of the 32nd AAAI Conferenceon Artificial Intelligence. Reston, VA, US:AIAA , 2017 .

KIM D , MOON S , HOSTALLERO D , et al . Learning to schedule communication in multi-agent reinforcement learning:arXiv : 1902 .01554[R ] . Ithaca,NY , US : Cornell University , 2019 :1902.01554.

HOWARD A , PARKER LE , SUKHATME GS . Experiments with a large heterogeneous mobile robot team: exploration, mapping, deployment and detection [J ] . The International Journal of Robotics Research , 2006 , 25 ( 5/6 ): 431 - 447 . doi: 10.1177/0278364906065378 https://dx.doi.org/10.1177/0278364906065378 . http://journals.sagepub.com/doi/10.1177/0278364906065378 http://journals.sagepub.com/doi/10.1177/0278364906065378 We describe the design and experimental validation of a large heterogeneous mobile robot team built for the DARPA Software for Distributed Robotics (SDR) program. The core challenge for the SDR program was to develop a multi-robot system capable of carrying out a specific mission: to deploy a large number of robots into an unexplored building, map the building interior, detect and track intruders, and transmit all of the above information to a remote operator. To satisfy these requirements, we developed a heterogeneous robot team consisting of approximately 80 robots. We sketch the key technical elements of this team, focusing on the novel aspects, and present selected results from supervised experiments conducted in a 600 m 2 indoor environment.

李超 , 王瑞星 , 黄建忠 , 等 . 稀疏奖励下基于强化学习的无人集群自主决策与智能协同 [J ] . 兵工学报 , 2023 , 44 ( 6 ): 1537 - 1546 . DOI: 10.12382/bgxb.2022.0177 http://doi.org/10.12382/bgxb.2022.0177 无人集群将深刻地塑造战争样式,为提升无人集群自主决策算法能力,对异构无人集群攻防对抗自主决策方法进行研究。对无人集群对抗模型设计进行总体概述,并对无人集群攻防对抗场景进行模型设计;针对无人集群自主决策采用强化学习技术广泛存在的稀疏奖励问题,提出基于局部回报重塑的奖励机制设定方法;在此基础上叠加优先经验回放,有效地改善稀疏奖励问题;通过程序仿真和演示系统设计,验证该方法的优越性。该方法的研究将加速基于强化学习技术的无人集群自主决策算法网络收敛过程,对无人集群自主决策算法研究具有重要意义。

LI C , WANG R X , HUANG J Z , et al . Autonomous decision-making and intelligent collaboration of UAV swarms based on reinforcement learning with sparse rewards [J ] . Acta Armamentarii , 2023 , 44 ( 6 ): 1537 - 1546 . (in Chinese) DOI: 10.12382/bgxb.2022.0177 http://doi.org/10.12382/bgxb.2022.0177 UAV swarms will profoundly shape the pattern of warfare. In order to improve the autonomous decision-making algorithm capability of UAV swarms, the autonomous decision-making method for heterogeneous UAV swarm attack-defense confrontation scenarios is studied. An overview of the design of the UAV swarm confrontation model and the model design of the UAV swarm attack-defense confrontation scenario are carried out. To solve the sparse reward problem which widely exists in the reinforcement learning technology in the autonomous decision-making of the UAV swarm, a reward mechanism setting method based on local reward reshaping is proposed. And then, the prioritized experience replay is superimposed, which effectively improves the sparse reward problem. Finally, the superiority of this method is verified by simulation and demonstration system design. This study will accelerate the network convergence process of the autonomous decision-making algorithm for UAV swarms based on reinforcement learning technology, which is of great significance to the research on autonomous decision-making algorithms of UAV swarms.

孔国杰 , 冯时 , 于会龙 , 等 . 无人集群系统协同运动规划技术综述 [J ] . 兵工学报 , 2023 , 44 ( 1 ): 11 - 26 . DOI: 10.12382/bgxb.2022.0930 http://doi.org/10.12382/bgxb.2022.0930 地面无人集群由多个地面无人移动平台构成，能够通过各无人平台间相互协同完成统一的系统协同目标，在军事和交通系统等领域能够发挥重要的作用。协同运动规划作为无人集群系统协同的关键技术之一，近年来在理论和应用等方面的研究受到越来越多的关注。针对此研究问题，归纳总结了近年来相关领域的无人集群系统协同运动规划的研究成果，阐述了无人集群协同运动规划技术的研究背景和意义，结合国内外发展现状和研究进展，对多车协同系统的应用进行了表述，并根据主流研究方法使用的框架和算法，对现有的协同运动规划技术进行分类，并讨论了各类方法的主要特点，同时对相关代表性工作进行讨论，提出了无人集群协同规划技术面临的挑战和未来发展方向。

KONG G J , FENG S , YU H L , et al . A review on cooperative motion planning of unmanned vehicles [J ] . Acta Armamentarii , 2023 , 44 ( 1 ): 11 - 26 . (in Chinese) DOI: 10.12382/bgxb.2022.0930 http://doi.org/10.12382/bgxb.2022.0930 An unmanned ground swarm system consists of multiple unmanned ground mobile platforms, which can achieve common objectives through cooperation and has promising applications in military and transportation systems. Cooperative motion planning is one of the key technologies in the cooperation of unmanned swarm systems or vehicles. It has received increasing attention in both theoretical and application research. This review summarizes and analyzes recent advances in cooperative motion planning of unmanned swarm systems, and provides the background of relevant research. Then the techniques utilized in cooperative motion planning and its applications are briefly discussed considering its development in China and beyond. These techniques are categorized according to different frameworks and algorithms. With such a classification, representative works are discussed regarding their features. Moreover, the challenges and future development of cooperative motion planning are proposed.

HAUERT S , LEVEN S , VARGA M , et al . Reynolds flocking in reality with fixed-wing robots:communication range vs. maximum turning rate [C ] //Proceedings of the IEEE/RSJ International Conference on Intelligent Robots and Systems. Piscataway, NJ , US : IEEE , 2011 : 08855 - 1331 .

VÁSÁRHELYI G , VIRÁGH C , SOMORJAI G , et al . Optimized flocking of autonomous drones in confined environments [J ] . Science Robotics , 2018 , 3 ( 20 ): eaat3536 . DOI: 10.1126/scirobotics.aat3536 http://doi.org/10.1126/scirobotics.aat3536 https://www.science.org/doi/10.1126/scirobotics.aat3536 https://www.science.org/doi/10.1126/scirobotics.aat3536 A flocking model that uses an evolutionary optimization framework is validated with a self-organized swarm of 30 drones.

SORIA E , SCHIANO F , FLOREANO D . Predictive control of aerial swarms in cluttered environments [J ] . Nature Machine Intelligence , 2021 , 3 ( 6 ): 545 - 554 . DOI: 10.1038/s42256-021-00341-y http://doi.org/10.1038/s42256-021-00341-y

ZHOU X , WEN X Y , WANG Z P , et al . Swarm of micro flying robots in the wild [J ] . Science Robotics , 2022 , 7 ( 66 ): eabm5954 . DOI: 10.1126/scirobotics.abm5954 http://doi.org/10.1126/scirobotics.abm5954 https://www.science.org/doi/10.1126/scirobotics.abm5954 https://www.science.org/doi/10.1126/scirobotics.abm5954

谢伟 , 陶浩 , 龚俊斌 , 等 . 海上无人系统集群发展现状及关键技术研究进展 [J ] . 中国舰船研究 , 2021 , 16 ( 1 ): 7 - 17 , 31.

XIE W , TAO H , GONG J B , et al . Research advances in the development status and key technology of unmanned marine vehicle swarm operation [J ] . Chinese Journal of Ship Research , 2021 , 16 ( 1 ): 7 - 17 , 31. (in Chinese)

李易珊 . 云洲智能:无人系统领航深蓝探索 [J ] . 海洋与渔业 , 2018 ( 8 ): 68 - 69 .

LI Y S . Yunzhou intelligence:unmanned system piloted deep blue exploration [J ] . Ocean and Fishery , 2018 ( 8 ): 68 - 69 . (in Chinese)

杨翊 , 周星群 , 胡志强 , 等 . 基于视觉定位的水下机器人无通信高精度编队技术研究 [J ] . 数字海洋与水下攻防 , 2022 ( 1 ): 50 - 58 .

YANG Y , ZHOU X Q , HU Z Q , et al. Research on high-precision unmanned underwater vehicles team formation without communication based on visual positioning technology [J ] . Digital Ocean & Underwater Warfare , 2022 ( 1 ): 50 - 58 . (in Chinese)

喻俊志 , 周子烨 , 刘金存 . 多自主水下机器人系统研究进展与分析 [J ] . 信息与控制 , 2021 , 50 ( 2 ): 129 - 140 . DOI: 10.13976/j.cnki.xk.2021.0572 http://doi.org/10.13976/j.cnki.xk.2021.0572 多自主水下机器人系统（MAUV）越来越多地被应用于海洋勘探与资源开发，受到研究者们的普遍关注.本文探讨了常见的多自主水下机器人系统所执行的任务及所用方法，并从新功能涌现的角度对多自主水下机器人系统的协作这一关键问题进行了全面回顾与分析.首先，将合作按照任务空间、运动空间和量测空间及其组合进行分类；其次，从三个视角介绍了不同类别的水下多机器人系统及其技术方法；最后，展望了多自主水下机器人系统未来的研究思路.该综述有望为复杂的海洋应用场景中水下多机器人系统高效协同完成水下任务提供策略和途径.

YU J Z , ZHOU Z Y , LIU J C . Research progress and analysis on multiple autonomous underwater vehicle systems [J ] . Information and Control , 2021 , 50 ( 2 ): 129 - 140 . (in Chinese) DOI: 10.13976/j.cnki.xk.2021.0572 http://doi.org/10.13976/j.cnki.xk.2021.0572 Multiple autonomous underwater vehicles (MAUV) are increasingly applied to marine exploration and resource development. As such, they have been widely explored. In this paper, the tasks and methods of common MAUV are discussed, and the key problem of cooperation among MAUV is comprehensively investigated from the perspective of emerging functions. Firstly, the cooperation among MAUV is classified according to task-, motion-, and measurement-space aspects and their combination. Secondly, different kinds of MAUV and their realization methods are introduced from three perspectives. Lastly, the future research direction of MAUV is discussed. This survey is conducted to provide strategies and approaches for MAUV and help accomplish underwater tasks efficiently and cooperatively in complex ocean-related application scenarios.

张伟 , 王乃新 , 魏世琳 , 等 . 水下无人潜航器集群发展现状及关键技术综述 [J ] . 哈尔滨工程大学学报 , 2020 , 41 ( 2 ): 289 - 297 .

ZHANG W , WANG N X , WEI S L , et al. Overview of unmanned underwater vehicle swarm development status and key technologies [J ] . Journal of Harbin Engineering University , 2020 , 41 ( 2 ): 289 - 297 . (in Chinese)

DORIGO M , THERAULAZ G , TRIANNI V . Swarm robotics: past, present, and future [Point of View] [J ] . Proceedings of the IEEE , 2021 , 109 ( 7 ): 1152 - 1165 . DOI: 10.1109/JPROC.2021.3072740 http://doi.org/10.1109/JPROC.2021.3072740 https://ieeexplore.ieee.org/document/9460560/ https://ieeexplore.ieee.org/document/9460560/

张建东 , 王鼎涵 , 杨啟明 , 等 . 基于分层强化学习的无人机空战多维决策 [J ] . 兵工学报 , 2023 , 44 ( 6 ): 1547 - 1563 . DOI: 10.12382/bgxb.2022.0711 http://doi.org/10.12382/bgxb.2022.0711 针对无人机空战过程中面临的智能决策问题,基于分层强化学习架构建立无人机智能空战的多维决策模型。将空战自主决策由单一维度的机动决策扩展到雷达开关、主动干扰、队形转换、目标探测、目标追踪、干扰规避、武器选择等多个维度,实现空战主要环节的自主决策;为解决维度扩展后决策模型状态空间复杂度、学习效率低的问题,结合Soft Actor-Critic算法和专家经验训练和建立元策略组,并改进传统的Option-Critic算法,设计优化策略终止函数,提高策略的切换的灵活性,实现空战中多个维度决策的无缝切换。实验结果表明,该模型在无人机空战全流程的多维度决策问题中具有较好的对抗效果,能够控制智能体根据不同的战场态势灵活切换干扰、搜索、打击、规避等策略,达到提升传统算法性能和提高解决复杂决策效率的目的。

ZHANG J D , WANG D H , YANG Q M , et al . Multi-dimensional decision-Making for UAV air combat based on hierarchical reinforcement learning [J ] . Acta Armamentarii , 2023 , 44 ( 6 ): 1547 - 1563 . (in Chinese) DOI: 10.12382/bgxb.2022.0711 http://doi.org/10.12382/bgxb.2022.0711 To solve the intelligent decision-making problem in the process of UAV air combat, a multi-dimensional decision-making model for UAV intelligent air combat based on the hierarchical reinforcement learning architecture is established, allowing the autonomous decision-making of air combat to be extended from a single-dimensional maneuver decision to a multi-dimensional one including radar switch, active jamming, formation conversion, target detection, target tracking, interference avoidance, weapon selection, etc., so that autonomous decision-making in the main steps of air combat is realized. In order to solve the problems of state-space complexity and low learning efficiency of the decision-making model after the dimension expansion, a meta-strategy group is trained and established with the Soft Actor-Critic algorithm and expert experience, and the traditional Option-Critic algorithm is improved. The strategy termination function is designed and optimized to improve the flexibility of strategy switching and realize seamless multi-dimensional decision-making switching in air combat.. The experimental results show that the proposed method has good countermeasure effectiveness for the multi-dimensional decision-making during the whole process of UAV air combat, which can control the agent to flexibly switch among interference, search, strike, and avoidance strategies according to different battlefield situations with the purpose of improving the performance of traditional algorithms and the efficiency of solving complex decision-making processes.