欢迎访问《兵工学报》官方网站,今天是 2025年5月29日 星期四

兵工学报 ›› 2024, Vol. 45 ›› Issue (10): 3385-3396.doi: 10.12382/bgxb.2023.0862

• • 上一篇    下一篇

基于演员-评论家框架的层次化多智能体协同决策方法

傅妍芳1, 雷凯麟1,*(), 魏佳宁2, 曹子建1, 杨博1, 王炜3, 孙泽龙4, 李秦洁1   

  1. 1 西安工业大学 计算机科学与工程学院, 陕西 西安 710021
    2 北京机电工程研究所, 北京 100083
    3 95810部队, 北京 100076
    4 西安工业大学 兵器科学与技术学院, 陕西 西安 710021

A Hierarchical Multi-Agent Collaborative Decision-making Method Based on the Actor-critic Framework

FU Yanfang1, LEI Kailin1,*(), WEI Jianing2, CAO Zijian1, YANG Bo1, WANG Wei3, SUN Zelong4, LI Qinjie1   

  1. 1 School of Computer Science and Engineering, Xi’an Technological University, Xi’an 710021, Shaanxi, China
    2 Beijing Electro-Mechanical Engineering Institute, Beijing 100083, China
    3 Unit 95810 of PLA, Beijing 100076, China
    4 School of Armament Science and Technology, Xi’an Technological University, Xi’an 710021, Shaanxi, China
  • Received:2023-09-05 Online:2024-10-28

摘要:

针对复杂作战环境下多智能体协同决策中出现的任务分配不合理、决策一致性较差等问题,提出一种基于演员-评论家(Actor-Critic,AC)框架的层次化多智能体协同决策方法。通过将决策过程分为不同层次,并使用AC框架来实现智能体之间的信息交流和决策协同,以提高决策效率和战斗力。在高层次,顶层智能体制定任务决策,将总任务分解并分配给底层智能体。在低层次,底层智能体根据子任务进行动作决策,并将结果反馈给高层次。实验结果表明,所提方法在多种作战仿真场景下均取得了较好的性能,展现了其在提升军事作战协同决策能力方面的潜力。

关键词: 深度强化学习, 层次化多智能体, 信息共享, 智能兵棋推演

Abstract:

A hierarchical multi-agent collaborative decision-making method based on the actor-critic (AC) frameworkis proposed to address the issues of improper task allocation and weak decision consistency in the collaborative decision-making of multiple agents in complex operational environments. The proposed method divides the decision-making process into different levels and utilizes the AC framework to facilitate information exchange and decision coordination among the agents, thereby enhancing thedecision efficiency and combat effectiveness. At the higher level, the top-level agents formulate thetask decisions by decomposing and assigning overall tasks to the lower-level agents. At the lower level, the lower-level agents make action decisions based on subtasks and provide feedback to the higher level. Experimental results demonstrate that the proposed method performs well in various operational simulation scenarios, showcasing its potential to enhance themilitary operational collaborative decision-making capability.

Key words: deep reinforcement learning, hierarchical multi-agent, information sharing, intelligent war-gaming simulation

中图分类号: