欢迎访问《兵工学报》官方网站,今天是 分享到:

兵工学报 ›› 2021, Vol. 42 ›› Issue (3): 663-672.doi: 10.3969/j.issn.1000-1093.2021.03.024

• 论文 • 上一篇    

基于多组并行深度Q网络的连续空间追逃博弈算法

刘冰雁1,2, 叶雄兵1, 岳智宏2, 董献洲1, 张其扬2   

  1. (1.军事科学院, 北京 100091; 2.32032部队, 北京 100094)
  • 上线日期:2021-04-26
  • 通讯作者: 叶雄兵(1969—),男,研究员,博士生导师 E-mail:yexb1123@sina.com
  • 作者简介:刘冰雁(1988—),男,助理研究员,博士研究生。E-mail:bingyanl@outlook.com

Continuous Space Pursuit-evasion Game Algorithm Based on Multi-group Deep Q Network

LIU Bingyan1,2, YE Xiongbing1, YUE Zhihong2, DONG Xianzhou1, ZHANG Qiyang2   

  1. (1.Academy of Military Sciences, Beijing 100091, China; 2.Unit 32032 of PLA, Beijing 100094, China)
  • Online:2021-04-26

摘要: 为解决连续空间追逃博弈(PEG)问题,提出一种基于多组并行深度Q网络(DQN)的连续空间PEG算法。应对连续行为空间中为避免传统强化学习存在的维数灾难不足,通过构建Takagi-Sugeno-Kang模糊推理模型来表征连续空间;为应对离散动作集自学习复杂且耗时不足,设计基于多组并行DQN的PEG算法。以4轮战车PEG问题为例设计仿真环境与运动模型,进行了运动计算,并与Q-learning算法、基于资格迹的强化学习算法、基于奖励的遗传算法结果相比对。仿真实验结果表明,连续空间PEG算法能够较好地解决连续空间PEG问题,且随着学习次数的增加不断提升问题处理能力,具备自主学习耗时少、追捕应用时间短的比较优势。

关键词: 追逃博弈, 连续空间, 深度Q网络, 神经网络, 微分对策, 智能战车

Abstract: A continuous space pursuit-evasion game algorithm based on multi-group deep reinforcement learning is proposed to solve the problems in continuous space pursuit-evasion game(PEG). In order to avoid the insufficient curse of dimensionality of continuous space in traditional reinforcement learning,a TSK fuzzy inference model is established to represent the continuous space.And a pursuit-evasion game algorithm based on multi-group deep reinforcement learning is designed for the complex and time-consuming problems of discrete action self-learning.The simulation environment and motion model were designed by taking the PEG problem of a four-wheel vehicle as an example, and the simulation experiments were carried out with Q-learning algorithm, reinforcement learning algorithm based on qualification trace and genetic algorithm based on reward, respectively. The simulated results show that the continuous space PEG algorithm can be used to solve the problem of continuous space pursuit-evasion game well,and continuously improve the ability to address problems with the increase in learning times,and has the comparative advantages of less time consuming for independent learning and short application time.

Key words: pursuit-evasiongame, continuousspace, deepQnetwork, neuralnetwork, differentialgame, intelligentvehicle

中图分类号: