Acta Armamentarii ›› 2023, Vol. 44 ›› Issue (6): 1537-1546.doi: 10.12382/bgxb.2022.0177
Previous Articles Next Articles
LI Chao1,2, WANG Ruixing1,*(), HUANG Jianzhong1, JIANG Feilong3, WEI Xuemei1, SUN Yanxin1
Received:
2022-03-21
Online:
2023-06-30
Contact:
WANG Ruixing
LI Chao, WANG Ruixing, HUANG Jianzhong, JIANG Feilong, WEI Xuemei, SUN Yanxin. Autonomous Decision-making and Intelligent Collaboration of UAV Swarms Based on Reinforcement Learning with Sparse Rewards[J]. Acta Armamentarii, 2023, 44(6): 1537-1546.
Fig.8 Framework for autonomous decision-making and intelligent collaboration strategy learning method for UAV swarm confrontation based on local reward reshaping and prioritized experience replay
算法 | 算法效果 | 性能提升 |
---|---|---|
DQN +局部回报重塑算法 | 训练2000代,策略收敛,胜率约80%。 | |
Double DQN+局部回报重塑算法 | 训练1500代,策略收敛,胜率约80%。 | 提升25% |
Double DQN+局部回报重塑+PER算法 | 训练700代,策略收敛,胜率约80%。 | 提升65% |
Table 1 Efficiency comparison forattack-defense confrontation algorithms of UAV swarms
算法 | 算法效果 | 性能提升 |
---|---|---|
DQN +局部回报重塑算法 | 训练2000代,策略收敛,胜率约80%。 | |
Double DQN+局部回报重塑算法 | 训练1500代,策略收敛,胜率约80%。 | 提升25% |
Double DQN+局部回报重塑+PER算法 | 训练700代,策略收敛,胜率约80%。 | 提升65% |
[1] |
王莉. 人工智能在军事领域的渗透与应用思考[J]. 科技导报, 2017, 35(15):15-19.
|
|
|
[2] |
罗德林, 徐扬, 张金鹏. 无人机集群对抗技术新进展[J]. 科技导报, 2017, 35(7): 26-31.
|
|
|
[3] |
梁晓龙, 侯岳奇, 胡利平, 等. 无人集群试验评估研究现状分析及理论方法[J]. 南京航空航天大学学报, 2020, 52(6): 846-854.
|
|
|
[4] |
朱建文, 赵长见, 李小平, 等. 基于强化学习的集群多目标分配与智能决策方法[J]. 兵工学报, 2021, 42(9): 2040-2048.
|
doi: 10.3969/j.issn.1000-1093.2021.09.025 |
|
[5] |
杜威, 丁世飞. 多智能体强化学习综述[J]. 计算机科学, 2019, 46(8):1-8.
doi: 10.11896/j.issn.1002-137X.2019.08.001 |
doi: 10.11896/j.issn.1002-137X.2019.08.001 |
|
[6] |
郭宪, 方勇纯. 深入浅出强化学习[M]. 北京: 电子工业出版社, 2018:1-10.
|
|
|
[7] |
陈智超. 基于深度强化学习的无人潜航器智能对抗决策[D]. 哈尔滨: 哈尔滨工业大学, 2020.
|
|
|
[8] |
doi: 10.1109/TNSRE.2017.2700395 pmid: 28475063 |
[9] |
|
[10] |
|
[11] |
|
[12] |
|
[13] |
杨瑞, 严江鹏, 李秀. 强化学习稀疏奖励算法研究——理论与实验[J]. 智能系统学报, 2020, 15(5): 888-899.
|
|
|
[14] |
方嘉良. 基于强化学习的稀疏奖励问题研究[D]. 北京: 中国地质大学, 2020:29-39.
|
|
|
[15] |
杨惟轶, 白辰甲, 蔡超, 等. 深度强化学习中稀疏奖励问题研究综述[J]. 计算机科学, 2020, 47(3):182-191.
doi: 10.11896/jsjkx.190200352 |
|
|
[16] |
王瑞星. 含有稀疏奖励的异构多智能体强化学习对抗方法研究[D]. 哈尔滨: 哈尔滨工业大学, 2021.
|
|
|
[17] |
王瑞星, 董诗音, 江飞龙, 等. 稀疏奖励下基于强化学习的异构多智能体对抗[J]. 信息技术, 2021(5):12-20.
|
|
|
[18] |
李理, 李旭光, 郭凯杰, 等. 国产化环境下基于强化学习的地空协同作战仿真[J]. 兵工学报, 2022, 43(增刊1): 74-81.
|
doi: 10.12382/bgxb.2022.A005 |
|
[19] |
|
[20] |
doi: 10.1109/TITS.2020.3042670 URL |
[21] |
|
[22] |
|
[23] |
|
[24] |
|
[1] | CAO Zijian, SUN Zelong, YAN Guochuang, FU Yanfang, YANG Bo, LI Qinjie, LEI Kailin, GAO Linghang. Simulation of Reinforcement Learning-based UAV Swarm Adversarial Strategy Deduction [J]. Acta Armamentarii, 2023, 44(S2): 126-134. |
[2] | LI Song, MA Zhuangzhuang, ZHANG Yunlin, SHAO Jinliang. Multi-agent Coverage Path Planning Based on Security Reinforcement Learning [J]. Acta Armamentarii, 2023, 44(S2): 101-113. |
[3] | YANG Jiaxiu, LI Xinkai, ZHANG Hongli, WANG Hao. Robust Tracking of Quadrotor UAVs Based on Integral Reinforcement Learning [J]. Acta Armamentarii, 2023, 44(9): 2802-2813. |
[4] | ZHANG Jiandong, WANG Dinghan, YANG Qiming, SHI Guoqing, LU Yi, ZHANG Yaozhong. Multi-Dimensional Decision-Making for UAV Air Combat Based on Hierarchical Reinforcement Learning [J]. Acta Armamentarii, 2023, 44(6): 1547-1563. |
[5] | ZHENG Zexin, LI Wei, ZOU Kun, LI Yanfu. Anti-jamming Waveform Design of Ground-based Air Surveillance Radar Based on Reinforcement Learning [J]. Acta Armamentarii, 2023, 44(5): 1422-1430. |
[6] | LI Jiajian, SHI Yanjun, YANG Yu, LI Bo, ZHAO Xijun. Multi-agent Reinforcement Learning-based Offloading Decision for UAV Cluster Combat Tasks [J]. Acta Armamentarii, 2023, 44(11): 3295-3309. |
[7] | JIANG Yan, DING Yuyan, ZHANG Xinglong, XU Xin. A Human-machine Collaborative Control Algorithm for Intelligent Vehicles Based on Model Prediction and Policy Learning [J]. Acta Armamentarii, 2023, 44(11): 3465-3477. |
[8] | ZHAO Wenfei, CHEN Jian, WANG Yan, TENG Kenan. Dynamic Firepower Allocation for Cooperative Air Defense of Strategic Locations on the Sea Based on Reinforcement Learning [J]. Acta Armamentarii, 2023, 44(11): 3516-3528. |
Viewed | ||||||||||||||||||||||||||||||||||||||||||||||||||
Full text 1186
|
|
|||||||||||||||||||||||||||||||||||||||||||||||||
Abstract 798
|
|
|||||||||||||||||||||||||||||||||||||||||||||||||