DRL-based Active Disturbance Rejection Control Strategy for Integrated Guidance and Control

WANG Wenwen; WU Mingyu; GAO Zhanpeng; HUANG Zhengui

doi:10.12382/bgxb.2025.0452

您当前的位置：

首页 >

文章列表页 >

DRL-based Active Disturbance Rejection Control Strategy for Integrated Guidance and Control

更新时间：2026-04-24

- DRL-based Active Disturbance Rejection Control Strategy for Integrated Guidance and Control
- Acta Armamentarii Vol. 47, Issue 3, Pages: 250452(2026)
- 作者机构：
  
  南京理工大学瞬态物理全国重点实验室，江苏南京 210094
  南京理工大学能源与动力工程学院，江苏南京 210094
- 作者简介：
- 基金信息：
- DOI：10.12382/bgxb.2025.0452
  CLC： TJ765
- Received：04 June 2025，
  
  Online First：04 April 2026，
  
  Published：2026-03
- 稿件说明：
移动端阅览
WANG Wenwen, WU Mingyu, GAO Zhanpeng, et al. DRL-based Active Disturbance Rejection Control Strategy for Integrated Guidance and Control[J]. Acta Armamentarii, 2026, 47(3): 250452.
DOI：

WANG Wenwen, WU Mingyu, GAO Zhanpeng, et al. DRL-based Active Disturbance Rejection Control Strategy for Integrated Guidance and Control[J]. Acta Armamentarii, 2026, 47(3): 250452. DOI： 10.12382/bgxb.2025.0452.

摘要

制导控制一体化系统具有强非线性、时变及多不确定性等特点，传统自抗扰控制方法虽能有效估计并补偿系统不确定性，但其参数整定复杂，且固定参数难以适应动态战场环境。为此，提出一种基于深度强化学习的自抗扰控制智能参数整定方法。基于制导控制一体化模型构建了深度强化学习环境，采用泄漏近端策略优化算法训练智能体，实现参数的动态优化。仿真结果表明：通过融合深度强化学习算法和传统自抗扰控制策略，有效提升了导弹制导精度，降低了自抗扰控制参数整定复杂度，并验证了所提方法在保留传统自抗扰控制优势的同时，通过智能调参显著提升了控制器的环境适应性，为高动态系统的控制优化提供了创新性解决方案。

Abstract

The integrated guidance and control（IGC）system exhibits strong nonlinearity

time-varying dynamics

and multiple uncertainties. Although the traditional active disturbance rejection control（ADRC）method can effectively estimate and compensate for system uncertainties

its parameter tuning is complex

and the fixed parameters are difficult to adapt to the dynamic battlefield environments. To address this

an intelligent parameter tuning method for ADRC based on deep reinforcement learning（DRL）is proposed. A DRL environment is constructed based on the IGC model

and the leaky proximal policy optimization algorithm is employed to train the agent for dynamic parameter optimization. Simulated results demonstrate that the missile guidance accuracy is significantly improved and the complexity of ADRC parameter tuning is reduced by integrating DRL algorithm with traditional ADRC strategy. The proposed method retains the advantages of conventional ADRC while enhancing the environmental adaptability of controller through intelligent parameter tuning. This research provides an innovative solution for control optimization in highly dynamic systems.

关键词

Keywords

references

赵斌,梁乐成,蒋瑞民,等.终端角度约束制导及制导控制一体化方法综述[J].宇航学报,2022,43(5):563-579.

ZHAO B,LIANG L C,JIANG R M,et al. Review of guidance and control methods under terminal angle constraints[J]. Journal of Astronautics,2022,43(5):563-579. (in Chinese)

郭建国,梁乐成,周敏,等.高速飞行器俯冲段制导控制一体化综述[J].航空兵器,2023,30(1):1-10.

GUO J G, LIANG L C, ZHOU M, et al. Overview of integrated guidance and control for hypersonic vehicles in dive phase[J]. Aero Weaponry,2023,30(1):1-10. (in Chinese)

程林,蒋方华,李俊峰.深度学习在飞行器动力学与控制中的应用研究综述[J].力学与实践,2020,42(3):267-276.

CHENG L,JIANG F H,LI J F. A review on the applications of deep learning in aircraft dynamics and control[J]. Mechanics in Engineering,2020,42(3):267-276. (in Chinese)

MING C, WANG X M,SUN R S. A novel non-singular terminal sliding mode control-based integrated missile guidance and control with impact angle constraint[J]. Aerospace Science and Technology,2019,94:105368.

吴炎烜,陆胥坛,王正杰.基于滑模控制的飞行器螺旋机动、制导与控制一体化设计研究[J].北京理工大学学报,2022, 42(5):523-529.

WU Y X,LU X T,WANG Z J. Research on integrated design of aircraft spiral maneuver, guidance and control based on sliding mode control[J]. Transactions of Beijing Institute of Technology, 2022,42(5):523-529. (in Chinese)

KHANKALANTARY S, SHEIKHOLESLAM F. Robust extended state observer-based three dimensional integrated guidance and control design for interceptors with impact angle and input saturation constraints[J]. ISA Transactions,2020,104:299-309.

ZHANG X Y, ZHUO X B, ZHANG Q Z. Enhanced fruit fly optimization algorithm based backstepping-HOSMC for integrated guidance and control of hypersonic gliding vehicle[J]. IEEE Transactions on Aerospace and Electronic Systems,2023,59(6):9342-9356.

安通,王鹏,王建华,等.弹性高超声速飞行器动态面制导控制一体化设计方法[J].系统工程与电子技术,2022,44(3):956-966.

AN T,WANG P,WANG J H,et al. Integrated guidance and control schemes for dynamic surface of flexible hypersonic vehicles[J]. Systems Engineering and Electronics,2022,44(3):956-966. (in Chinese)

张宽桥,刘连照,马晖,等.考虑攻击角约束和输入饱和的制导控制一体化设计[J].弹道学报,2021,33(3):9-18.

ZHANG K Q,LIU L Z,MA H,et al. Design on integrated guidance and control considering the constraint of impact angle and input saturation[J]. Journal of Ballistics, 2021, 33 (3): 9-18. (in Chinese)

张宽桥,周旋风,门星火,等.固定时间收敛的三维制导控制一体化设计[J].北京航空航天大学学报,2023,49 (4):842-852.

ZHANG K Q, ZHOU X F, MEN X H, et al. Three-dimensional integrated guidance and control design with fixed-time convergence[J]. Journal of Beijing University of Aeronautics and Astronautics, 2023,49(4):842-852. (in Chinese)

BAO C Y,WANG P,TANG G J. Integrated method of guidance, control and morphing for hypersonic morphing vehicle in glide phase[J]. Chinese Journal of Aeronautics,2021,34(5):535-553.

何昊,王鹏.高速变形飞行器制导控制一体化设计方法[J].航空学报,2024,45(增刊1):299-312.

HE H, WANG P. Integrated guidance and control method for high-speed morphing wing aircraft[J]. Acta Aeronautica et Astronautica Sinica,2024,45(S1):299-312. (in Chinese)

胡诚伟,韦悠,汪学平.考虑落角约束和气动不确定性的固定时间制导控制一体化设计方法[J].弹箭与制导学报, 2023,43(4):98-104.

HU C W,WEI Y,WANG X P. Fixed-time integrated guidance and control for impact angle constrained interception with multiple uncertainties[J]. Journal of Projectiles,Rockets,Missiles and Guidance,2023,43(4):98-104. (in Chinese)

韩京清.自抗扰控制技术：估计补偿不确定因素的控制技术[M].北京：国防工业出版社,2013.

HAN J Q. Active disturbance rejection control technique: The technique for estimating and compensating the uncertainties[M]. Beijing:National Defense Industry Press,2013. (in Chinese)

GUO B Z, WU Z H. Output tracking for a class of nonlinear systems with mismatched uncertainties by active disturbance rejection control[J]. Systems & Control Letters,2017,100:21-31.

ZHUANG H X,SUN Q L,CHEN Z Q,et al. Active disturbance rejection control for attitude control of missile systems based on back-stepping method[J]. International Journal of Control, Automation,and Systems,2021,19(11):3642-3656.

WANG R,ZHOU Z,ZHU X P,et al. Responses and suppression of joined-wing UAV in wind field based on distributed model and active disturbance rejection control[J]. Aerospace Science and Technology,2021,115:106803.

YU X,ZHU Y K,QIAO J Z,et al. Antidisturbance controllability analysis and enhanced antidisturbance controller design with application to flexible spacecraft[J]. IEEE Transactions on Aerospace and Electronic Systems,2021,57(5):3393-3404.

赵坤,曹登庆,黄文虎.突防导弹机动、制导与控制一体化设计[J].系统工程与电子技术,2018,40(9):2040-2047.

ZHAO K, CAO D Q, HUANG W H. Integrated design of maneuver, guidance and control for penetration missile[J]. Systems Engineering and Electronics,2018,40(9):2040-2047. (in Chinese)

ZHENG Q,GAO L D Q, GAO Z Q. On validation of extended state observer through analysis and experimentation[J]. Journal of Dynamic Systems, Measurement, and Control, 2012, 134 (2):024505.

GAO Z Q. Scaling and bandwidth-parameterization based controller tuning[C]∥Proceedings of the 2003 American Control Conference. Denver,CO,US:IEEE,2003:4989-4996.

杨婷婷,李爱军,侯震.基于粒子群算法的自抗扰飞行控制器优化设计[J].计算机仿真,2009,26(9):59-61,69.

YANG T, LI A J, HOU Z. An ADRC flight controller design method based on PSO algorithm[J]. Computer Simulation,2009, 26(9):59-61,69. (in Chinese)

胡丹丹,张宇辰.基于改进粒子群算法的四旋翼自抗扰控制器优化设计[J].计算机应用研究,2019,36(6):1762-1766.

HU D,ZHANG Y C. ADRC controller optimization design based on improved PSO algorithm for quad-rotor[J]. Application Research of Computers,2019,36(6):1762-1766. (in Chinese)

LIU C F, ZANG B. Application and the parameter tuning of ADRC based on CPSO[C]∥Proceedings of the 201224th Chinese Control and Decision Conference. Taiyuan,China:IEEE, 2012,3277-3281.

付京博,邵会兵,詹韬.基于深度强化学习的飞行器自抗扰控制技术[J].计算机仿真,2022,39(10):54-59.

FU J B, SHAO H B, ZHAN T. Aircraft active disturbance rejection control technology based on deep reinforcement learning[J].Computer Simulation,2022,39(10):54-59. (in Chinese)

安柯欣,陈光武,石建强.基于强化学习的高速列车速度跟踪改进自抗扰控制[J].北京交通大学学报,2023,47(2):86-94.

AN K X, CHEN G W, SHI J Q. Improved active disturbance rejection control of high-speed train speed tracking based on reinforcement learning[J]. Journal of Beijing Jiaotong University,2023,47(2):86-94. (in Chinese)

周雪松,韩静,马幼捷,等.基于DQN算法的直流微电网负载接口变换器自抗扰控制策略[J].电力系统保护与控制, 2025,53(1):95-103.

ZHOU X S,HAN J,MA Y J,et al. Active disturbance rejection control strategy of a DC microgrid load interface converter based on DQN algorithm[J]. Power System Protection and Control, 2025,53(1):95-103. (in Chinese)

裴培,何绍溟,王江,等.一种深度强化学习制导控制一体化算法[J].宇航学报,2021,42(10):1293-1304.

PEI P,HE S M,WANG J,et al. Integrated guidance and control for missile using deep reinforcement learning[J]. Journal of Astronautics,2021,42(10):1293-1304. (in Chinese)

GAUDET B,FURFARO R,LINARES R. Reinforcement learning for angle-only intercept guidance of maneuvering targets[J]. Aerospace Science and Technology,2020,99:105746.

ZHANG J R, ZHANG K P, ZHANG Y, et al. Near-optimal interception strategy for orbital pursuit-evasion using deep reinforcement learning[J]. Acta Astronautica,2022,198:9-25.

WANG N Y,WANG X G,CUI N G,et al. Deep reinforcement learning-based impact time control guidance law with constraints on the field-of-view[J]. Aerospace Science and Technology, 2022,128:107765.

WANG Z, HAO Y T. Reinforcement learning adaptive risk-sensitive fault-tolerant IGC method for a class of STT missile[J]. Nonlinear Dynamics,2024,112:18195-18218.

YANG Z Q,LIU X D,LIU H K. Impact time control guidance law with time-varying velocity based on deep reinforcement learning[J]. Aerospace Science and Technology,2023,142:108603.

先苏杰,王康,曾鑫,等.基于深度强化学习的落角和视场角约束制导律[J].兵工学报,2025,46(4):257-271.

XIAN S J,WANG K,ZENG X,et al. An impact angle and field of view constraints guidance law based on deep reinforcement learning[J]. Acta Armamentarii,2025,46(4):257-271. (in Chinese)

王文文,刘晓利,李志坚.基于自抗扰控制的微型导弹导引控制一体化设计[J].指挥控制与仿真,2015,37(3):121-125.

WANG W W,LIU X L,LI Z J. Integrated guidance and control of micro missile based on active disturbance rejection control[J]. Command Control & Simulation, 2015, 37(3): 121-125. (in Chinese)

HAN X C,AFIFI H,MOUNGLA H,et al. Leaky PPO:a simple and efficient RL algorithm for autonomous vehicles[C]∥Proceedings of 2024 International Joint Conference on Neural Networks. Yokohama,Japan:IEEE,2024:1-7.

ZHU Y Y,PAN M X,ZHOU W X,et al. Intelligent direct thrust control for multivariable turbofan engine based on reinforcement and deep learning methods[J]. Aerospace Science and Technology,2022,131:107972.

Views

下载量

CNKI被引量

Alert me when the article has been cited

提交

Tools

Publicity Resources

Safe-DRL: A Safety-conscious Deep Reinforcement Learning Decision-making Algorithm for Unmanned Platforms

Motion Control of Quadruped Robot Based on Deep Reinforcement Learning and Nonlinear Model Predictive Control Technique

Multi-UAV Collaborative Dynamic Search Decision-making Method for Unknown Moving Targets

Energy Management Strategy Optimized by Munchausen-PER-DDQN for Hybrid Tracked Vehicle

Related Author

WU Mingyu

YI Wenjun

YANG Fan

LI Xueyuan

DU Minggang

JIANG Yutong

LIU Qi

CHEN Yaowei

Related Institution

School of Mechanical Engineering, Beijing Institute of Technology

Chinese Scholartree Ridge SKL, China Northern Vehicle Research Institute

School of Automation, Nanjing University of Science and Technology

School of Astronautics, Northwestern Polytechnical University

Sichuan Jiuzhou Electric Appliance Group Co. , Ltd.

Postal code：100089
Tel：010-68963060/68962718 Fax：010-68963025 Email：bgxb@cos.org.cn
Technical support is provided by Beijing Founder electronics co., LTD 京ICP备05059581号-4 京公网安备11010802024360号
It is recommended to read the content of this site in Chrome&IE9+. Please switch to extreme mode in browser 360.
Cookies We use cookies to help provide and enhance our service and tailor content. By continuing, you agree to the use of cookies.
Total Visits：119050 Visits Today：37

⁰