Cooperative Reentry Guidance for Intelligent Lateral Maneuver of Hypersonic Vehicle Based on Downrange Analytical Solution

doi:10.3969/j.issn.1000-1093.2021.07.007

Abstract

Abstract: A cooperative guidance law based on deep Q-learning network (DQN) algorithm for lateral maneuver of hypersonic vehicles is proposed to meet the requirement of cooperative saturation attack. The longitudinal and lateral guidance laws are deigned. The longitudinal lift-to-drag ratio is calculated to obtain the modulus of bank angle based on the high-precision longitudinal analytical solution. The lateral bank reversal logic is abstracted as a Markov decision process (MDP), and the reinforcement learning method can be used. A lateral intelligent maneuver decision-making device based on DQN algorithm is designed. The intelligent agent can be generated by offline training according to the mission requirements, and is called online to generate the sign of bank angle. Simulated results show that the proposed guidance law can autonomously generate bank angle reversals on-line, strictly meets the requirements of flight time constraints and energy management, and has high guidance accuracy and good mission adaptability. Compared with the coordinative reentry guidance law based on three-dimensional analytical solutions, the proposed guidance law is used to fully achieve the lateral maneuverability of vehicle and make it have higher penetration potential.

Key words: hypersonicvehicle, cooperativereentryguidance, downrangeanalyticalsolution, deepreinforcementlearning, deepQ-learningnetwork

CLC Number:

V448.235

ZHANG Wanqing, YU Wenbin, LI Jinglin, CHEN Wanchun. Cooperative Reentry Guidance for Intelligent Lateral Maneuver of Hypersonic Vehicle Based on Downrange Analytical Solution[J]. Acta Armamentarii, 2021, 42(7): 1400-1411.

References

［1］解发瑜, 李刚, 徐忠昌. 高超声速飞行器概念及发展动态［J］. 飞航导弹, 2004(5):27-31,54.
XIE F Y, LI G, XU Z C. Concept and development trend of hypersonic vehicles［J］. Aerodynamic Missile Journal, 2004(5):27- 31,54. (in Chinese)
［2］蔺君, 何英姿, 黄盘兴. 带推力高超声速飞行器非连续点火再入轨迹研究［J］. 兵工学报, 2020, 41(7): 1307-1316.
LIN J, HE Y Z, HUANG P X. Research on reentry trajectory of powered hypersonic vehicle with discontinuous ignition［J］. Acta Armamentarii, 2020, 41(7): 1307-1316. (in Chinese)
［3］李文, 尚腾, 姚寅伟, 等. 速度时变情况下多飞行器时间协同制导方法研究［J］. 兵工学报, 2020, 41(6): 1096-1110.
LI W, SHANG T, YAO Y W, et al. Research on time-cooperative guidance of multiple flight vehicles with time-varying velocity［J］. Acta Armamentarii, 2020, 41(6): 1096-1110. (in Chinese)
［4］ HARPOLD J C, GRAVES C A. Shuttle entry guidance［J］. Journal of the Astronautical Sciences, 1979, 26(3): 239-268.
［5］ ROENNEKE A J, MARKL A. Re-entry control to a drag-vs-energy profile［J］. Journal of Guidance Control & Dynamics, 1994, 17(5):916-920.
［6］ ZHANG W Q, CHEN W C, YU W B, et al. Autonomous entry guidance based on 3-D gliding trajectory analytical solution［C］∥Proceedings of 2019 IEEE 10th International Conference on Mechanical and Aerospace Engineering. Brussels, Belgium: IEEE, 2019:24-31.
［7］ ZHANG W Q, CHEN W C, YU W B. Entry guidance for high-L/D hypersonic vehicle based on drag-vs-energy profile［J］. ISA Transactions, 2018, 83:176-188.
［8］ BRAUN R D, POWELL R W. Predictor-corrector guidance algorithm for use in high-energy aerobraking system studies［J］. Journal of Guidance, Control, and Dynamics, 1992, 15(3): 672-678.
［9］ JOSHI A, SIVAN K, AMMA S S. Predictor-corrector reentry guidance algorithm with path constraints for atmospheric entry vehicles［J］. Journal of Guidance, Control, and Dynamics, 2007, 30(5):1307-1318.
［10］ LIU S Y, LIANG Z X, LI Q D, et al. Predictor-corrector gui-
dance for entry with terminal altitude constraint［C］∥Proceedings
of 2016 35th Chinese Control Conference. Chengdu, China: IEEE, 2016: 5557-5562.
［11］ CHU H Y, LI J, DONG Y, et al. Improved MPSP method-based cooperative re-entry guidance for hypersonic gliding vehicles［J］. MATEC Web of Conferences, 2017, 114:01002.
［12］方科, 张庆振, 倪昆, 等. 高超声速飞行器时间协同再入制导［J］. 航空学报, 2018, 39(5):197-212.
FANG K, ZHANG Q K, NI K, et al. Time-coordinated reentry guidance law for hypersonic vehicle［J］. Acta Aeronautica et Astronautica Sinica, 2018, 39(5):197-212. (in Chinese)
［13］ YU W B, CHEN W C, JIANG Z G, et al. Analytical entry gui-
dance for coordinated flight with multiple no-fly-zone constraints
［J］. Aerospace Science and Technology, 2018, 84:273-290.
［14］ LI Z H, HE B, WANG M H, et al. Time-coordination entry guidance for multi-hypersonic vehicles［J］. Aerospace Science and Technology, 2019, 89:123-135.
［15］王肖, 郭杰, 唐胜景, 等. 基于解析剖面的时间协同再入制导［J］. 航空学报, 2019, 40(3): 322565.
WANG X, GUO J, TANG S J, et al. Time-cooperative entry guidance based on analytical profile［J］. Acta Aeronautica et Astronautica Sinica, 2019, 40(3): 322565. (in Chinese)
［16］ YU J L, DONG X W, LI Q D, et al. Cooperative guidance stra- tegy for multiple hypersonic gliding vehicles system［J］. Chinese Journal of Aeronautics, 2020, 33(3):990-1005.
［17］周锐, 陈宗基. 强化学习在导弹制导中的应用［J］. 控制理论与应用, 2001, 18(5): 748-750.
ZHOU R, CHEN Z J. Application of reinforcement learning in missile guidance［J］. Control Theory and Applications, 2001, 18(5): 748-750. (in Chinese)
［18］余跃, 王宏伦. 基于深度学习的高超声速飞行器再入预测校正容错制导［J］. 兵工学报, 2020, 41(4): 656-669.
YU Y, WANG H L. Deep learning-based reentry predictor-corrector fault-tolerant guidance for hypersonic vehicles［J］. Acta Armamentarii, 2020, 41(4): 656-669. (in Chinese)
［19］高嘉时. 升力式再入飞行器轨迹优化与制导方法研究［D］. 武汉:华中科技大学, 2019.
GAO J S. Research on reentry trajectory optimization and guidance method for lifting vehicle［D］. Wuhan: Huazhong University of Science and Technology, 2019. (in Chinese)
［20］方科, 张庆振, 倪昆, 等. 飞行时间约束下的再入制导律［J］. 哈尔滨工业大学学报, 2019, 51(10):90-97.
FANG K, ZHANG Q Z, NI K, et al. Reentry guidance law with flight time constraint［J］. Journal of Harbin Institute of Technology, 2019, 51(10): 90-97. (in Chinese)
［21］李天任, 杨奔, 汪韧, 等. 基于Q-Learning算法的再入飞行器制导方法［J］. 战术导弹技术, 2019(5):44-49.
LI T R, YANG B, WANG R, et al. Reentry vehicle guidance method based on Q-learning algorithm［J］. Tactical Missile Technology, 2019(5):44-49. (in Chinese)
［22］ SUTTON E S, BARTO A G. Reinforcement learning: an introduction［M］. Cambridge, MA, US: MIT Press, 2005.
［23］郭宪, 方勇纯.深入浅出强化学习: 原理入门［M］.北京:电子工业出版社,2018.
GUO X, FANG Y C. Reinforcement learning: introduction to the principles ［M］. Beijing: Publishing House of Electronics Industry, 2018. (in Chinese)
［24］ MNIH V, KAVUKCUOGLU K, SILVER D, et al. Human-level control through deep reinforcement learning［J］. Nature, 2015, 518(7540):529-533.
［25］周慧钟, 李忠应, 王瑾. 有翼导弹飞行动力学［M］. 北京:北京航空航天大学出版社, 1993.
ZHOU H Z, LI Z Y, WANG J. Flight dynamics of winged missile［M］. Beijing: Beihang University Press, 1993. (in Chinese)
［26］ SONG S H, HA I J. A Lyapunov-like approach to performance analysis of 3-dimensional pure PNG laws［J］. IEEE Transactions on Aerospace & Electronic Systems, 2002, 30(1):238-248.
［27］ PHILLIPS T H. A common aero vehicle (CAV) model, description, and employment guide［R］. Arlington, VA, US: Schafer Corporation for AFRL and AFSPC, 2003.

[1]	MA Ye， FAN Wenhui， CHANG Tianqing. Optimization Method of Unmanned Swarm Defensive Combat Scheme Based on Intelligent Algorithm [J]. Acta Armamentarii, 2022, 43(6): 1415-1425.
[2]	YU Zhefeng， XU Jianyu， LUO Yue， YANG Ying， LIU Jinbo， LAN Jingchuan. Time Series Extraction of Ablation Recession of Hypersonic Vehicle and Its Prediction Based on LSTM Neural Network [J]. Acta Armamentarii, 2021, 42(6): 1230-1237.
[3]	GAO Ang， DONG Zhiming， YE Hongbing， SONG Jinghua， GUO Qisheng. Loitering Munition Penetration Control Decision Based on Deep Reinforcement Learning [J]. Acta Armamentarii, 2021, 42(5): 1101-1110.
[4]	YUE Bin, MA Wen, HU Weijun. Adaptive Fault-tolerant Control of Hypersonic Vehicles [J]. Acta Armamentarii, 2021, 42(3): 521-529.
[5]	LIN Jun, HE Yingzi, HUANG Panxing. Research on Reentry Trajectory of Powered Hypersonic Vehicle with Discontinuous Ignition [J]. Acta Armamentarii, 2020, 41(7): 1307-1316.
[6]	YU Yue， WANG Honglun. Deep Learning-based Reentry Predictor-corrector Fault-tolerant Guidance for Hypersonic Vehicles [J]. Acta Armamentarii, 2020, 41(4): 656-669.
[7]	YU Zhefeng, CHEN Xuming, YANG Ying, BU Shaoqing, XIE Aimin, HUANG Jie, LIU Sen. Wake Transition of the Hypersonic Vehicle and Its Influence on RCS [J]. Acta Armamentarii, 2019, 40(12): 2467-2472.
[8]	CAI Guangbin, ZHAO Yang, ZHANG Shengxiu, YANG Xiaogang. Robust Multi-objective Linear Parameter-varying Control for Hypersonic Vehicle [J]. Acta Armamentarii, 2019, 40(11): 2229-2240.
[9]	ZHANG Shen， WANG Qing， DONG Chao-yang， YANG Ge. Anti-disturbance Control for Hypersonic Vehicles Based on Acceleration Measurement [J]. Acta Armamentarii, 2018, 39(9): 1733-1740.
[10]	MAO Hong-xia， JIA Ju-hong， FU De-bin， JIANG Yi. Research on Aero Thermodynamics and Influencing Factors for HIFiRE-1 [J]. Acta Armamentarii, 2018, 39(3): 528-536.
[11]	ZHAO He-wei, YANG Xiu-xia, SHEN Ru-song, HU Yun-an. Prescribed Performance Fine Attitude Control for Aeroelastic Hypersonic Vehicle [J]. Acta Armamentarii, 2017, 38(3): 501-511.