A DDPG-based Trajectory Planning Method for Collision Avoidance of Morphing Spacecraft

doi:10.12382/bgxb.2023.1182

Abstract

Abstract:

To address the problem of the strong coupling between the guidance and morphing decision of morphing spacecraft, a morphing collision avoidance trajectory planning method of considering obstacle constraint based on deep deterministic policy gradient (DDPG) is proposed. A kinematic model of morphing aerospace craft is established according to morphing parameter. A longitudinal guidance law with a range error correction function and a lateral guidance law based on line-of-sight angle deviation are designed to realize the obstacle circumvention and ensure the terminal guidance accuracy. Then a Markov decision model is constructed to facilitate a continuous morphing. The angle of attack, Mach, and relative distance from the spacecraft to the obstacle are taken as the state space. The potential field penalty function considering collision and the smallest terminal guidance error reward function is considered in the design. The DDPG network is then trained to generate a map of decision instruction from the state space and obtain the optimal shape decision instruction. The simulated results show that, compared with configuration-fixed spacecraft, the guidance accuracy and lateral obstacle avoidance ability of morphing spacecraft are improved by optimizing the shape, and the requirement for the detection ability of air borne radar is reduced to save the detection cost.

Key words: morphing spacecraft, deep deterministic policy gradient, intelligent decision-making, trajectory planning, collision avoidance

CLC Number:

V448.235

DING Tianyun, XIA Yi, MEI Zewei, SHAO Xingling, LIU Jun. A DDPG-based Trajectory Planning Method for Collision Avoidance of Morphing Spacecraft[J]. Acta Armamentarii, 2024, 45(11): 3903-3914.

Figures/Tables 20

Fig.1 Structure block diagram of trajectory planning technology for morphing intelligent decision

Fig.2 Schematic diagram of no-fly zone

Fig.3 Training process of DDPG algorithm

Fig.4 Structure diagram of neural network of DDPG

Table 1 Initialization parameters setting for simulation

训练参数	数值
Critic网络学习率	0.001
Actor网络学习率	0.0001
折扣因子	0.99
抽样样本大小	128
经验数据集大小	1000000
探索噪声方差	0.1
软更新率	0.01
航向角阈值ψ_des/(°)	5

Fig.5 Comparison of flight paths

Fig.6 Reward curves for variable configuration training

Fig.7 Variation of related morphing parameters during flying

Fig.8 Profiles of different angles of attack

Table 2 Simulation results in Scenario 1

类别	经纬度偏差/(°)	剩余航程/km
变外形	(0.0115,-0.0158)	2.17
外形1	(0.0468,-0.0672)	9.101
外形2	(0.041,-0.0122)	4.754
外形3	(0.0581,-0.0345)	7.514

Fig.9 Single shape track diagram of different speed nodes

Fig.10 Variable shape track diagram of different speed nodes

Fig.11 Single shape track chart with maximum value of different angles of attack

Fig.12 Variable shape track chart with maximum value of different angles of attack

Fig.13 Comparison of emergency obstacle avoidance

Table 3 The minimum allowable detection radius

变外形	外形1	外形2	外形3
83%	87%	100%	93%

Fig.14 Flight path diagram of two methods

Fig.15 Variation diagram of related parameters of two methods

Fig.16 Flight path diagram of two methods

Fig.17 Variation diagram of related parameters of two methods

References 24

[1]	杜万闪, 周洲, 拜昱, 等. 组合式飞行器多体动力学建模与飞行力学特征[J]. 兵工学报, 2023, 44(8): 2245-2262. doi: 10.12382/bgxb.2022.0282
	DU W S, ZHOU Z, BAI Y, et al. Multi-body dynamics modeling and flight mechanics characteristics of combined aircraft[J]. Acta Armamentarii, 2023, 44(8): 2245-2262. (in Chinese)
[2]	张晚晴, 余文斌, 李静琳, 等. 基于纵程解析解的飞行器智能横程机动再入协同制导[J]. 兵工学报, 2021, 42(7): 1400-1411.
	ZHANG W Q, YU W B, LI J L, et al. Intelligent cross-range maneuvering reentry cooperative guidance for air-craft based on longitudinal analytic solution[J]. Acta Armamentarii, 2021, 42(7):1400-1411. (in Chinese)
[3]	姜斌, 孟庆开, 杨浩. 航天器姿轨控制研究综述:微分几何控制方法[J]. 控制与决策, 2023, 38(8): 2079-2092.
	JIANG B, MENG Q K, YANG H. A survey on spacecraft attitude and orbit control: differential geometric control approaches[J]. Control and Decision, 2023, 38(8): 2079-2092. (in Chinese)
[4]	陈树生, 贾苜梁, 刘衍旭, 等. 变体飞行器变形方式及气动布局设计关键技术研究进展[J]. 航空学报, 2024, 45(5): 629595.
	CHEN S S, JIA M L, LIU Y X, et al. Development status on deformation modes and key technologies of aerodynamic layout design about morphing aircraft[J]. Acta Aeronautica et Astronautica Sinica, 2024, 45(5):629595. (in Chinese)
[5]	吴康盛. 变体飞行器飞行控制与外形决策研究[D]. 长沙: 湖南大学, 2022.
	WU K S. The research on flight control and shape decision-making of morphing aircraft[D]. Changsha: Hunan University, 2022. (in Chinese)
[6]	张尧, 张婉, 别大卫, 等. 智能变体飞行器研究综述与发展趋势分析[J]. 飞航导弹, 2021(6):14-23.
	ZHANG Y, ZHANG W, BIE D W, et al. Research summary and development trend analysis of intelligent variable aircraft[J]. Aerodynamic Missile Journal, 2021(6):14-23. (in Chinese)
[7]	冉茂鹏, 王成才, 刘华华, 等. 变体飞行器控制技术发展现状与展望[J]. 航空学报, 2022, 43(10):432-449.
	RAN M P, WANG C C, LIU H H, et al. Research status and future development of morphing aircraft control technology[J]. Acta Aeronautica et Astronautica Sinica, 2022, 43(10): 432-449. (in Chinese)
[8]	甄子洋, 刘攀, 陆宇平. 变体飞行器智能变形与飞行控制技术研究进展[J]. 南京航空航天大学学报, 2022, 54(6): 995-1006.
	ZHEN Z Y, LIU P, LU Y P. Research progress on intelligent deformation and flight control technology of morphing aircraft[J]. Journal of Nanjing University of Aeronautics & Astronautics, 2022, 54(6): 995-1006. (in Chinese)
[9]	王子健. 变体飞行器的变形辅助机动控制[D]. 哈尔滨: 哈尔滨工业大学, 2022.
	WANG Z J. Morphing-aided maneuver control of morphing aircraft[D]. Harbin: Harbin Institute of Technology, 2022. (in Chinese)
[10]	LITTMAN M L. Reinforcement learning improves behaviour from evaluative feedback[J]. Nature, 2015, 521:445-451.
[11]	王青, 刘华华. 变体飞行器智能自主决策与控制[J]. 现代防御技术, 2020, 48(6): 5-11. doi: 10.3969/j.issn.1009-086x.2020.06.002
	WANG Q, LIU H H. Intelligent autonomous decision-making and control of morphing aircraft[J]. Modern Defence Technology, 2020, 48(6):5-11. (in Chinese)
[12]	徐小野, 李爱军, 张丛丛, 等. 基于Q学习的变体无人机控制系统设计[J]. 西北工业大学学报, 2012, 30(3):340-344.
	XU X Y, LI A J, ZHANG C C, et al. Applying Q-learning to designing feasible morphing UAV control system[J]. Journal of Northwestern Polytechnical University, 2012, 30(3): 340-344. (in Chinese)
[13]	闫斌斌, 李勇, 戴沛, 等. 基于增强学习的变体飞行器自适应变体策略与飞行控制方法研究[J]. 西北工业大学学报, 2019, 37(4): 656-663.
	YAN B B, LI Y, DAI P, et al. Adaptive wing morphing strategy and flight control method of a morphing aircraft based on reinforcement learning[J]. Journal of Northwestern Polytechnical University, 2019, 37(4):656-663. (in Chinese)
[14]	LILLICRAP T P, HUNT J J, PRITZEL A, et al. Continuous control with deep reinforcement learning[J]. Computing Research Repository, 2015, abs/1509.02971.
[15]	温暖, 刘正华, 祝令谱, 等. 深度强化学习在变体飞行器自主外形优化中的应用[J]. 宇航学报, 2017, 38(11):1153-1159.
	WEN N, LIU Z H, ZHU L P, et al. Deep reinforcement learning and its application on autonomous shape optimization for morphing aircraft[J]. Journal of Astronautics, 2017, 38(11): 1153-1159. (in Chinese)
[16]	XU D, HUI Z, LIU Y Q, et al. Morphing control of a new bionic morphing UAV with deep reinforcement learning[J]. Aerospace Science and Technology, 2019, 92: 232-243.
[17]	桑晨, 郭杰, 唐胜景, 等. 基于DDPG算法的变体飞行器自主变形决策[J]. 北京航空航天大学学报, 2022, 48(5):910-919.
	SANG C, GUO J, TANG S J, et al. Autonomous deformation decision making of morphing aircraft based on DDPG algorithm[J]. Journal of Beijing University of Aeronautics and Astronautics, 2022, 48(5): 910-919. (in Chinese)
[18]	张远, 黄万伟, 聂莹, 等. 一种高速可变形飞行器智能变形决策方法[J]. 宇航学报, 2022, 43(12):1665-1675.
	ZHANG Y, HUANG W W, NIE Y, et al. An intelligent deformation decision-making method for high-speed morphing flight vehicle[J]. Journal of Astronautics, 2022, 43(12): 1665-1675. (in Chinese)
[19]	吴则良, 叶建川, 王江, 等. 基于深度自动编码器神经网络的飞行器翼型参数降维与优化设计[J]. 兵工学报, 2022, 43(6):1326-1336. doi: 10.12382/bgxb.2021.0346
	WU Z L, YE J C, WANG J, et al. Parameter dimensionality reduction and optimal design of aircraft airfoil based on deep autoencoder neural network[J]. Acta Armamentarii, 2022, 43(6): 1326-1336. (in Chinese) doi: 10.12382/bgxb.2021.0346
[20]	田若岑, 张庆振, 郭云鹤, 等. 基于禁飞区规避的高超声速飞行器再入制导律设计[J]. 空天防御, 2022, 5(2): 65-74.
	TIAN R C, ZHANG Q Z, GUO Y H, et al. Design of hypersonic vehicle reentry guidance law based on no-fly zone avoidance[J]. Air-space Defense Force Structure, 2022, 5(2): 65-74. (in Chinese)
[21]	周亮, 王昊宇, 尚海滨, 等. 基于高斯伪谱法的天基再入飞行器滑翔轨迹优化设计研究[J]. 空天防御, 2020, 3(3): 89-95.
	ZHOU L, WANG H Y, SHANG H B, et al. Optimization design of glide trajectory of space-based reentry vehicle based on Gaussian pseudo-spectrum method[J]. Air-space Defense Force Structure, 2020, 3(3): 89-95. (in Chinese)
[22]	PENG Z H, WANG D, LI T S, et al. Output-feedback cooperative formation maneuvering of autonomous surface vehicles with connectivity preservation and collision avoidance[J]. IEEE Transactions on Cybernetics, 2020, 50(6): 2527-2535. doi: 10.1109/TCYB.2019.2914717 pmid: 31180878
[23]	YADOLLAH R, AMIR K, KEN S C, et al. A potential field-based model predictive path-planning controller for autonomous road vehicles[J]. IEEE Transactions on Intelligent Transportation Systems, 2017, 18(5):1255-1267.
[24]	ZHANG H P, WANG H L, LI N, et al. Time-optimal memetic whale optimization algorithm for hypersonic vehicle reentry trajectory optimization with no-fly zones[J]. Neural Computing & Applications, 2020, 32(7):2735-2749.