Anti-jamming Waveform Design of Ground-based Air Surveillance Radar Based on Reinforcement Learning

doi:10.12382/bgxb.2022.0069

Abstract

Abstract:

The electronic warfare capability of the detected target seriously affects the detection and identification performance of the ground-based air surveillance radar. To address the anti-jamming problem of the air surveillance radar, an anti-jamming waveform design method based on reinforcement learning is proposed. From the perspective of game theory, the reinforcement learning method is used to establish a dynamic confrontation model between radar and target jamming, calculate the state and action value of all parties in the game. The strategy iteration method is employed to generate the optimal strategy, and the time domain waveform is synthesized based on the phase iteration method. The simulation results shows that: when using the radar transmitted signal designed by the proposed method, compared with linear frequency-modulated signals and frequency-agile signals, the signal-to-interference-noise ratio is increased by 6.39dB and 1.12dB respectively; when the total signal power is 11W, compared with the frequency-agile signal, the target detection probability is increased by 5%, meaning that the designed signal can improve the anti-interception performance of radar signals while achieving anti-jamming at low power.

Key words: ground-based air surveillance radar, waveform design, reinforcement learning, anti-jamming

ZHENG Zexin, LI Wei, ZOU Kun, LI Yanfu. Anti-jamming Waveform Design of Ground-based Air Surveillance Radar Based on Reinforcement Learning[J]. Acta Armamentarii, 2023, 44(5): 1422-1430.

Figures/Tables 11

Fig.1 Radar detection scene

Fig.2 Signal model

Fig.3 Schematic diagram of Markovdecision process

Table 1 Parameter setting of reinforcement learning model

马尔可夫决策模型	参数设置
状态空间	S={7位6进制数}
动作空间	A={7位6进制数}
状态转移概率	P(s,a)=1/(6⁷-1)
γ	0.9

Table 2 Reward function structure

SINR/dB	<0	0~10	10~20	20~30	30~40	>40
Reward	-30	5	10	15	20	30

Table 3 Simulation parameters

雷达系统	仿真参数设置
工作波段	X波段
中心频率/GHz	9.5
信号带宽/MHz	140
子频带带宽/MHz	20
目标飞机速度/(m·s^-1)	250
$σ h 2$ (f)	{1, 0.2, 1.2, 6, 19, 1, 5}
噪声	1

Table 3 Simulation parameters

雷达系统	仿真参数设置
工作波段	X波段
中心频率/GHz	9.5
信号带宽/MHz	140
子频带带宽/MHz	20
目标飞机速度/(m·s^-1)	250
$σ h 2$ (f)	{1, 0.2, 1.2, 6, 19, 1, 5}
噪声	1

Fig.4 Game process one of radar and jamming

Fig.5 Game process two of radar and jamming

Fig.6 Radar strategy

Fig.7 Jamming strategy

Fig.8 Comparison of optimal waveform strategy and traditional radar signal

References 32

[1]	方文, 全英汇, 沙明辉, 等. 捷变频联合波形熵的密集假目标干扰抑制算法[J]. 系统工程与电子技术, 2021, 43(6):1506-1514.
	FANG W, QUAN Y H, SHA M H, et al. Dense false targets jamming suppression algorithm based on frequency agility and waveform entropy[J]. Systems Engineering and Electronic Technology, 2021, 43(6): 1506-1514. (in Chinese)
[2]	全英汇, 方文, 沙明辉, 等. 频率捷变雷达波形对抗技术现状与展望[J]. 系统工程与电子技术, 2021, 43(11):3126-3136.
	QUAN Y H, FANG W, SHA M H, et al. Present situation and prospects of frequency agility radar waveform countermeasures[J]. System Engineering and Electronic Technology, 2021, 43 (11): 3126-3136. (in Chinese)
[3]	晏艺翡, 苏军海, 李英军. 雷达低截获波形抗主瓣干扰技术研究与试验[J]. 火控雷达技术, 2021, 50(2):31-35.
	YAN Y F, SU J H, LI Y J. The research and experiment of anti-interference ability on radar with low-acquisition wave[J]. Fire Control Radar Technology, 2021, 50(2):31-35. (in Chinese)
[4]	THORNTON C, BUEHRER R M, MARTONE A. constrained contextual bandit learning for adaptive radar waveform selection[J]. IEEE Transactions on Aerospace and Electronic Systems, 2022, 58(2):1133-1148. doi: 10.1109/TAES.2021.3109110 URL
[5]	夏栋, 张凯旋, 丁友宝, 等. 基于相位编码波形捷变和CFAR技术抗同频干扰[J]. 系统工程与电子技术, 2022, 44(4):1210-1219.
	XIA D, ZHANG K X, DING Y B, et al. Anti-co-frequency interference based on phase-encoded waveform agility and CFAR technology[J]. Systems Engineering and Electronic Technology, 2022, 44(4):1210-1219. (in Chinese)
[6]	HU H T, LUI R H, ZHANG J Y. An improved radar anti-jamming method[C]∥Proceedings of the 2018 IEEE 3rd International Conference on Signal and Image Processing.Shenzhen, China:IEEE, 2018:439-443.
[7]	WANG H Y, YANG X K, LI Y. Radar anti-retransmitted jamming technology based on agility waveforms[C]∥Proceedings of the 2019 IEEE International Conference on Signal, Information and Data Processing.Chongqing, China:IEEE, 2019:1-6.
[8]	彭博, 陈齐乐, 李锐. 基于变分正则化参数估计的伪码调相引信重构式干扰方法[J]. 兵工学报, doi: 10.12382/bgxb.2022.0572. doi: 10.12382/bgxb.2022.0572
	PENG B, CHEN Q L, LI R. Parameters estimation of binary phase-coded modulation radar waveform based on non-convex total variation regularization[J]. Acta Armamentarii, doi: 10.12382/bgxb.2022.0572. (in Chinese) doi: 10.12382/bgxb.2022.0572
[9]	HE B, SU H T. Joint power allocation and beamforming between a multistatic radar and jammer based on game theory[C]∥Proceedings of the 2019 IEEE 11st International Conference on Communication Software and Networks. Chongqing, China:IEEE, 2019:337-341.
[10]	YE F, ZHAO T, LI Y B, et al. Anti-interference strategy selection method based on the minimum loss criterion[C] ∥Proceedings of the 2020 IEEE USNC-CNC-URSI North American Radio Science Meeting.Montreal, QC, Canada:IEEE, 2020:79-80.
[11]	CHEN H W, TA S Y, SUN B. Cooperative game approach to power allocation for target tracking in distributed MIMO radar sensor networks[J]. IEEE Sensors Journal, 2015, 15(10): 5423-5432. doi: 10.1109/JSEN.2015.2431261 URL
[12]	SHI C G, WEI Q, WANG F, et al. Cooperative LPI performance optimization for multistatic radar system: a stackelberg game[C]∥Proceedings of the 2019 International Applied Computational Electromagnetics Society Symposium-China.Nanjing, China:IEEE, 2019:1-2.
[13]	WANG D, LI K M, ZHANG Q, et al. A cooperative task allocation game for Multi-target imaging in radar networks[J]. IEEE Sensors Journal, 2021, 21(6):7541-7550. doi: 10.1109/JSEN.7361 URL
[14]	李伟, 王泓霖, 郑家毅, 等. 博弈条件下雷达波形设计策略研究[J]. 电子与信息学报, 2019, 41(11):2654-2660.
	LI W, WANG H L, ZHENG J Y, et al. Research on radar waveform design strategy under game conditions[J]. Journal of Electronics and Information, 2019, 41(11): 2654-2660. (in Chinese)
[15]	WANG H L, LI W, WANG H, et al. Radar waveform strategy based on game theory[J]. Radio Engineering, 2019, 28(4):757-764.
[16]	甘奕夫, 李伟, 赵俊龙. 干扰条件下基于Bayesian博弈的认知制导雷达波形设计[J]. 空军工程大学学报(自然科学版), 2021, 22(2):91-98.
	GAN Y F, LI W, ZHAO J L. A cognitive guidance radar waveform design based on bayesian game under conditions of jamming[J]. Journal of Air Force Engineering University(Natural Science Edition), 2021, 22(2):91-98. (in Chinese)
[17]	XING Q, ZHU W G, JIA X. Research on method of intelligent radar confrontation based on reinforcement learning[C]∥Proceedings of the 2017 2nd IEEE International Conference on Computational Intelligence and Applications. Beijing, China: IEEE, 2017:471-475.
[18]	XING Q, ZHU W G, JIA X. Intelligent countermeasure design of radar working-modes unknown[C]∥Proceedings of the 2017 IEEE International Conference on Signal Processing, Communications and Computing. Xiamen,China:IEEE, 2017:1-5.
[19]	SELVI E, BUEHRER R M, MARTON A, et al. On the use of markov decision processes in cognitive radar: an application to target tracking[C]∥Proceedings of the 2018 IEEE Radar Conference. Oklahoma City,OK,US:IEEE, 2018:0537-0542.
[20]	KANG L J, JIU B, LIU H W, et al. Reinforcement learning based anti-jamming frequency hopping strategies design for cognitive radar[C]∥Proceedings of the 2018 IEEE International Conference on Signal Processing, Communications and Computing. Qingdao,China:IEEE, 2018:1-5.
[21]	LI K, JIU B, LIU H W. Deep Q-network based anti-jamming strategy design for frequency agile radar[C]∥Proceedings of the 2019 International Radar Conference.Toulon, France:IEEE, 2019:1-5.
[22]	WANG L, FORTUNATI S, GRECO M S, et al. Reinforcement learning-based waveform optimization for MIMO multi-target detection[C]∥Proceedings of the 2018 52nd Asilomar Conference on Signals, Systems, and Computers.Pacific Grove, CA, US:IEEE, 2018:1329-1333.
[23]	CAO X, ZHENG Z, AN D. Adaptive waveform selection algorithm based on reinforcement learning for cognitive radar[C]∥Proceedings of the 2019 IEEE 2nd International Conference on Automation,Electronics and Electrical Engineering.Shenyang, China:IEEE, 2019:208-213.
[24]	WANG Q, QIAO Y M, GAO L R. A cognitive radar waveform optimization approach based on deep reinforcement learning[C]∥Proceedings of the 2019 IEEE International Conference on Signal, Information and Data Processing.Chongqing, China:IEEE, 2019:1-6.
[25]	AK S, BRÜGGENWIRTH S. Avoiding jammers: a reinforcement learning approach[C] ∥Proceedings of the 2020 IEEE International Radar Conference.Washington, DC, US: IEEE, 2020:321-326.
[26]	SHI Y C, JIU B, YAN J K, et al. Data-driven simultaneous multibeam power allocation: when multiple targets tracking meets deep reinforcement learning[J]. IEEE Systems Journal, 2021, 15(1):1264-1274. doi: 10.1109/JSYST.2020.2984774 URL
[27]	AILIYAW, YIW, YUANY. Reinforcement learning-based joint adaptive frequency hopping and pulse-width allocation for radar anti-jamming[C]∥Proceedings of the 2020 IEEE Radar Conference.Florence, Italy:IEEE, 2020:1-6.
[28]	ROMERO R A, BAE J, GOODMAN N A. Theory and application of SNR and mutual information matched illumination waveforms[J]. IEEE Transactions on Aerospace and Electronic Systems, 2011, 47(2): 912-927. doi: 10.1109/TAES.2011.5751234 URL
[29]	STEVEN M K. 统计信号处理基础: 估计与检测理论[M].罗鹏飞,张文明,刘忠,等,译. 北京: 电子工业出版社, 2014: 425-445.
	STEVEN M K. Fundamentals of statistical signal processing: estimation and detection theory[M]. LUO P F, ZHANG W M, LIU Z, et al. translated. Beijing: Publishing House of Electronics Industry, 2014:425-445. (in Chinese)
[30]	SUTTON R S, BARTO A G. Reinforcement learning: an introduction[M]. Cambridge,MA, US: MIT Press, 1998:67-98.
[31]	黎湘, 范梅梅. 认知雷达及其关键技术研究进展[J]. 电子学报, 2012, 40(9):1863-1870. doi: 10.3969/j.issn.0372-2112.2012.09.025
	LI X, FAN M M. Research progress of cognitive radar and its key technologies[J]. Acta Electronica Sinica, 2012, 40(9):1863-1870. (in Chinese)
[32]	JACKSON L, KAY S, VANKAYALAPATI N. Iterative method for nonlinear FM synthesis of radar signals[J]. IEEE Transactions on Aerospace and Electronic Systems, 2010, 46(2):910-917. doi: 10.1109/TAES.2010.5461666 URL