Tracking and Aiming Adaptive Control for Unmanned Combat Ground Vehicle on the Move Based on Reinforcement LearningCompensation

doi:10.12382/bgxb.2021.0786

Abstract

Abstract: To deal with the nonlinear interference caused by chassis movement and road surface undulations with the tracking and aiming of unmanned combat ground vehicles, a tracking and aiming adaptive control method for unmanned combat ground vehicles on the move based on reinforcement learning compensation is proposed. This method consists of a main controller and a compensation controller. The main controller uses the PID control algorithm combined with the current tracking error to obtain the main control quantity, and the compensation controller uses the Dueling DQN reinforcement learning network to process the current state of the combat vehicle as well as the road surface undulation information near the local planning path to obtain the compensation control quantity. Firstly, the integrated kinematics model of the unmanned combat ground vehicle is established. Then, the compensation control algorithm based on reinforcement learning is described. Finally, simulation and verification are performed in three-dimensional scenes based on the V-REP dynamic software. The experimental results show that the tracking and aiming control method based on reinforcement learning compensation has good adaptive ability for chassis movement and road surface undulations, which effectively improves the tracking/aiming accuracy and stability of unmanned combat vehicles.

Key words: unmannedcombatgroundvehicle, trackingandaimingonthemove, reinforcementlearning, adaptivecontrol, compensationcontrol

CLC Number:

TJ810.2

WEI Lianzhen, GONG Jianwei, CHEN Huiyan, LI Zirui, GONG Cheng. Tracking and Aiming Adaptive Control for Unmanned Combat Ground Vehicle on the Move Based on Reinforcement LearningCompensation[J]. Acta Armamentarii, 2022, 43(8): 1947-1955.

References

［1］陈慧岩, 张玉. 军用地面无人机动平台技术发展综述[J]. 兵工学报, 2014, 35(10): 1696-1706.
CHEN H Y, ZHANG Y. An overview of research on military unmanned ground vehicles[J]. Acta Armamentarii, 2014, 35(10): 1696-1706. (in Chinese)
[2］陈建荣, 郭齐胜, 刘军. 地面运动目标攻击命中概率模型及仿真[J]. 火力与指挥控制, 2007, 32(7): 43-46.
CHEN J R, GUO Q S, LIU J. Hit probability model of attacking the mobile ground target and simulation[J]. Fire Control and Command Control, 2007, 32(7): 43-46. (in Chinese)
[3］罗来科, 王耀北, 王顺岭. 稳像火控系统误差分析[J]. 火力与指挥控制, 2002, 27(5): 30-32, 35.
LUO L K, WANG Y B, WANG S L. Error analysis of the image-stabilized fire control system[J]. Fire Control and Command Control, 2002, 27(5): 30-32, 35. (in Chinese)
[4］郝强, 南立军, 刘斌, 等. 坦克火控系统瞄准线平移的补偿方法[J]. 火炮发射与控制学报, 2018, 39(3): 71-75.
HAO Q, NAN L J, LIU B, et al. Compensation method of aiming line translation of tank fire control system[J]. Journal of Gun Launch & Control, 2018, 39(3): 71-75. (in Chinese)
[5］钟洲, 姜毅, 刘群. 车载防空导弹行进间发射过程动力学数值分析[J]. 兵工学报, 2014, 35(1): 83-87.
ZHONG Z, JIANG Y, LIU Q.Dynamics numerical analysis of vehicle-mounted antiaircraft missile launching on the move[J]. Acta Armamentarii, 2014, 35(1): 83-87. (in Chinese)
[6］慕巍, 张宝宜, 王新明, 等. 适用于光电跟踪仪的高速目标跟踪控制算法[J]. 激光与红外, 2020, 50(4): 468-474.
MU W, ZHANG B Y, WANG X M, et al.High speed target tracking control algorithm fbr electro-optical tracker[J]. Laser & Infrared, 2020, 50(4): 468-474. (in Chinese)
[7］熊珍凯, 陈汀峰. 精确跟瞄控制技术研究[J]. 强激光与粒子束, 2012, 24(6): 1339-1343.
XIONG Z K, CHEN T F. High precision tracking and pointing control technique[J]. High Power Laser and Particle Beams, 2012, 24(6): 1339-1343. (in Chinese)
[8］张卫民, 梁建奇, 马红卫, 等. 自行火炮自动直瞄控制方法研究[J]. 兵工学报, 2015, 36(1): 182-186.
ZHANG W M, LIANG J Q, MA H W, et al. An automatic direct aiming control method of self-propelled artillery[J]. Acta Armamentarii, 2015, 36(1): 182-186. (in Chinese)
[9］朱斌, 谢杰, 孙皓泽, 等. 某新型坦克稳瞄系统自抗扰控制器的设计[J]. 计算机工程与应用, 2013, 49(增刊3): 71-75.
ZHU B, XIE J, SUN H Z, et al. Design of active disturbance rejection controller for some new type tank steady sighting system[J]. Computer Engineering and Applications, 2013, 49(S3): 71-75. (in Chinese)
[10］张文丽, 郭俊文, 曲俊海, 等. 基于自适应差分进化算法的武器稳定系统参数辨识[J]. 火力与指挥控制, 2020, 45(5): 119-124.
ZHANG W L, GUO J W, QU J H, et al. Parameter identification of weapon stability system based on adaptive differential evolution algorithm[J]. Fire Control and Command Control, 2020, 45(5): 119-124. (in Chinese)
[11］鲁浩. 基于瞬时转向中心实时估计的滑动转向车辆运动轨迹预测方法研究[D]. 北京:北京理工大学, 2016.
LU H. Trajectory prediction based on estimation of instantaneous centers of rotation in real time for skid-steer vehicles[D]. Beijing: Beijing Institute of Technology, 2016. (in Chinese)
[12］盖江涛, 刘春生, 马长军, 等. 考虑履带滑转滑移的电驱动履带车辆转向控制 [J]. 兵工学报, 2021, 42(10): 2092-2101.
GAI J T, LIU C S, MA C J, et al.Steering control of electric drive tracked vehicle considering tracks' skid and slip[J]. Acta Armamentarii, 2021, 42(10): 2092-2101. (in Chinese)
[13］李壮. 基于深度强化学习的六自由度机械臂避障规划[D]. 北京理工大学, 2020.
LI Z. Obstacle avoidance planning of six degrees of freedom manipulator based on deep reinforcement learning[D]. Beijing: Beijing Institute of Technology, 2020. (in Chinese)
[14］冷鹏飞, 徐朝阳. 一种深度强化学习的雷达辐射源个体识别方法[J]. 兵工学报, 2018, 39(12): 2420-2426.
LENG P F, XU C Y. Specific emitter identification based on deep reinforcement learning[J]. Acta Armamentarii, 2018, 39(12): 2420-2426. (in Chinese)
[15］ LIU B Y, WANG L J, LIU M. Lifelong federated reinforcement learning: a learning architecture for navigation in cloud robotic systems[J]. IEEE Robotics Automation Letters, 2019, 4(4): 4555-4562.
[16］ GONG C, LI Z, ZHOU X, et al. Orientation-aware planning for parallel task execution of omni-directional mobile robot[C]∥Proceedings of 2021 International Conference on Intelligent Robots and Systems. Prague, Czech: IEEE/RSJ, 2021: 6891-6898.
[17］ MNIH V, KAVUKCUOGLU K, SILVER D, et al. Human-level control through deep reinforcement learning[J]. Nature, 2015, 518(7540): 529-533.
[18］ WANG Z, SCHAUL T, HESSEL M, et al. Dueling network architectures for deep reinforcement learning[C]∥Proceedings of the 33rd International Conference on Machine Learning. New York, NY, US: PMLR, 2016: 1995-2003.
[19］ DADAY M J A, MILLADO K F M R. Enhanced reinforcement learning with targeted dropout[C]∥Proceedings of 2019 International Conference on Digitization (ICD). Sharjah, Emirate: IEEE, 2019: 207-211.
[20］ ROHMER E, SINGH S P N, FREESE M. V-REP: A versatile and scalable robot simulation framework[C]∥Proceedings of 2013 International Conference on Intelligent Robots and Systems (IROS). Tokyo, Japan: IEEE/RSJ, 2013: 1321-1326.
[21］ PERLIN K. Improving noise[J]. ACM Transactions on Graphics, 2002, 21(3): 681-682.

[1]	LI Li， LI Xuguang， GUO Kaijie， SHI Chao， CHEN Zhaowen. Simulation of Ground-air Cooperative Combat Based on Reinforcement Learning in Localization Environment [J]. Acta Armamentarii, 2022, 43(S1): 74-81.
[2]	MIAO Shuangquan， ZHANG Baoquan， WANG Mingchao， WANG Xinwei， SHEN Yu. Adaptive Exponential Time-varying Sliding Mode Control based on Disturbance Observer for Photoelectric Stabilized AirbornePlatform [J]. Acta Armamentarii, 2022, 43(7): 1636-1645.
[3]	MA Ye， FAN Wenhui， CHANG Tianqing. Optimization Method of Unmanned Swarm Defensive Combat Scheme Based on Intelligent Algorithm [J]. Acta Armamentarii, 2022, 43(6): 1415-1425.
[4]	ZHANG Tongtong, JIANG Huhai, YUE Wei, SI Chen, YUAN Man. Adaptive Control Based on RBF Neural Network for Electro-optical System [J]. Acta Armamentarii, 2022, 43(3): 556-564.
[5]	ZHU Jianwen, ZHAO Changjian, LI Xiaoping, BAO Weimin. Multi-target Assignment and Intelligent Decision Based on Reinforcement Learning [J]. Acta Armamentarii, 2021, 42(9): 2040-2048.
[6]	CHEN Zhongyuan, WEI Wenshu, CHEN Wanchun. Reinforcement Learning-based Intelligent Guidance Law for Cooperative Attack of Multiple Missiles [J]. Acta Armamentarii, 2021, 42(8): 1638-1647.
[7]	ZHANG Wanqing, YU Wenbin, LI Jinglin, CHEN Wanchun. Cooperative Reentry Guidance for Intelligent Lateral Maneuver of Hypersonic Vehicle Based on Downrange Analytical Solution [J]. Acta Armamentarii, 2021, 42(7): 1400-1411.
[8]	WANG Huidong， ZHOU Laihong. Design of A Backstepping Integral Adaptive Controller for Quadrotor UAV [J]. Acta Armamentarii, 2021, 42(6): 1283-1289.
[9]	GAO Ang， DONG Zhiming， YE Hongbing， SONG Jinghua， GUO Qisheng. Loitering Munition Penetration Control Decision Based on Deep Reinforcement Learning [J]. Acta Armamentarii, 2021, 42(5): 1101-1110.
[10]	YUE Bin, MA Wen, HU Weijun. Adaptive Fault-tolerant Control of Hypersonic Vehicles [J]. Acta Armamentarii, 2021, 42(3): 521-529.
[11]	LIAO Zili， SHU Xin， GAO Qiang， LI Jiaqi. Double Channel Compensation Control of Bus Voltage for Armored Vehicle with Electric Transmission [J]. Acta Armamentarii, 2021, 42(10): 2082-2091.
[12]	JING Anyan， SHE Huqing. Neural Network Observer-based Output Attitude Control of a Towed Underwater Vehicle [J]. Acta Armamentarii, 2020, 41(12): 2504-2513.
[13]	LI Xiaohang， ZHANG Zhihu， SONG Chunlin， WU Qiang， DONG Fangyong. Adaptive Compensation Control under False Data Injection Attack [J]. Acta Armamentarii, 2020, 41(11): 2260-2265.
[14]	LIU Lei, YAO Jianyong, MA Dawei, WANG Guangwen. Low-frequency Learning-based Robust Adaptive Control for Electro-hydraulic Position Servo System [J]. Acta Armamentarii, 2019, 40(4): 737-743.
[15]	YIN Tingting, JIA Fangxiu, YU Jiyan, WANG Xiaoming. Robust Adaptive Control of Roll Position of Fixed Rudder for Dual-spin Projectile with Improved LuGre Friction Model [J]. Acta Armamentarii, 2019, 40(12): 2425-2432.