基于模型预测与策略学习的智能车辆人机协同控制算法

doi:10.12382/bgxb.2022.0815

摘要/Abstract

摘要：

针对智能车辆在复杂环境下高机动运动控制的难题,提出一种基于模型预测与策略学习的人机协同控制算法。该算法利用人类驾驶员对环境的理解和综合处理能力在决策规划层面辅助机器进行局部轨迹规划,包括速度调节和动态路径生成,实现决策规划层面的人机协同;面向高机动行驶的车辆在线优化规划与控制存在时效性问题,一方面在局部规划层采用较长采样间隔和简化的动力学模型设计基于模型预测控制的局部轨迹规划方法,以实现高效在线轨迹优化;另一方面在控制层采用基于滚动时域强化学习的学习型预测控制方法在线优化控制策略,以提升在线优化控制的计算效率与适应性。驾驶员在环的山区公路高机动仿真结果表明:新方法能遵从驾驶员的加减速指令和转向指令生成安全、平滑的规划轨迹,而且能够精确控制车辆沿期望轨迹行驶;在人机协同控制模式下,6位驾驶员完成相同驾驶任务的时间比手动驾驶平均缩短了8.3%,转向操作负荷降低了51.1%。

关键词: 智能车辆, 人机协同, 高机动运动, 强化学习, 模型预测控制

Abstract:

A human-machine collaborative control algorithm based on model prediction and policy learning is proposed for the optimal decision-making and high maneuvering motion control of intelligent vehicles in complex environments. The algorithm takes advantage of the human driver’s understanding of the environment and comprehensive processing ability to assist the machine in local trajectory planning at the decision planning level, including speed adjustment and dynamic path generation, to achieve the human-machine collaboration.For the timeliness of the online optimal planning and control of vehicles with high maneuverability, on the one hand, a long sampling interval and a simplified dynamics model are used to design a local trajectory planning method based on model predictive control at the local planning level in order to achieve efficient online trajectory optimization. On the other hand, a learning-based predictive control method based on rolling time-domain reinforcement learning is used to optimize the control strategy in the control layer in order to improve the computational efficiency and adaptability of online optimal control. In the driving simulation on the mountain highway with the driver in the loop, the proposed method not only complies with the driver’s acceleration and deceleration commands and steering commands to generate a safe and smooth planning trajectory for human-machine cooperation, but also can accurately control the vehicle to travel along the desired trajectory in real time. In the human-machine cooperative control mode, the time to complete the same driving task is reduced by 8.3% on average and the steering operation load is reduced by 51.1% compared with the manual driving by six ordinary drivers.

Key words: intelligent vehicles, human-machine collaboration, high maneuvering motion, reinforcement learning, model predictive control

中图分类号:

TP13

蒋岩, 丁语嫣, 张兴龙, 徐昕. 基于模型预测与策略学习的智能车辆人机协同控制算法[J]. 兵工学报, 2023, 44(11): 3465-3477.

JIANG Yan, DING Yuyan, ZHANG Xinglong, XU Xin. A Human-machine Collaborative Control Algorithm for Intelligent Vehicles Based on Model Prediction and Policy Learning[J]. Acta Armamentarii, 2023, 44(11): 3465-3477.

图/表 19

图1 智能车辆高机动运动人机协同控制系统框架

Fig.1 Framework of human-machine collaborative control system for high maneuvering motion of intelligent vehicle

图2 考虑横摆、俯仰和侧倾的车辆单轨动力学模型

Fig.2 Vehicle dynamics model considering yaw, pitch and roll

图3 道路坡度对车辆动力学模型影响示意

Fig.3 Influence of road grade on vehicle dynamics model

图4 车辆加速度限制示意

Fig.4 Schematic diagram of vehicle acceleration limit

图5 局部行驶路线规划撒点采样示意(o1为静态障碍物,o2为动态障碍物,o2,k表示k时刻碰撞处理后的障碍物2多边形包络,s表示沿道路中心线方向)

Fig.5 Schematic diagram of local driving route planning sprinkler sampling (o1 is a static obstacle, o2 is a dynamic obstacle, o2,k represents the polygonal envelope of obstacle 2 after collision processing at time k, and s is the direction of road centreline)

图6 点质量车辆动力学模型

Fig.6 Point mass vehicle dynamics model

图7 车辆轨迹跟踪误差示意

Fig.7 Schematic diagram of vehicle trajectory tracking error

图8 基于学习预测控制的智能车辆高机动运动控制器结构

Fig.8 Structure diagram of intelligent vehicle high maneuvering motion controller based on learning predictive control

图9 山区公路场景及行驶路线

Fig.9 Mountain road scene and driving route

表1 车辆模型参数以及规划器和控制器参数

Table 1 Vehicle model parameters, and planner and controller parameters

参数	数值	参数	数值
m/kg	1555	γ	0.95
h_CG/m	0.665	n_s	15
I_x/(kg·m²)	846.6	ζ	1.1
I_y/(kg·m²)	1 816	n_d	6
I_z/(kg·m²)	1 816	N_p	30
l_f/m	1.85	T_p/s	0.2
l_r/m	1.80	q_c	0.01
K_ϕ/(N·m·rad^-1)	51 600	q_l	0.1
D_ϕ/(N·m·s·rad^-1)	5 300	q_ω	5.0
K_θ/(N·m·rad^-1)	45 000	$r a x$	0.01
D_θ/(N·m·s·rad^-1)	2 600	$r a y$	0.5
C_cf/(N·rad^-1)	76 500	$r Δ a x$	0.2
C_cr/(N·rad^-1)	76 500	$r Δ a y$	1
w/m	1.7	N_c	50
a_y,_max/(m·s^-2)	4	T_c/s	0.02
a_y,_min/(m·s^-2)	-4	i_max	5
a_x,_max/(m·s^-2)	-4	/m	0.2
a_x,_min/(m·s^-2)	-6	$q e x$	0.2
w₁	0.5	$q e y$	0.2
w₂	3.0	$q e φ$	0.1
w₃	0.5	$r F x$	10^-9
w₄	0.5	$r δ f$	0.05

表1 车辆模型参数以及规划器和控制器参数

Table 1 Vehicle model parameters, and planner and controller parameters

参数	数值	参数	数值
m/kg	1555	γ	0.95
h_CG/m	0.665	n_s	15
I_x/(kg·m²)	846.6	ζ	1.1
I_y/(kg·m²)	1 816	n_d	6
I_z/(kg·m²)	1 816	N_p	30
l_f/m	1.85	T_p/s	0.2
l_r/m	1.80	q_c	0.01
K_ϕ/(N·m·rad^-1)	51 600	q_l	0.1
D_ϕ/(N·m·s·rad^-1)	5 300	q_ω	5.0
K_θ/(N·m·rad^-1)	45 000	$r a x$	0.01
D_θ/(N·m·s·rad^-1)	2 600	$r a y$	0.5
C_cf/(N·rad^-1)	76 500	$r Δ a x$	0.2
C_cr/(N·rad^-1)	76 500	$r Δ a y$	1
w/m	1.7	N_c	50
a_y,_max/(m·s^-2)	4	T_c/s	0.02
a_y,_min/(m·s^-2)	-4	i_max	5
a_x,_max/(m·s^-2)	-4	/m	0.2
a_x,_min/(m·s^-2)	-6	$q e x$	0.2
w₁	0.5	$q e y$	0.2
w₂	3.0	$q e φ$	0.1
w₃	0.5	$r F x$	10^-9
w₄	0.5	$r δ f$	0.05

表2 性能指标

Table 2 Performance indicators

符号	描述
t_task	完成驾驶任务总时长
L_2c	车辆在违反安全区域边界情况下的行驶距离
$E δ h$	转向操作做功,定义为 $E δ h$ = $∫ t t a s k 0$ ( $δ h 2$ + $δ · h 2$ )dt

表2 性能指标

Table 2 Performance indicators

符号	描述
t_task	完成驾驶任务总时长
L_2c	车辆在违反安全区域边界情况下的行驶距离
$E δ h$	转向操作做功,定义为 $E δ h$ = $∫ t t a s k 0$ ( $δ h 2$ + $δ · h 2$ )dt

图10 上坡场景人机协同驾驶仿真结果示例

Fig.10 Simulated results of human-machine collaborative driving in uphill scene

图11 上坡场景手动驾驶仿真结果示例

Fig.11 Simulated results of manual driving in uphill scene

图12 下坡场景人机协同驾驶仿真结果示例

Fig.12 Simulated results of human-machine collaborative driving in downhill scene

图13 下坡场景手动驾驶仿真结果示例

Fig.13 Simulated results of manual driving in downhill scene

图14 弯曲山路会车场景仿真结果示例

Fig.14 Simulated results of the scene of vehicles meeting on the winding mountain road

图15 不同时刻两车位置及轨迹

Fig.15 Positions and trajectories of two vehicles at different times

表3 山区公路场景下不同控制方法性能指标统计

Table 3 Statistics of performance indicators of different control methods in mountain road scene

方法	指标	测试人员
方法	指标	1	2	3	4	5	6
手动驾驶	t_task/s	268.4	244.9	247.9	233.5	227.7	248.1
	L₂_c/m	190.2	6.0	29.0	56.8	96.9	173.7
	$E δ h$	2945	2155	2089	2766	2587	1037
对比方法^[21]	t_task/s	226.1	237.4	238.7	229.3	245.6	232.4
	L₂_c/m	2.7	8.5	3.1	5.5	0	1.8
	$E δ h$	1557	1157	1256	1564	1437	975
本文方法	t_task/s	224.6	217.9	234.8	218.9	234.5	217.4
	L₂_c/m	0	0	0	0	0	0
	$E δ h$	1717	914	1011	975	1168	852

表3 山区公路场景下不同控制方法性能指标统计

Table 3 Statistics of performance indicators of different control methods in mountain road scene

方法	指标	测试人员
方法	指标	1	2	3	4	5	6
手动驾驶	t_task/s	268.4	244.9	247.9	233.5	227.7	248.1
	L₂_c/m	190.2	6.0	29.0	56.8	96.9	173.7
	$E δ h$	2945	2155	2089	2766	2587	1037
对比方法^[21]	t_task/s	226.1	237.4	238.7	229.3	245.6	232.4
	L₂_c/m	2.7	8.5	3.1	5.5	0	1.8
	$E δ h$	1557	1157	1256	1564	1437	975
本文方法	t_task/s	224.6	217.9	234.8	218.9	234.5	217.4
	L₂_c/m	0	0	0	0	0	0
	$E δ h$	1717	914	1011	975	1168	852

图16 山区公路场景下不同控制方法性能比较

Fig.16 Performance comparison of different control methods in mountain road scene

参考文献 21

[1]	陈慧岩, 陈舒平, 龚建伟. 智能汽车横向控制方法研究综述[J]. 兵工学报, 2017, 38(6):1203-1214. doi: 10.3969/j.issn.1000-1093.2017.06.021
	CHEN H Y, CHEN S P, GONG J W. Review on the research of lateral control for intelligent vehicles[J]. Acta Armamentarii, 2017, 38(6):1203-1214. (in Chinese)
[2]	胡云峰, 曲婷, 刘俊, 等. 智能汽车人机协同控制的研究现状与展望[J]. 自动化学报, 2019, 45(7):1261-1280.
	HU Y F, QU T, LIU J, et al. Human-machine cooperative control of intelligent vehicle: recent developments and future perspectives[J]. Acta Automatica Sinica, 2019, 45(7):1261-1280. (in Chinese)
[3]	ANSARI S, NAGHDY F, DU H P. Human-machine shared driving: challenges and future directions[J]. IEEE Transactions on Intelligent Vehicles, 2022, 7(3):499-519. doi: 10.1109/TIV.2022.3154426 URL
[4]	刘俊. 智能车辆人机协同转向控制策略研究[D]. 长春: 吉林大学, 2020:13-29.
	LIU J. Research on driver-automation cooperative steering control strategy of intelligent vehicle[D]. Changchun: Jilin University, 2020:13-29. (in Chinese)
[5]	LI M J, CAO H T, SONG X L, et al. Shared control driver assistance system based on driving intention and situation assessment[J]. IEEE Transactions on Industrial Informatics, 2018, 14(11): 4982-4994. doi: 10.1109/TII.2018.2865105 URL
[6]	XU S B, PENG H. Design, analysis, and experiments of preview path tracking control for autonomous vehicles[J]. IEEE Transactions on Intelligent Transportation Systems, 2019, 21(1): 48-58. doi: 10.1109/TITS.6979 URL
[7]	JI X W, HE X K, LÜ C, et al. Adaptive-neural-network-based robust lateral motion control for autonomous vehicle at driving limits[J]. Control Engineering Practice, 2018, 76: 41-53. doi: 10.1016/j.conengprac.2018.04.007 URL
[8]	MOUSAVINEJAD E, HAN Q L, YANG F, et al. Integrated control of ground vehicles dynamics via advanced terminal sliding mode control[J]. Vehicle System Dynamics, 2017, 55(2): 268-294. doi: 10.1080/00423114.2016.1256489 URL
[9]	NIE L Z, GUAN J Y, LU C H, et al. Longitudinal speed control of autonomous vehicle based on a self‐adaptive PID of radial basis function neural network[J]. IET Intelligent Transport Systems, 2018, 12(6): 485-494. doi: 10.1049/itr2.v12.6 URL
[10]	BEAl C E, GERDES J C. Model predictive control for vehicle stabilization at the limits of handling[J]. IEEE Transactions on Control Systems Technology, 2012, 21(4):1258-1269. doi: 10.1109/TCST.2012.2200826 URL
[11]	JI J, KHAJEPOUR A, MELEK W W, et al. Path planning and tracking for vehicle collision avoidance based on model predictive control with multiconstraints[J]. IEEE Transactions on Vehicular Technology, 2016, 66(2): 952-964. doi: 10.1109/TVT.2016.2555853 URL
[12]	LINIGER A, DOMAHIDI A, MORARI M. Optimization-based autonomous racing of 1: 43 scale RC cars[J]. Optimal Control Applications and Methods, 2015, 36(5): 628-647. doi: 10.1002/oca.v36.5 URL
[13]	LIAN C Q, XU X, CHEN H, et al. Near-optimal tracking control of mobile robots via receding-horizon dual heuristic programming[J]. IEEE Transactions on Cybernetics, 2015, 46(11):2484-2496. doi: 10.1109/TCYB.2015.2478857 URL
[14]	ZHANG X L, JIANG Y, LU Y, et al. A receding-horizon reinforcement learning approach for kinodynamic motion planning of autonomous vehicles[J]. IEEE Transactions on Intelligent Vehicles, 2022, 7(3): 556-568. doi: 10.1109/TIV.2022.3167271 URL
[15]	龚建伟, 姜岩, 徐威. 无人驾驶车辆模型预测控制[M]. 北京: 北京理工大学出版社, 2020.
	GONG J W, JIANG Y, XU W. Model predictive control for self-driving vehicles[M]. Beijing: Beijing Institute of Technology Press, 2020. (in Chinese)
[16]	BERNTORP K. Derivation of a six degrees-of-freedom ground-vehicle model for automotive applications[R]. Sweden: Department of Automatic Control, Lund University, Technical Report ISRN LUTFD2/TFRT-7627-SE, 2013.
[17]	SNIDER J M. Automatic steering methods for autonomous automobile path tracking[R]. Pittsburgh,PA,US:Robotics Institute, Technical Report CMU-RITR-09-08, 2009.
[18]	CHEN P Y. Effects of the entropy weight on TOPSIS[J]. Expert Systems with Applications, 2021, 168: 114186. doi: 10.1016/j.eswa.2020.114186 URL
[19]	LINIGER A, DOMAHIDI A, MORARI M. Optimization-based autonomous racing of 1: 43 scale RC cars[J]. Optimal Control Applications and Methods, 2015, 36(5):628-647. doi: 10.1002/oca.v36.5 URL
[20]	FRISON G, DIEHL M. HPIPM:a high-performance quadratic programming framework for model predictive control[J]. IFAC-PapersOnLine, 2020, 53(2): 6563-6569. doi: 10.1016/j.ifacol.2020.12.073 URL
[21]	YUE M, FANG C, ZHANG H Z, et al. Adaptive authority allocation-based driver-automation shared control for autonomous vehicles[J]. Accident Analysis & Prevention, 2021, 160: 106301. doi: 10.1016/j.aap.2021.106301 URL