A Human-machine Collaborative Control Algorithm for Intelligent Vehicles Based on Model Prediction and Policy Learning

Yan JIANG; Yuyan DING; Xinglong ZHANG; Xin XU

doi:10.12382/bgxb.2022.0815

您当前的位置：

首页 >

文章列表页 >

A Human-machine Collaborative Control Algorithm for Intelligent Vehicles Based on Model Prediction and Policy Learning

更新时间：2025-08-18

- A Human-machine Collaborative Control Algorithm for Intelligent Vehicles Based on Model Prediction and Policy Learning
- Acta Armamentarii Vol. 44, Issue 11, Pages: 3465-3477(2023)
- 作者机构：
  
  国防科技大学智能科学学院, 湖南长沙 410073
- 作者简介：
- 基金信息：
- DOI：10.12382/bgxb.2022.0815
  CLC：
- Received：07 September 2022，
  
  Published Online：12 December 2023，
  
  Published：30 November 2023
- 稿件说明：
移动端阅览
Yan JIANG, Yuyan DING, Xinglong ZHANG, et al. A Human-machine Collaborative Control Algorithm for Intelligent Vehicles Based on Model Prediction and Policy Learning[J]. Acta Armamentarii, 2023, 44(11): 3465-3477.
DOI：

Yan JIANG, Yuyan DING, Xinglong ZHANG, et al. A Human-machine Collaborative Control Algorithm for Intelligent Vehicles Based on Model Prediction and Policy Learning[J]. Acta Armamentarii, 2023, 44(11): 3465-3477. DOI： 10.12382/bgxb.2022.0815.

摘要

针对智能车辆在复杂环境下高机动运动控制的难题

提出一种基于模型预测与策略学习的人机协同控制算法。该算法利用人类驾驶员对环境的理解和综合处理能力在决策规划层面辅助机器进行局部轨迹规划

包括速度调节和动态路径生成

实现决策规划层面的人机协同;面向高机动行驶的车辆在线优化规划与控制存在时效性问题

一方面在局部规划层采用较长采样间隔和简化的动力学模型设计基于模型预测控制的局部轨迹规划方法

以实现高效在线轨迹优化;另一方面在控制层采用基于滚动时域强化学习的学习型预测控制方法在线优化控制策略

以提升在线优化控制的计算效率与适应性。驾驶员在环的山区公路高机动仿真结果表明:新方法能遵从驾驶员的加减速指令和转向指令生成安全、平滑的规划轨迹

而且能够精确控制车辆沿期望轨迹行驶;在人机协同控制模式下

6位驾驶员完成相同驾驶任务的时间比手动驾驶平均缩短了8.3%

转向操作负荷降低了51.1%。

Abstract

A human-machine collaborative control algorithm based on model prediction and policy learning is proposed for the optimal decision-making and high maneuvering motion control of intelligent vehicles in complex environments. The algorithm takes advantage of the human driver’s understanding of the environment and comprehensive processing ability to assist the machine in local trajectory planning at the decision planning level

including speed adjustment and dynamic path generation

to achieve the human-machine collaboration.For the timeliness of the online optimal planning and control of vehicles with high maneuverability

on the one hand

a long sampling interval and a simplified dynamics model are used to design a local trajectory planning method based on model predictive control at the local planning level in order to achieve efficient online trajectory optimization. On the other hand

a learning-based predictive control method based on rolling time-domain reinforcement learning is used to optimize the control strategy in the control layer in order to improve the computational efficiency and adaptability of online optimal control. In the driving simulation on the mountain highway with the driver in the loop

the proposed method not only complies with the driver’s acceleration and deceleration commands and steering commands to generate a safe and smooth planning trajectory for human-machine cooperation

but also can accurately control the vehicle to travel along the desired trajectory in real time. In the human-machine cooperative control mode

the time to complete the same driving task is reduced by 8.3% on average and the steering operation load is reduced by 51.1% compared with the manual driving by six ordinary drivers.

关键词

Keywords

references

陈慧岩 , 陈舒平 , 龚建伟 . 智能汽车横向控制方法研究综述 [J ] . 兵工学报 , 2017 , 38 ( 6 ): 1203 - 1214 . DOI: 10.3969/j.issn.1000-1093.2017.06.021 http://doi.org/10.3969/j.issn.1000-1093.2017.06.021 智能汽车在提高行驶安全性和减少交通事故方面有很大的优势，已成为世界范围内的研究热点。综述了智能汽车横向控制的国内外发展历程与研究现状；介绍了车辆横向动力学和轮胎力学的研究历程和模型；阐述了智能汽车横向控制理论和方法以及自动转向执行机构的设计；给出智能汽车横向控制研究的重点和发展趋势。通过分析认为，系统非线性、不确定性和时变特性的智能汽车横向动力学建模和横向控制器设计，特别是高速时的横向控制，以及智能车辆感知决策系统与车辆本身系统的一体化设计，将是今后研究的重点。

CHEN H Y , CHEN S P , GONG J W . Review on the research of lateral control for intelligent vehicles [J ] . Acta Armamentarii , 2017 , 38 ( 6 ): 1203 - 1214 . (in Chinese)

胡云峰 , 曲婷 , 刘俊 , 等 . 智能汽车人机协同控制的研究现状与展望 [J ] . 自动化学报 , 2019 , 45 ( 7 ): 1261 - 1280 .

HU Y F , QU T , LIU J , et al . Human-machine cooperative control of intelligent vehicle: recent developments and future perspectives [J ] . Acta Automatica Sinica , 2019 , 45 ( 7 ): 1261 - 1280 . (in Chinese)

ANSARI S , NAGHDY F , DU H P . Human-machine shared driving: challenges and future directions [J ] . IEEE Transactions on Intelligent Vehicles , 2022 , 7 ( 3 ): 499 - 519 . DOI: 10.1109/TIV.2022.3154426 http://doi.org/10.1109/TIV.2022.3154426 https://ieeexplore.ieee.org/document/9721611/ https://ieeexplore.ieee.org/document/9721611/

刘俊 . 智能车辆人机协同转向控制策略研究 [D ] . 长春 : 吉林大学 , 2020 : 13 - 29 .

LIU J . Research on driver-automation cooperative steering control strategy of intelligent vehicle [D ] . Changchun : Jilin University , 2020 : 13 - 29 . (in Chinese)

LI M J , CAO H T , SONG X L , et al . Shared control driver assistance system based on driving intention and situation assessment [J ] . IEEE Transactions on Industrial Informatics , 2018 , 14 ( 11 ): 4982 - 4994 . DOI: 10.1109/TII.2018.2865105 http://doi.org/10.1109/TII.2018.2865105 https://ieeexplore.ieee.org/document/8434099/ https://ieeexplore.ieee.org/document/8434099/

XU S B , PENG H . Design, analysis, and experiments of preview path tracking control for autonomous vehicles [J ] . IEEE Transactions on Intelligent Transportation Systems , 2019 , 21 ( 1 ): 48 - 58 . DOI: 10.1109/TITS.6979 http://doi.org/10.1109/TITS.6979 https://ieeexplore.ieee.org/xpl/RecentIssue.jsp?punumber=6979 https://ieeexplore.ieee.org/xpl/RecentIssue.jsp?punumber=6979

JI X W , HE X K , LÜ C , et al . Adaptive-neural-network-based robust lateral motion control for autonomous vehicle at driving limits [J ] . Control Engineering Practice , 2018 , 76 : 41 - 53 . DOI: 10.1016/j.conengprac.2018.04.007 http://doi.org/10.1016/j.conengprac.2018.04.007 https://linkinghub.elsevier.com/retrieve/pii/S0967066118300881 https://linkinghub.elsevier.com/retrieve/pii/S0967066118300881

MOUSAVINEJAD E , HAN Q L , YANG F , et al . Integrated control of ground vehicles dynamics via advanced terminal sliding mode control [J ] . Vehicle System Dynamics , 2017 , 55 ( 2 ): 268 - 294 . DOI: 10.1080/00423114.2016.1256489 http://doi.org/10.1080/00423114.2016.1256489 https://www.tandfonline.com/doi/full/10.1080/00423114.2016.1256489 https://www.tandfonline.com/doi/full/10.1080/00423114.2016.1256489

NIE L Z , GUAN J Y , LU C H , et al . Longitudinal speed control of autonomous vehicle based on a self‐adaptive PID of radial basis function neural network [J ] . IET Intelligent Transport Systems , 2018 , 12 ( 6 ): 485 - 494 . DOI: 10.1049/itr2.v12.6 http://doi.org/10.1049/itr2.v12.6 https://onlinelibrary.wiley.com/toc/17519578/12/6 https://onlinelibrary.wiley.com/toc/17519578/12/6

BEAl C E , GERDES J C . Model predictive control for vehicle stabilization at the limits of handling [J ] . IEEE Transactions on Control Systems Technology , 2012 , 21 ( 4 ): 1258 - 1269 . DOI: 10.1109/TCST.2012.2200826 http://doi.org/10.1109/TCST.2012.2200826 http://ieeexplore.ieee.org/document/6226838/ http://ieeexplore.ieee.org/document/6226838/

JI J , KHAJEPOUR A , MELEK W W , et al . Path planning and tracking for vehicle collision avoidance based on model predictive control with multiconstraints [J ] . IEEE Transactions on Vehicular Technology , 2016 , 66 ( 2 ): 952 - 964 . DOI: 10.1109/TVT.2016.2555853 http://doi.org/10.1109/TVT.2016.2555853 http://ieeexplore.ieee.org/document/7458179/ http://ieeexplore.ieee.org/document/7458179/

LINIGER A , DOMAHIDI A , MORARI M . Optimization-based autonomous racing of 1: 43 scale RC cars [J ] . Optimal Control Applications and Methods , 2015 , 36 ( 5 ): 628 - 647 . DOI: 10.1002/oca.v36.5 http://doi.org/10.1002/oca.v36.5 https://onlinelibrary.wiley.com/toc/10991514/36/5 https://onlinelibrary.wiley.com/toc/10991514/36/5

LIAN C Q , XU X , CHEN H , et al . Near-optimal tracking control of mobile robots via receding-horizon dual heuristic programming [J ] . IEEE Transactions on Cybernetics , 2015 , 46 ( 11 ): 2484 - 2496 . DOI: 10.1109/TCYB.2015.2478857 http://doi.org/10.1109/TCYB.2015.2478857 http://ieeexplore.ieee.org/document/7339696/ http://ieeexplore.ieee.org/document/7339696/

ZHANG X L , JIANG Y , LU Y , et al . A receding-horizon reinforcement learning approach for kinodynamic motion planning of autonomous vehicles [J ] . IEEE Transactions on Intelligent Vehicles , 2022 , 7 ( 3 ): 556 - 568 . DOI: 10.1109/TIV.2022.3167271 http://doi.org/10.1109/TIV.2022.3167271 https://ieeexplore.ieee.org/document/9756946/ https://ieeexplore.ieee.org/document/9756946/

龚建伟 , 姜岩 , 徐威 . 无人驾驶车辆模型预测控制 [M ] . 北京 : 北京理工大学出版社 , 2020 .

GONG J W , JIANG Y , XU W . Model predictive control for self-driving vehicles [M ] . Beijing : Beijing Institute of Technology Press , 2020 . (in Chinese)

BERNTORP K . Derivation of a six degrees-of-freedom ground-vehicle model for automotive applications [R ] . Sweden: Department of Automatic Control, Lund University, Technical Report ISRN LUTFD2/TFRT-7627-SE , 2013 .

SNIDER J M . Automatic steering methods for autonomous automobile path tracking [R ] . Pittsburgh,PA,US:Robotics Institute, Technical Report CMU-RITR-09-08 , 2009 .

CHEN P Y . Effects of the entropy weight on TOPSIS [J ] . Expert Systems with Applications , 2021 , 168 : 114186 . DOI: 10.1016/j.eswa.2020.114186 http://doi.org/10.1016/j.eswa.2020.114186 https://linkinghub.elsevier.com/retrieve/pii/S0957417420309209 https://linkinghub.elsevier.com/retrieve/pii/S0957417420309209

FRISON G , DIEHL M . HPIPM:a high-performance quadratic programming framework for model predictive control [J ] . IFAC-PapersOnLine , 2020 , 53 ( 2 ): 6563 - 6569 . DOI: 10.1016/j.ifacol.2020.12.073 http://doi.org/10.1016/j.ifacol.2020.12.073 https://linkinghub.elsevier.com/retrieve/pii/S2405896320303293 https://linkinghub.elsevier.com/retrieve/pii/S2405896320303293

YUE M , FANG C , ZHANG H Z , et al . Adaptive authority allocation-based driver-automation shared control for autonomous vehicles [J ] . Accident Analysis & Prevention , 2021 , 160 : 106301 . DOI: 10.1016/j.aap.2021.106301 http://doi.org/10.1016/j.aap.2021.106301 https://linkinghub.elsevier.com/retrieve/pii/S0001457521003328 https://linkinghub.elsevier.com/retrieve/pii/S0001457521003328

Views

477

下载量

CNKI被引量

Alert me when the article has been cited

提交

Tools

Publicity Resources

A Mobile Robot Path Planning Algorithm Based on Ant Colony Optimization Guide Deep Q-Networks

Hierarchical Decision-making for UAV Air Combat Based on DDQN-D³PG

Model Predictive Control of Unmanned Vehicle Trajectory Tracking Based on Gaussian Process Regression

A Cooperative Guidance Law Based on Meta-learning and Reinforcement Learning for Multiple Aerial Vehicles

AUV Obstacle Avoidance and Path Planning Based on Artificial Potential Field and Improved Reinforcement Learning

Related Author

Hailiang LI

Zonggang LI

Xiaogang NING

Yajiang DU

Yu WANG

Yuanpeng LI

Zhongyu GUO

Shuo LI

Related Institution

Xi'an Kewei Industrial Development Co., Ltd.

School of Automation, Shenyang Aerospace University

State Key Laboratory of Advanced Design and Manufacturing Technology for Vehicle, Human University

National Key Laboratory for Advanced Vehicle Integration and Control, Beijing Institute of Technology

Postal code：100089
Tel：010-68963060/68962718 Fax：010-68963025 Email：bgxb@cos.org.cn
Technical support is provided by Beijing Founder electronics co., LTD 京ICP备05059581号-4 京公网安备11010802024360号
It is recommended to read the content of this site in Chrome&IE9+. Please switch to extreme mode in browser 360.
Cookies We use cookies to help provide and enhance our service and tailor content. By continuing, you agree to the use of cookies.
Total Visits：44491 Visits Today：471

⁰