欢迎访问《兵工学报》官方网站,今天是

兵工学报 ›› 2013, Vol. 34 ›› Issue (5): 627-633.doi: 10.3969/j.issn.1000-1093.2013.05.018

• 研究论文 • 上一篇    下一篇

基于GA-Q-learning 算法的虚拟维修作业规划模型

焦玉民1,2, 王强1, 徐婷1, 谢庆华2, 王海涛1   

  1. 1. 解放军理工大学野战工程学院, 江苏南京210007; 2. 解放军理工大学国防工程学院, 江苏南京210007
  • 上线日期:2013-07-22
  • 作者简介:焦玉民(1984—), 男, 博士研究生。
  • 基金资助:

    江苏省自然科学基金-青年基金项目(BK2012061)

GA-Q-learning Algorithm-based Operation Planning Model for Virtual Maintenance Process

JIAO Yu-min1,2, WANG Qiang1, XU Ting1, XIE Qing-hua2, WANG Hai-tao1   

  1. 1. College of Field Engineering, PLA University of Science and Technology, Nanjing 210007, Jiangsu, China; 2. College of Defense Engineering, PLA University of Science and Technology, Nanjing 210007, Jiangsu, China
  • Online:2013-07-22

摘要:

针对虚拟维修环境中任务执行过程存在的不确定性和随机性问题,提出了一种基于Q 学习算法的作业策略规划模型,该方法将虚拟维修过程转化为选取不同动作参与状态转移的过程。在该过程中,采用试错机制和逆向求解的方法求解动作策略规划问题,并将任务特征匹配机制和顺序约束机制作为启发机制,保证策略学习过程中持续进化可行策略;在进化过程中,将动作因子赋予概率值,并采用遗传算法(GA)进化动作因子的概率分布,避免了策略学习过程中强化早期Q 值较高的动作,为求解虚拟维修的最佳作业流程提供了一种行之有效的解决方法。将该方法应用于轮式挖掘机虚拟维修训练系统中,仿真结果表明,正确的动作在作业策略迭代过程中均能够获得较高的Q 值,验证了方法的可行性和实用性。

关键词: 人工智能, 虚拟维修, Q 学习, 遗传算法, 作业规划

Abstract:

To solve the uncertainty and randomization problems which happen in virtual maintenance process, a novel operation strategy planning model based on Q-learning algorithm is presented. The virtu- al maintenance process is transformed into a state transition process by using various actions. Correcting mechanism and inverse solution are used to solve task planning problem. To guarantee revolting continu- ously evolving feasible strategy, the characteristics matching mechanism and sequence constraint mecha- nism are proposed to aid in finding the optimal strategy. In the evolution process, a genetic algorithm is used to adjust the probability distribution of action value to avoid reinforcing early action with high Q-val- ue. Finally, an operation strategy optimal example for the virtual maintenance system is given to show that right action always can receive high Q-value in the evolution, which illustrates the feasibility and ap- plicability of the proposed methodology.

Key words: artificial intelligence, virtual maintenance, Q-learning, genetic algorithm, operation plan- ning

中图分类号: