基于部分可观察马尔可夫决策过程的多被动传感器组网协同反隐身探测任务规划

doi:10.3969/j.issn.1000-1093.2015.04.023

摘要/Abstract

摘要： 针对反隐身作战需求，提出多被动传感器组网协同战术。为提升反隐身探测效能，引入部分可观察马尔可夫决策过程（POMDP）理论，分析了POMDP任务规划要素，建立起多被动传感器组网协同反隐身探测任务规划POMDP模型。建立了多被动传感器协同控制系统结构，提出了基于无迹卡尔曼滤波（UKF）的信念状态更新方法和基于蒙特卡洛Rollout采样（MCRS）的Q值估计方法，并设计了CCSP基本策略。仿真结果表明，所建立的模型能够实现多被动传感器的高效管理调度，能够控制多被动传感器对隐身目标进行有效探测跟踪，即模型有效性得到了验证。

关键词: 控制科学与技术, 传感器技术, 反隐身, 多传感器组网, 部分可观察马尔可夫决策过程, 信念状态, 任务规划

Abstract: A cooperative strategy for multiple passive networked sensors to satisfy the requirements of anti-stealth is proposed. A POMDP model for the multiple passive networked sensors is established to improve the effectiveness of the anti-stealth detection. The elements of POMDP model are analyzed comprehensively based on the POMDP theory. A cooperative control system of multi-passive-sensors is put forward, and a UKF method for belief state updating together with a MCRS-based (Monte-Carlo Rollout sampling) algorithm for Q-value estimation is presented. A basic policy of CCSP is designed. Experiments show that the proposed model can be used for the efficient management and scheduling of multi-passive-sensors, as well as the effective detection and tracking of the stealth targets by deploying these sensors. The validity of the proposed algorithm is demonstrated.

Key words: control science and technology, sensor technology, anti-stealth, multiple sensors networking, POMDP, belief state, mission planning

中图分类号:

TP212

万开方，高晓光，李波，梅军峰. 基于部分可观察马尔可夫决策过程的多被动传感器组网协同反隐身探测任务规划[J]. 兵工学报, 2015, 36(4): 731-743.

WAN Kai-fang, GAO Xiao-guang, LI Bo, MEI Jun-feng. Mission Planning of Passive Networked Sensors for Cooperative Anti-stealth Detection Based on POMDP[J]. Acta Armamentarii, 2015, 36(4): 731-743.

参考文献

［1］卢盈齐, 祝长英. 雷达组网反隐身的一种优化布站方法［J］. 系统工程理论与实践,2007(6): 166-169.
LU Ying-qi, ZHU Chang-ying. An optimal deploy method of netted radar for detecting stealth target［J］. Systems Engineering-Theory & Practice, 2007(6): 166-169. (in Chinese)
［2］刘尚富, 甘怀锦. 雷达隐身与反隐身技术浅析［J］. 舰船电子工程,2010, 30(9): 28-30.
LIU Shang-fu, GAN Huai-jin. Brief review on stealth and anti-stealth techniques of radar［J］. Ship Electronic Engineering, 2010, 30(9): 28-30. (in Chinese)
［3］凌晓曙. 雷达隐身和反隐身技术［J］. 舰船电子对抗, 2007, 30(3): 40-42.
LING Xiao-shu. Stealth and anti-stealth techniques of radar［J］. Shipboard Electronic Countermeasure, 2007, 30(3): 40-42. (in Chinese)
［4］沈阳, 陈永光, 李修和,等. 多基地雷达反隐身分布式检测融合算法研究［J］. 电子学报, 2007, 35(3): 506-510.
SHEN Yang, CHEN Yong-guang, LI Xiu-he,et al. Study on fusion arithmetic of multi radar distributed detection system against stealthy targets ［J］. Acta Electronica Sinica, 2007,35(3): 506-510. (in Chinese)
［5］ Kuschel H, Heckenbach J, Muller S, et al. On the potentials of passive, multistatic, low frequency radars to counter stealth and detect low flying targets［C］∥2008 IEEE Radar Conference. Rome, Italy : IEEE Computer Society, 2008: 1-6.
［6］李伟, 柯涛. 雷达组网反隐身可行性仿真分析［J］. 舰船电子对抗,2010(5): 83-87.
LI Wei, KE Tao. Simulation analysis of anti-stealth feasibility of radar netting［J］. Shipboard Electronic Countermeasure,2010(5): 83-87. (in Chinese)
［7］ Kreucher C, Blatt D, Hero A, et al. Adaptive multi-modality sensor scheduling for detection and tracking of smart targets［J］. Digital Signal Processing, 2006, 16(5): 546-567.
［8］刘先省, 申石磊, 潘泉. 传感器管理及方法综述［J］. 电子学报,2002,30(3): 394-398.
LIU Xian-xing , SHEN Shi-lei, PAN Quan. A survey of sensor management and methods［J］. Acta Electronica Sinica, 2002,30(3): 394-398. (in Chinese)
［9］罗开平, 姜维, 李一军. 传感器管理述评［J］. 电子学报,2010,38(8): 1900-1907.
LUO Kai-ping, JIANG Wei, LI Yi-jun. Review of sensor management［J］. Acta Electronica Sinica, 2010,38(8): 1900-1907. (in Chinese)
［10］李彬彬, 冯新喜, 王朝英，等. 基于信息增量的多被动传感器资源分配算法［J］. 系统工程与电子技术, 2012(3): 502- 507.
LI Bin-bin, FENG Xin-xi, WANG Chao-ying, et al. Multi-passive sensors resource allocation algorithm based on information gain［J］. Systems Engineering and Electronics, 2012(3): 502-507. (in Chinese)
［11］ Williams J L. Information theoretic sensor management［D］. Massachusetts: Massachusetts Institute of Technology, 2007.
［12］ Jenkins K L, Castanon D A. Information-based adaptive sensor management for sensor networks［C］∥2011 American Control Conference. San Francisco, CA, US: AACC, 2011:4934-4940.
［13］ Wei M, Chen G, Blasch E. Game theoretic multiple mobile sensor management under adversarial environments［C］∥11th International Conference on Information Fusion Cologne. Germany: Air Force Research Laboratory, 2008:645-652.
［14］ Li X, Chen G, Blasch E. A geometric feature-aided game theoretic approach to sensor management［C］∥12th International Conference on Information Fusion. Seattle, WA, US: ISIF, 2009:1155-1162.
［15］ Lopez J M M, Rodriguez F J J, Corredera J R C. Fuzzy reasoning for multisensor management［C］∥IEEE International Conference on SMC. US: IEEE, 1995: 1398-1403.
［16］ Smith J F, Rhyne R D. A fuzzy logic algorithm for optimal allocation of distributed resources［C］∥Proceedings of the Second International Conference on Information Fusion. Mountain View, CA: International Society for Infonmation Fusion, 1999: 402-409.
［17］王博, 周一宇, 鲁建华，等. 基于实值粒子群优化的STSS系统传感器管理算法研究［J］. 系统仿真学报,2009,21(22): 7287-7292.
WANG Bo, ZHOU Yi-yu, LU Jian-hua, et al. Research on sensor management algorithm of STSS based on real-number particle swarm optimization［J］. Journal of System Simulation, 2009,21(22): 7287-7292. (in Chinese)
［18］杨博, 王向华, 邵利平,等. 基于群集智能的传感器管理方法研究［J］. 兵工学报，2012, 33(2): 155-161.
YANG Bo, WANG Xiang-hua, SHAO Li-ping, et al. Research on sensor management based on collective intelligence［J］. Acta Armamentarii, 2012, 33(2): 155-161. (in Chinese)
［19］ Williams J L, Fisher J W, Willsky A S. Approximate dynamic programming for communication-constrained sensor network management［J］. IEEE Transactions on Signal Processing, 2007, 55(8): 4300-4311.
［20］ Karmokar A K, Senthuran S, Anpalagan A. POMDP-based cross-layer power adaptation techniques in cognitive radio networks［C］∥Global Communications Conference. Anaheim, California, US：IEEE，2012: 1380-1385.
［21］ Hitchings D, Castanon D A. Receding horizon stochastic control algorithms for sensor management［C］∥American Control Conference. MD, US: AACC, 2010:6809-6815.
［22］ Krishnamurthy V. Algorithms for optimal scheduling and management of hidden Markov model sensors［J］. IEEE Transactions on Signal Processing, 2002, 50(6): 1382-1397.
［23］ Brehard T, Coquelin P A, Duflos E, et al. Optimal policies search for sensor management: application to the AESA radar［C］∥11th International Conference on Information Fusion. Cologne, Germany: Cologne, Germany: International Society for Information Fusion, 2008: 1-8.
［24］ Krishnamurthy V, Djonin D V. Optimal threshold policies for multivariate POMDPs in radar resource Management［J］. IEEE Transactions on Signal Processing, 2009, 57(10): 3954-3969.
［25］ Li Y, Krakow L W, Chong E K, et al. Approximate stochastic dynamic programming for sensor scheduling to track multiple targets［J］. Digital Signal Processing,2009, 19(6): 978-989.
［26］ He Y, Chong E K. Sensor scheduling for target tracking: a Monte Carlo sampling approach［J］. Digital Signal Processing, 2006, 16(5): 533-545.
［27］ Nourbakhsh I, Powers R, Birchfield S. DERVISH an office-navigating robot［J］. AI Magazine, 1995, 16(2): 53-60.
［28］ Simmons R, Koenig S. Probabilistic robot navigation in partially observable environments［C］∥Proceedings of the International Joint Conference on Artificial Intelligence.Canberra, Australia:World Scientific Publishing Co Pte Ltd, 1995.
［29］ Dallaire P, Besse C, Ross S, et al. Bayesian reinforcement learning in continuous POMDPs with Gaussian processes［C］∥International Conference on Intelligent Robots and Systems. St Louis, MO, US：IEEE, 2009: 2604-2609.
［30］ Martinez-Cantin R, De Freitas N, Brochu E, et al. A Bayesian exploration-exploitation approach for optimal online sensing and planning with a visually guided mobile robot［J］. Autonomous Robots,2009, 27(2): 93-103.
［31］ Pyeatt L D, Howe A E. Integrating POMDP and reinforcement learning for a two layer simulated robot architecture［C］∥The Third Annual Conference on Autonomous Agents. New York, US: ACM, 1999: 168-174.
［32］ Eker B I C S, Ak I N H L. Solving decentralized POMDP problems using genetic algorithms［J］. Autonomous Agents and Multi-Agent Systems, 2013, 27(1): 161-196.
［33］ Chong E K P, Kreucher C M, Hero A O. Monte-Carlo-based partially observable Markov decision process approximations for adaptive sensing［C］∥9th International Workshop on Discrete Event Systems . Goteborg, Sweden:IEEE,2008: 173-180.
［34］ Chong E K, Kreucher C M, Hero Iii A O. Foundations and Applications of Sensor Management［M］. NY: Springer, 2008: 95- 119.
［35］ Li Y, Krakow L W, Chong E K P, et al. Dynamic sensor management for multisensor multitarget tracking［C］∥40th Annual Conference on Information Sciences and Systems. Princeton, NJ: IEEE,2006: 1397-1402
［36］慈元卓. 面向移动目标搜索的多星任务规划问题研究［D］. 长沙：国防科学技术大学, 2008.
CI Yuan-zhuo. Multi-satellite mission planning for moveing target search［D］. Changsha: National University of Defence Technology,2008. (in Chinese)
［37］ Tharmarasa R, Kirubarajan T. Sensor management for large-scale multisensor-multitarget tracking ［D］. Canada: McMaster University,2007.
［38］ Bertsekas D P, Castanon D A. Rollout algorithms for stochastic scheduling problems［J］. Journal of Heuristics, 1999,5(1): 89-108.
［39］ Guy Shani. POMDP solver-a Java implementation arranged as an Eclipse package of most of the point-based algorithms for solving POMDPs ［EB/OL］. ［2013-06-01］. http:∥www.bgu.ac.il/~ shanigu/.

[1]	杜伟伟，陈小伟. 作战任务层次化分解方法[J]. 兵工学报, 2021, 42(12): 2771-2782.
[2]	刘昊，张策，丁文韬. 基于智能对抗进化的联合火力打击任务规划方法[J]. 兵工学报, 2019, 40(6): 1287-1296.
[3]	苗建明，王少萍，范磊，李元. 欠驱动自主水下航行器空间曲线路径跟踪控制研究[J]. 兵工学报, 2017, 38(9): 1786-1796.
[4]	查峰，覃方君，李京书，叶斌. 三轴旋转惯性导航系统的旋转控制建模研究[J]. 兵工学报, 2017, 38(8): 1610-1618.
[5]	伍明，张国良，李琳琳，付光远，李承剑. 基于动态和静态环境对象观测一致性约束的移动机器人多传感器标定优化方法[J]. 兵工学报, 2017, 38(8): 1630-1641.
[6]	王世峰，都凯悦，孟颖，王锐. 基于机器学习的车辆路面类型识别技术研究[J]. 兵工学报, 2017, 38(8): 1642-1648.
[7]	周荻，董金鲁. 带襟翼的飞行器非线性控制系统分析和设计[J]. 兵工学报, 2017, 38(7): 1322-1329.
[8]	郑睿，赵伟，方明星，杜友武. 基于多圈环绕结构的超流体陀螺噪声抑制方法[J]. 兵工学报, 2017, 38(7): 1330-1335.
[9]	邢丽, 熊智, 刘建业, 杭义军. 基于对偶四元数的圆锥及划船误差补偿改进算法[J]. 兵工学报, 2017, 38(7): 1336-1347.
[10]	庞辉, 陈嘉楠, 刘凯. 汽车磁流变半主动悬架系统自适应反推跟踪控制[J]. 兵工学报, 2017, 38(7): 1430-1442.
[11]	马艳彤，郑荣，韩晓军. 面向海底光学探测使命的自治水下机器人水平路径跟随控制[J]. 兵工学报, 2017, 38(6): 1147-1153.
[12]	陈慧岩，陈舒平，龚建伟. 智能汽车横向控制方法研究综述[J]. 兵工学报, 2017, 38(6): 1203-1214.
[13]	廖自力，阳贵兵，高强，袁东. 多轮独立电驱动车辆转向稳定性集成控制研究[J]. 兵工学报, 2017, 38(5): 833-842.
[14]	王斌锐，沈国阳，金英连，王凌. 基于干扰观测器的级联气动肌肉肘关节滑模控制[J]. 兵工学报, 2017, 38(4): 793-801.
[15]	李阁强，顾永升，李健，李跃松，郭冰菁. 被动式电液力伺服系统的自适应反步滑模控制[J]. 兵工学报, 2017, 38(3): 616-624.