基于脑机接口与人机闭环的远程脑控无人机系统

doi:10.12382/bgxb.2023.0866

摘要/Abstract

摘要：

随着现代军事战争的迅速演变,远程脑控无人机在实现战场信息获取、目标监视和战术部署方面扮演着愈发重要的角色。提出一种应用于远程脑控无人机的压缩感知控制范式和人机闭环控制算法,基于该控制范式及控制算法搭建面向军事应用场景的远程脑控无人机系统。在线实验结果表明:8名被试人员通过该脑控无人机系统进行导航任务,平均任务完成率为0.95,平均任务完成时间为100.46s,显著优于基于人机开环控制算法的脑控无人机系统;新提出的脑控无人机系统可以应用于军事场景下的战场侦察,大幅度提高作战人员的无人机远程控制能力,拓展作战人员的战场感知范围。

关键词: 脑控无人机, 脑机接口, 人机闭环, 压缩感知

Abstract:

With the rapid evolution of modern military warfare, the remote brain-controlled unmanned aerial vehicle (UAV) systems are playing an increasingly important role in battlefield information gathering, target surveillance, and tactical deployment. This research proposes a compressed sensing control paradigm and a human-machine closed-loop control algorithm for remote brain-controlled UAV. Based on this control paradigm and algorithm, a remote brain-controlled UAV system for military applications is constructed. Online experiments conducted in this study demonstrate that eight participants successfully completed the navigation tasks using the brain-controlled UAV system based on the compressed sensing control paradigm and human-machine closed-loop control algorithm. The average task completion rate of the proposed brain-controlled UAV system is 0.95, and its average task completion time is 100.46s, which significantly outperformes the brain-controlled UAV system based on human-machine open-loop control algorithms. In the future, the proposed brain-controlled UAV system can be used for battlefield reconnaissance in military scenarios, significantly enhancing the remote-control capabilities of military personnel and expanding their battlefield awareness.

Key words: brain-controlled unmanned aerial vehicle, brain-machine interface, human-machine closed-loop, compressed sensing

中图分类号:

TJ85

刘思宇, 张德雨, 明致远, 刘梦真, 刘紫玉, 陈启明, 张健, 吴景龙, 闫天翼. 基于脑机接口与人机闭环的远程脑控无人机系统[J]. 兵工学报, 2024, 45(9): 3191-3203.

LIU Siyu, ZHANG Deyu, MING Zhiyuan, LIU Mengzhen, LIU Ziyu, CHEN Qiming, ZHANG Jian, WU Jinglong, YAN Tianyi. Remote Brain-controlled Unmanned Aerial Vehicle System Based on Brain-machine Interface and Human-machine Closed Loop[J]. Acta Armamentarii, 2024, 45(9): 3191-3203.

图/表 20

图1 传统SSVEP控制范式的示意图

Fig.1 Schematic diagram of traditional SSVEP control paradigm

图2 压缩感知控制范式

Fig.2 Compressed sensing control paradigm

图3 压缩感知控制范式的实现流程

Fig.3 Realization process of compressed sensing control paradigm

图4 闪烁刺激诱发下相应频段的脑电能量值显著升高(以8.0Hz闪烁刺激为例)

Fig.4 Significant increase in flash stimulation-induced energy value of corresponding frequency band (taking 8.0Hz flash stimulation as an example)

图5 无人机靠近敌机时产生逆向速度矢量的示意图

Fig.5 Schematic diagram of reverse speed vector when a UAV is close to the enemy aircraft

图6 原始无人机控制指令生成方法示意图

Fig.6 Schematic diagram of the original UAV control instruction generation method

图7 PID控制的控制流程

Fig.7 Control process of PID

图8 脑控无人机实验场景

Fig.8 Brain-controlled UAV experiment scenarios

图9 传统SSVEP控制范式

Fig.9 Schematic diagram of traditional SSVEP control paradigm

图10 实验流程图

Fig.10 Experimental flowchart

图11 被试人员关注8.0Hz的闪烁刺激时两种控制范式下所诱发大脑信号的功率图

Fig.11 The power chart of brain signal induced by the two control paradigms when the subjects pay attention to the flashes of 8.0Hz

图12 两种控制范式下的平均分类准确率

Fig.12 The average classification accuracy under the two control paradigms

图13 两种控制范式下的平均ITR

Fig.13 The average ITR under two control paradigms

图14 分析时长为2.0s,被试的平均分类准确率和ITR (*表示经过t检验后发现具有显著性差异:P<0.05)

Fig.14 The average classification accuracy and ITR of the subjects with analysis duration of 2.0s (* denotes significant differences found aftert-tests: P<0.05)

图15 被试完成3种实验任务的任务成功率(*:P<0.05)

Fig.15 The success rate of the task of completing the three experimental tasks (*:P<0.05)

图16 被试完成3种实验任务的任务完成时间(*:P<0.05)

Fig.16 The test completion time of completing the three experimental tasks (*:P<0.05)

图17 两种控制算法下敌机数量与远程无人机任务完成率之间的关系

Fig.17 The relationship between the number of enemy aircraft and the task completion rate of remote UAV under the two control algorithms

图18 两种控制算法下远程无人机任务完成率的均值 (*:P<0.05)

Fig.18 Average value of task completion rate of remote UAV under the two control algorithms(*:P< 0.05)

图19 两种控制算法下敌机数量与远程无人机任务完成时间之间的关系

Fig.19 The relationship between the number of enemy aircraft and the task completion time of remote UAV under the two control algorithms

图20 两种控制算法下远程无人机任务完成时间的均值(*:P<0.05)

Fig.20 Average value of task completion time of remote UAV under the two control algorithms (*:P<0.05)

参考文献 23

[1]	陈慧岩, 张玉. 军用地面无人机动平台技术发展综述[J]. 兵工学报, 2014, 35(10):1696-1706. doi: 10.3969/j.issn.1000-1093.2014.10.026
	CHEN H Y, ZHANG Y. An overview of research on military unmanned ground vehicles[J]. Acta Armamentarii, 2014, 35(10):1696-1706. (in Chinese)
[2]	张楠, 王潇, 王军力. 无人机在陆军精确打击体系中的应用研究[J]. 兵工学报, 2010, 31(增刊2):107-111.
	ZHANG N, WANG X, WANG J L. An overview of research on military unmanned ground vehicles[J]. Acta Armamentarii, 2010, 31(S2):107-111. (in Chinese)
[3]	LIU S Y, ZHANG D Y, QIAO M, et al. Mind controlled vehicle based on lidar SLAM navigation and SSVEP technology[C]//Proceedings of the 2021 9th International Winter Conference on Brain-Computer Interface. Gangwon, Korea:IEEE, 2021:1-4.
[4]	ZHAO J, LI W, LI M F. Comparative study of SSVEP-and P300-based models for the telepresence control of humanoid robots[J]. PLoS One, 2015, 10(11): e0142168.
[5]	GUO N, WANG X J, DUANMU D H, et al. SSVEP-based brain computer interface controlled soft robotic glove for post-stroke hand function rehabilitation[J]. IEEE Transactions on Neural Systems and Rehabilitation Engineering, 2022, 30:1737-1744.
[6]	ZHANG W C, TAN C Q, SUN F C, et al. A review of EEG-based brain-computer interface systems design[J]. Brain Science Advances, 2018, 4(2):156-167.
[7]	DEHZANGI O, FAROOQ M. Portable brain-computer interface for the intensive care unit patient communication using subject-dependent SSVEP ide.pngication[J]. BioMed Research International, 2018:1-14.
[8]	CHEN X G, WANG Y J, NAKANISHI M, et al. High-speed spelling with a noninvasive brain-computer interface[C]//Proceedings of the National Academy of Sciences of the United States of America, 2015, 112(44):E6058-E6067.DOI:10.1073/pnas.1508080112.
[9]	NAKANISHI M, WANG Y J, CHEN X G, et al. Enhancing detection of SSVEPs for a high-speed brain speller using task-related component analysis[J]. IEEE Transactions on Biomedical Engineering, 2018, 65(1):104-112.
[10]	LIU B C, CHEN X G, SHI N L, et al. Improving the performance of individually calibrated SSVEP-BCI by task-discriminant component analysis[J]. IEEE Transactions on Neural Systems and Rehabilitation Engineering, 2021, 29:1998-2007.
[11]	CHEN X G, LIU B C, WANG Y J, et al. A spectrally-dense encoding method for designing a high-speed SSVEP-BCI with 120 stimuli[J]. IEEE Transactions on Neural Systems and Rehabilitation Engineering, 2022, 30:2764-2772. doi: 10.1109/TNSRE.2022.3208717 pmid: 36136927
[12]	李家伟, 张洪欣, 徐瑞林. 基于脑机接口的无人机编队控制系统设计[J]. 航空科学技术, 2023, 34(2):104-110.
	LI J W, ZHANG H X, XU R L. Design of UAV formation control system based on brain-computer interface data[J]. Aeronautical Science & Technology, 2023, 34(2):104-110. (in Chinese)
[13]	刘亚群. 多模态脑控无人机地面站软件系统设计与实现[J]. 现代工程科技, 2023, 2(6):45-48.
	LIU Y Q. Design and implementation of multimodal brain-controlled aerial vehicle ground station software system modern[J]. Engineering Technology, 2023, 2(6):45-48. (in Chinese)
[14]	HIRECHE A, ZENNAIA Y, AYAD R, et al. A decoding algorithm for non-invasive SSVEP-based drone flight control[C]//Proceedings of the 2021 IEEE International Conference on Bioinformatics and Biomedicine. Houston, TX, US:IEEE, 2021:3616-3623.
[15]	CHIUZBAIAN A, JAKOBSEN J, PUTHUSSERYPADY S. Mind controlled drone:an innovative multiclass SSVEP based brain computer interface[C]//Proceedings of the 2019 7th International Winter Conference on Brain-Computer Interface. Gangwon, Korea:IEEE, 2019:1-5.
[16]	ZHANG D Y, LIU S Y, ZHANG J, et al. Brain-controlled 2D navigation robot based on a spatial gradient controller and predictive environmental coordinator[J]. IEEE Journal of Biomedical and Health Informatics, 2022, 26(12):6138-6149.
[17]	CHEN Y H, YANG C, CHEN X G, et al. A novel training-free recognition method for SSVEP-based BCIs using dynamic window strategy[J]. Journal of Neural Engineering, 2021, 18(3):036007.
[18]	YIN E W, ZHOU Z T, JIANG J, et al. A dynamically optimized SSVEP brain-computer interface(BCI) speller[J]. IEEE Transactions on Biomedical Engineering, 2014, 62(6):1447-1456.
[19]	KALUNGA E K, CHEVALLIER S, et al. Online SSVEP-based BCI using riemannian geometry[J]. Neurocomputing, 2016, 191:55-68.
[20]	LAFLEUR K, CASSADY K, DOUD A, et al. Quadcopter control in three-dimensional space using a noninvasive motor imagery-based brain-computer interface[J]. Journal of Neural Engineering, 2013, 10(4):046003.
[21]	KIM S, LEE S, KANG H, et al. P300 brain-computer interface-based drone control in virtual and augmented reality[J]. Sensors, 2021, 21(17):5765.
[22]	PASZKIEL S, SIKORA M. The use of brain-computer interface to control unmanned aerial vehicle[C]//Proceedings of the Automation 2019:Progress in Automation, Robotics and Measurement Techniques. Warsaw, Poland: Springer, 2020: 583-598.
[23]	DMYTRIYEV Y, INSERO F, CARNEVALE M, et al. Brain-computer interface and hand-guiding control in a human-robot collaborative assembly task[J]. Machines, 2022, 10(8):654.