A Review of Research on Cognitive Fatigue in Intelligent Equipment

doi:10.12382/bgxb.2024.0546

Abstract

Abstract:

The soldiers’ combat environments and equipment are undergoing unprecedented changes with the evolution of modern warfare and the rapid development of intelligent technologies. Intelligent equipment, such as augmented reality helmets, wearable devices, and remote operation systems, has become the key to enhancing the soldiers’ combat effectiveness. However, the integration and use of these technologies also bring new challenges to soldiers, especially in terms of fatigue. Fatigue affects the decision-making ability, reaction time and attention concentration of soldiers, thereby impacting battlefield performance and task execution efficiency. The current state of research on cognitive fatigue is reviewed, and the causes and impacts of cognitive fatigue as well as monitoring technologies and mitigation strategies are explored. Through the analysis of existing literature, this article aims to provide a reference for future research on cognitive fatigue issues arising from the interaction between soldiers and intelligent equipment. It also proposes suggestions for the military training and the design of intelligent equipment to alleviate cognitive fatigue and enhance soldiers’ combat capabilities.

Key words: combat effectiveness, intelligent equipment, cognitive ergonomics, monitoring and evaluation

CLC Number:

E257

WANG Duo, LIU Qi, ZHANG Bin, ZHANG Cheng, FAN Changcun. A Review of Research on Cognitive Fatigue in Intelligent Equipment[J]. Acta Armamentarii, 2024, 45(S1): 209-218.

Figures/Tables 5

References 51

[1]	单时卓, 裴天翼, 刘泽轩. 基于人机混合智能的协同作战研究[J]. 计算机测量与控制, 2024, 32(5):296-301.
	SHAN S Z, PEI T Y, LIU Z X. Research on cooperative combat based on man-machine combination intelligence[J]. Computer Measurement & Control, 2024, 32(5):296-301. (in Chinese)
[2]	张荣, 梁馨月. 航空人员疲劳检测方法研究综述[J/OL]. 计算机工程与应用, 2024,(2024-07-08)[2024-07-09]. http://kns.cnki.net/kcms/detail/11.2127.TP.20240705.1348.008.html.
	ZHANG R, LIANG X Y. Review of research on fatique detection methods for aviation personnel[J/OL]. Computer Engineering and Applications, 2024,(2024-07-08)[2024-07-09]. http://kns.cnki.net/kcms/detail/11.2127.TP.20240705.1348.008.html. (in Chinese)
[3]	宋明箫, 范利君, 陈学伟, 等. 军事作业疲劳现场监测及评价技术研究进展[J]. 军事医学, 2024, 48(2):143-147. doi: 10.7644/j.issn.1674-9960.2024.02.010
	SONG M X, FAN L J, CHEN X W, et al. Research progress in technologies for on-site monitoring and evaluation of fatigue during military operations[J]. Military Medcal Sciences, 2024, 48(2):143-147. (in Chinese)
[4]	ISMAIL L E, KARWOWSKI W. Applications of EEG indices for the quantification of human cognitive performance: a systematic review and bibliometric analysis[J]. PLoS One, 2020, 15(12): e0242857.
[5]	KAKKOS I, DIMITRAKOPOULOS G N, GAO L, et al. Mental workload drives different reorganizations of functional cortical connectivity between 2D and 3D simulated flight experiments[J]. IEEE Transactions on Neural Systems and Rehabilitation Engineering, 2019, 27(9):1704-1713. doi: 10.1109/TNSRE.2019.2930082 pmid: 31329123
[6]	SUN J C, CHENG S, MA J, et al. Assessment of the static upright balance index and brain blood oxygen levels as parameters to evaluate pilot workload[J]. PLoS One, 2019, 14(3): e0214277..
[7]	孙瑞山, 王平. 基于结构方程模型的飞行员疲劳影响因素研究[J]. 安全与环境学报, 2022, 22(6):3252-3258.
	SUN R S, WANG P. Research on the factors influencing pilot fatigue based on structural equation modeling[J]. Journal of Safety and Environment, 2022, 22(6):3252-3258. (in Chinese)
[8]	杨菁华, 程珊, 孙继成, 等. 飞行任务负荷下疲劳的监测及干预措施浅析[J]. 空军军医大学学报, 2022, 43(1):82-85.
	YANG J H, CHENG S, SUN J C, et al. Analysis of flight fatigue monitoring and intervention measures under task load[J]. Air Force Med Univ, 2022, 43(1):82-85. (in Chinese)
[9]	KECKLUND G, AKERSTEDT T. Sleepiness in long distance truck driving: an ambulatory EEG study of night driving[J]. Ergonomics, 1993, 36(9):1007-1017. pmid: 8404830
[10]	唐帮备, 李衍, 王盛学, 等. 驾驶疲劳在人机共驾环境中的检测与缓解研究综述[J]. 汽车文摘, 2024(8):9-16.
	TANG B B, LI Y, WANG S X, et al. A review of driver fatigue detection and mitigation in man-machine co-driving environment[J]. Automotive Digest, 2024(8):9-16. (in Chinese)
[11]	杨柳, 何萌, 刘清. 船员工作负荷检测方法研究综述[J]. 交通信息与安全, 2021, 39(3):1-7,16.
	YANG L, HE M, LIU Q. A review on the methods of detecting workload of seafarers[J]. Journal of Transport Information and Safety, 2021, 39(3):1-7,16. (in Chinese)
[12]	刘伟, 吴玲, 朱诗慧, 等. LCT标准试验环境下的驾驶次任务试验与绩效指标研究[J]. 中国公路学报, 2015, 28(6):104-111.
	LIU W, WU L, ZHU S H, et al. Research on secondary task experiment and driving performance indicator under standard lane change test[J]. China Journal of Highwag and Transport, 2015, 28(6):104-111. (in Chinese)
[13]	王宥霖, 马如梦, 李晓京, 等. 人机功能分配对于长航时飞行负荷评价的初步研究[J]. 转化医学电子杂志, 2016, 3(2):84-88.
	WANG Y L, MA R M, LI X J, et al. Preliminary study on the evaluation of long endurance flight load through human machine function allocation[J]. E-Journal of Translational Medicine, 2016, 3(2):84-88. (in Chinese)
[14]	NAEERI S, MANDAL S, KANG Z. Analyzing pilots’ fatigue for prolonged flight missions: multimodal analysis approach using vigilance test and eye tracking[J]. Proceedings of the Human Factors and Ergonomics Society Annual Meeting, 2019, 63(1): 111-115.
[15]	MARQUART G, WINTER J D. Workload assessment for mental arithmetic tasks using the task-evoked pupillary response[J]. Peer J Computer Science, 2015, 1(8):e16.
[16]	PIGNONI G, KOMANDUR S. Development of a quantitative evaluation tool of cognitive workload in field studies through eye tracking[J]. Engineering Psychology and Cognitive Ergonomics, 2019(11571):106-122.
[17]	PIGNONI G, HAREIDE O S, KOMANDUR S, et al. Trial application of pupillometry for a maritime usability study in field conditions[J]. Necesse, 2019, 4(1):130-146.
[18]	YAN S Y, WEI Y Y, TRAN C C, et al. Evaluation and prediction mental workload in user interface of maritime operations using eye response[J]. International Journal of Industrial Ergonomics, 2019(71):117-127.
[19]	李坤宸, 曹龙. 驾驶分心检测方法综述[J]. 汽车实用技术, 2021, 46(2):102-104.
	LI K C, CAO L. Review of methods for detection of distracted driving[J]. Automobile Applied Technology, 2021, 46(2):102-104. (in Chinese)
[20]	WU X, WANYAN X R, ZHUANG D M. Pilot’s visual attention allocation modeling under fatigue[J]. Technology & Health Care, 2015, 23(S2):S373-81..
[21]	DI STASI L L, MCCAMY M B, MARTINEZ-CONDE S, et al. Effects of long and short simulated flights on the saccadic eye movement velocity of aviators[J]. Physiology & Behavior, 2016, 153:91-96.
[22]	王禹, 肖毅, 周前祥, 等. 基于脑电信号的脑力负荷监测技术研究现状[J]. 航天医学与医学工程, 2018, 31(5):577-582.
	WANG Y, XIAO Y, ZHOU Q X, et al. Research progress of mental workload monitoring technology based on EEG[J]. Space Medicine & Medical Engineering, 2018, 31(5):577-582. (in Chinese)
[23]	迟健男, 张国胜, 刘琳娜. 驾驶疲劳监测方法综述[J]. 交通节能与环保, 2015, 11(4):84-89.
	CHI J N, ZHANG G S, LIU L N. A survey of driving fatigue monitoring[J]. Energy Conservation & Environmental Protection in Transportation, 2015, 11(4):84-89. (in Chinese)
[24]	沈卓, 单立新, 李晔, 等. 执行功能预测手机分心情境下的驾驶警觉反应时和脑电分析[J]. 人类工效学, 2023, 29(6):29-39.
	SHEN Z, SHAN L X, LI Y, et al. The executive function predict reaction time and EEG of driving vigilance in mobile phone distraction context[J]. Chinese Journal of Ergonomics, 2023, 29(6):29-39. (in Chinese)
[25]	杜盼盼. 基于脑电信号的飞行员疲劳状态检测[D]. 郑州: 中原工学院, 2023.
	DU P P. Pilot fatigue state detection based on EEG[D]. Zhengzhou: Zhongyuan University of Technology, 2023. (in Chinese)
[26]	OKOGBAA O G; SHELL R L, FILIPUSIC D. On the investigation of the neurophysiological correlates of knowledge worker mental fatigue using the EEG signal[J]. Applied Ergonomics, 1994, 25(6):355-365. pmid: 15676987
[27]	CRAIG A, TRAN Y, WIJESURIYA N, et al. Regional brain wave activity changes associated with fatigue[J]. Psychophysiology, 2012, 49(4):574-582. doi: 10.1111/j.1469-8986.2011.01329.x pmid: 22324302
[28]	ZHAO C L, ZHAO M, LIU J P, et al. Electroencephalogram and electrocardiograph assessment of mental fatigue in a driving simulator[J]. Accident Analysis & Prevention, 2012.
[29]	SIMON M, SCHMIDT E A, KINCSES W E, et al. EEG alpha spindle measures as indicators of driver fatigue under real traffic conditions[J]. Clinical Neurophysiology, 2011, 122(6):1168-1178. doi: 10.1016/j.clinph.2010.10.044 pmid: 21333592
[30]	SAUVET, BOUGARD, COROENNE, et al. In-flight automatic detection of vigilance states using a single EEG channel[J]. IEEE Transactions on Biomedical Engineering, 2014, 61(12): 2840-2847.
[31]	VERDIÈRE K J, ROY R N, DEHAIS F. Detecting pilot’s engagement using fNIRS connectivity features in an automated vs. manual landing scenario[J]. Frontiers in Human Neuroscience, 2018, 12: 6.
[32]	HASAN A, BANU O, KURTULUS I, et al. Continuous monitoring of brain dynamics with functional near infrared spectroscopy as a tool for neuroergonomic research: empirical examples and a technological development[J]. Frontiers in Human Neuroscience, 2013, 7(871):871.
[33]	DEROSIÈRE G, MANDRICK K, DRAY G, et al. NIRS-measured prefrontal cortex activity in neuroergonomics: strengths and weaknesses[J]. Frontiers in Human Neuroscience, 2013, 7:583. doi: 10.3389/fnhum.2013.00583 pmid: 24065906
[34]	AYAZ, HASAN, SCHELDRUP, et al. Prefrontal hemodynamics of physical activity and environmental complexity during cognitive work[J]. Human factors: The journal of the Human Factors Society, 2017, 59(1):147-162.
[35]	LI Z, ZHANG M, ZHANG X, et al. Assessment of cerebral oxygenation during prolonged simulated driving using near infrared spectroscopy: its implications for fatigue development[J]. European Journal of Applied Physiology, 2009, 107(3):281. doi: 10.1007/s00421-009-1122-6 pmid: 19578870
[36]	NGUYEN T, AHN S, JANG H, et al. Utilization of a combined EEG/NIRS system to predict driver drowsiness[J]. Scientific Reports, 2017, 7(1): 43933.
[37]	MUTHALIB M, ANWAR A R, PERREY S, et al. Multimodal integration of fNIRS, fMRI and EEG neuroimaging[J]. Clinical Neurophysiology, 2013, 124(10):2060-2062. doi: 10.1016/j.clinph.2013.03.018 pmid: 23648071
[38]	HAMANN A, CARSTENGERDES N. Assessing the development of mental fatigue during simulated flights with concurrent EEG-fNIRS measurement[J]. Scientific Reports, 2023, 13(1): 4738. doi: 10.1038/s41598-023-31264-w pmid: 36959334
[39]	邢祥莹. MR技术的发展与现状及其在教育中的应用[J]. 中国现代教育装备, 2024,(6):31-33.
	XING X Y. The development and current status of MR technology and its application in education[J]. China Modern Educational Equipment, 2024,(6):31-33. (in Chinese)
[40]	BAUMEISTER J, SSIN S Y, ELSAYED N A M, et al. Cognitive cost of using augmented reality displays[J]. IEEE Transactions on Visualization and Computer Graphics, 2017(99):1.
[41]	MERENDA C, KIM H, TANOUS K, et al. Augmented reality interface design approaches for goal-directed and stimulus-driven driving tasks[J]. IEEE Transactions on Visualization and Computer Graphics, 2018, 24(11):2875-2885.
[42]	LI W C, HORN A, ZHANG J. The influences of flight deck interface design on pilot situation awareness and perceived workload[C]// Proceedings of the International Conference on Ergonomics and Human Factors. 2019.
[43]	HE H, ZHANG H, CHEN L, et al. AR technology based fatigue driving detecting system, has EEG signal collecting device for collecting EEG signal of driver brain wave, and fatigue state determining module for judging fatigue state of driver according to EEG signal:CN109875583-B[P]. (2021-01-29)[ 2024-03-20].
[44]	LI D. Method for warning fatigue driving of driver of vehicle, involves judging fatigue driving state type, and vibrating reality glasses worn by driver to warn driver under condition of fatigue driving state type of driver is a target type: CN114037981-A[P]. (2022-02-11) [2024-03-20].
[45]	ZHANG H, JIANG G. Augmented reality display control system for reducing visual fatigue, has instruction and feedback unit connected with model analysis unit, and central processor unit used for target composite motion computing trend to display control model: augmented reality display control system for reducing visual fatigue, has instruction and feedback unit connected with model analysis unit, and central processor unit used for target composite motion computing trend to display control model[P]. (2022-12-20) [2024-03-20].
[46]	金钰, 谷全祥. 2023年国外军用无人机装备技术发展综述[J]. 战术导弹技术, 2024,(1):33-47.
	JIN Y, GU Q X. Overview of the development of foreign military UAV[J]. Tactical Missile Technology, 2024,(1):33-47. (in Chinese)
[47]	张琳. 美国陆军新型无人地面平台[J]. 坦克装甲车辆, 2023,(17):24-29.
	ZHANG L. US Army’s new unmanned ground platform[J]. Tank & Armoured Vehicle, 2023,(17):24-29. (in Chinese)
[48]	王荣浩, 文晓, 向峥嵘. 人机融合系统协同与优化方法研究进展[J/OL]. 指挥控制与仿真, (2024-07-22)[2024-07-22]. http://kns.cnki.net/kcms/detail/32.1759.TJ.20240130.1727.004.html.
	WANG R H, WEN X, XIANG Z R, et al. Research status of collaboration and optimization method for human-machine fusion system[J/OL]. Command Control & Simulation, (2024-07-22) [2024-07-22]. http://kns.cnki.net/kcms/detail/32.1759.TJ.20240130.1727.004.html. (in Chinese)
[49]	牛佳然, 吴晓莉, 张蓝, 等. 基于视觉认知特性的作战显控界面信息结构布局研究[J/OL]. 包装工程,(2023-07-20) [ 2024-03-20]. http://kns.cnki.net/kcms/detail/50.1094.TB.20230720.1305.002.html.
	NIU J R, WU X L, ZHANG L, et al. Research on information structure layout of combat display and control interface based on visual cognition characteristics[J/OL]. Packaging Engineering,(2023-07-20) [2024-03-20]. http://kns.cnki.net/kcms/detail/50.1094.TB.20230720.1305.002.html. (in Chinese)
[50]	张力, 周易川, 贾惠侨, 等. 数字化控制系统信息显示特征对操纵员信息捕获绩效的影响及优化研究[J]. 中国安全生产科学技术, 2016, 12(10):62-67.
	ZHANG L, ZHOU Y C, JIA H Q, et al. Research on influence and optimization of information display characteristics on operator’s information capture performance in digital control system[J]. Journal of Safety Science and Technology, 2016, 12(10):62-67. (in Chinese)
[51]	杨姣姣, 李嘉辉. 识别驾驶分心方法综述[J]. 时代汽车, 2023,(6):169-171.
	YANG J J, LI J H. A review of methods for identifying driving distractions[J]. Traffic and Safety, 2023,(6):169-171. (in Chinese)

序号	描述	等级
1	极度警醒	1
2	非常警醒	2
3	警醒	3
4	比较警醒	4
5	不太警醒但也无困意	5
6	有一些困意倾向	6
7	有困意,但是不需要努力保持清醒	7
8	有困意,且需要一定的努力保持清醒	8
9	非常困倦,需要极大的努力保持清醒	9

序号	描述	等级
1	极度警醒	1
2	非常警醒	2
3	警醒	3
4	比较警醒	4
5	不太警醒但也无困意	5
6	有一些困意倾向	6
7	有困意,但是不需要努力保持清醒	7
8	有困意,且需要一定的努力保持清醒	8
9	非常困倦,需要极大的努力保持清醒	9

序号	描述	等级
1	感觉有活力、生机、警觉、清醒	1
2	处于高水平的功能状态,但非顶峰状态,能够集中注意力	2
3	松弛、清醒、反应能力不是最佳	3
4	多少有些不清醒,情绪不高	4
5	头脑不清醒;有点想睡;思维减慢	5
6	困倦,努力坚持者不睡着;希望躺下	6
7	无法继续保持清醒,很快可以入睡;出现类似做梦的思维	7

序号	描述	等级
1	感觉有活力、生机、警觉、清醒	1
2	处于高水平的功能状态,但非顶峰状态,能够集中注意力	2
3	松弛、清醒、反应能力不是最佳	3
4	多少有些不清醒,情绪不高	4
5	头脑不清醒;有点想睡;思维减慢	5
6	困倦,努力坚持者不睡着;希望躺下	6
7	无法继续保持清醒,很快可以入睡;出现类似做梦的思维	7