多模态告警和认知负荷对装甲车辆乘员反应的影响

doi:10.12382/bgxb.2022.0018

摘要/Abstract

摘要：

为探究敌车方向的多模态告警对不同认知负荷水平下装甲车辆乘员反应的影响,招募20名成年男性开展告警类型和认知负荷水平双因素工效学实验。基于装甲车辆乘员任务虚拟仿真实验平台,面向装甲车辆乘员某一假定作业任务,针对敌车方向对不同认知负荷水平下的乘员进行告警。实验结果表明:相比视觉告警,包含触觉的多模态告警更容易使乘员理解且更符合乘员的认知状态,显著减少了乘员的反应时间、歼敌时间和反应错误率;相比低认知负荷水平,高认知负荷水平下乘员的信息理解能力显著下降,反应时间、歼敌时间和反应错误率显著增加;在视觉通道信息量过载或认知负荷增大时,可采用多模态告警来加快乘员的反应时间,提升作战绩效。该研究可为装甲车辆舱室多模态人机交互告警设计提供理论依据。

关键词: 装甲车辆, 乘员, 告警, 多模态, 认知负荷

Abstract:

To explore the effects of multimodal warning on the response of armored vehicle occupants regarding the direction of enemy vehicles under different cognitive load levels, 20 adult males were recruited to take two-factor ergonomic experiments on the warning type and cognitive load level. The experiments were based on a virtual simulated armored vehicle occupant task platform designed for a hypothetical operational task for armored vehicle occupants and warned the occupants under different cognitive load levels about enemy vehicle directions. Results show that multimodal warning with haptics was easier for occupants to understand the situation and more in line with their cognitive state than visual warning. This type of warning significantly reduced their reaction time, annihilation time, and response error rate. Occupants with a high cognitive load level showed a significant decrease in information comprehension and a significant increase in reaction time, annihilation time, and response error rate compared to those with a lower load. The findings suggest that multimodal warning can be used to speed up occupant reaction and improve combat performance when the visual channel is overloaded or the cognitive load is high. This study provides a theoretical basis for designing multimodal human-computer interaction warnings in armored vehicle compartments.

Key words: armored vehicle, occupant, warning, multimodal, cognitive load

孙晓东, 金晓萍, 解芳, 孙厚杰, 郑思涓. 多模态告警和认知负荷对装甲车辆乘员反应的影响[J]. 兵工学报, 2023, 44(4): 972-981.

SUN Xiaodong, JIN Xiaoping, XIE Fang, SUN Houjie, ZHENG Sijuan. Effects of Multimodal Warning and Cognitive Load on the Response of Armored Vehicle Occupants[J]. Acta Armamentarii, 2023, 44(4): 972-981.

图/表 18

参考文献 32

[1]	MAO M, XIE F, HU J J, et al. Analysis of workload of tankcrew under the conditions of informatization[J]. Defence Technology, 2014, 10(1):17-21. doi: 10.1016/j.dt.2013.12.008 URL
[2]	傅斌贺, 刘维平, 聂俊峰, 等. 考虑认知行为差异的乘员信息作业绩效研究[J]. 兵工学报, 2019, 40(3):659-665. doi: 10.3969/j.issn.1000-1093.2019.03.026
	FU B H, LIU W P, NIE J F, et al. Research on crew’s information operation performance with the difference of cognitive behavior[J]. Acta Armamentarii, 2019, 40(3): 659-665. (in Chinese)
[3]	GLUMM M M, KEHRING K L, WHITE T L. Effects of multimodal displays about threat location on target acquisition and attention to visual and auditory communications: ARL-TR-4074[R]. Aberdeen Proving Ground, MD, US:U.S.Army Research Laboratory, 2007:20-26.
[4]	ELLIOTT L R, SCHMEISSER E T, REDDENE S. Development of tactile and haptic systems for US infantry navigation and communication[C]//SMITH M J, SALVENDY G. Human Interface and the Management of Information. Interacting with Information. Orlando, FL, US:Springer, 2011:399-407.
[5]	GRAY R, SPENCE C, HO C, et al. Efficient multimodal cuing of spatial attention[J]. Proceedings of the IEEE, 2013, 101(9):2113-2122. doi: 10.1109/JPROC.2012.2225811 URL
[6]	ERNST M O, BULTHOOF H H. Merging the senses into a robust percept[J]. Trends in Cognitive Sciences, 2004, 8(4): 162-169. doi: 10.1016/j.tics.2004.02.002 pmid: 15050512
[7]	SPENCE C, DRIVER J. Crossmodal space and crossmodal attention[M]. New York, NY, US: Oxford University Press, 2004:148-156.
[8]	HO C, SPENCE C. The multisensory driver:implications for ergonomic car interface design[M]. London, UK: CRC Press, 2017:18-26.
[9]	WICKENS C D. Multiple resources and performance prediction[J]. Theoretical Issues in Ergonomics Science, 2002, 3(2):159-177. doi: 10.1080/14639220210123806 URL
[10]	WICKENS C D. Multiple resources and mental workload[J]. Human Factors, 2008, 50(3):449-455. pmid: 18689052
[11]	WANG Y W, WU B H, MA S, et al. Effect of mapping characteristic on audiovisual warning:evidence from a simulated driving study[J]. Applied Ergonomics, 2022, 99:103638. doi: 10.1016/j.apergo.2021.103638 URL
[12]	HUANG G J, PITTS B J. The effects of age and physical exercise on multimodal signal responses:implications for semi-autonomous vehicle takeover requests[J]. Applied Ergonomics, 2022, 98:103595. doi: 10.1016/j.apergo.2021.103595 URL
[13]	LUNDQVIST L M, ERIKSSON L. Age, cognitive load, and multimodal effects on driver response to directional warning[J]. Applied Ergonomics, 2019, 76:147-154. doi: 10.1016/j.apergo.2019.01.002 URL
[14]	OSKARSSON P A, ERIKSSON L, CARLANDER O. Enhanced perception and performance by multimodal threat cueing in simulated combat vehicle[J]. Human Factors, 2012, 54(1):122-137. doi: 10.1177/0018720811424895 URL
[15]	VAN ERP J B F, ERIKSSON L, LEVIN B, et al. Tactile cueing effects on performance in simulated aerial combat with high acceleration[J]. Aviation, Space, and Environmental Medicine, 2007, 78(12):1128-1134. pmid: 18064917
[16]	CALHOUN G L, FONTEGON J V, DRAPER M H, et al. Tactile versus aural redundant alert cues for UAV control applications[C]//Proceedings of the Human Factors and Ergonomics Society Annual Meeting. Los Angeles, CA, US: Sage Publications, 2004, 48(1):137-141.
[17]	NGO M K, PIERCE R S, SPENCE C. Using multisensory cues to facilitate air traffic management[J]. Human Factors, 2012, 54(6):1093-1103. pmid: 23397817
[18]	李家文, 成波. 驾驶员疲劳状态适应式复合预警方法的研究[J]. 汽车工程, 2012, 34(3):211-216.
	LI J W, CHENG B. A study on the adaptive multiple warnings scheme for driver drowsiness[J]. Automotive Engineering, 2012, 34(3):211-216. (in Chinese)
[19]	张子健. 人机共驾模式下多模态刺激对驾驶员情景意识唤醒及接管评价研究[D]. 重庆: 重庆大学, 2019.
	ZHANG Z J. Research on driver situation awareness arousal and takeover evaluation by multi-modal stimuli in man-machine shared driving mode[D]. Chongqing: Chongqing University, 2019. (in Chinese)
[20]	陈慧娟. 飞机驾驶舱多通道人机交互设计研究[D]. 南京: 东南大学, 2016.
	CHEN H J. Research on multi-modal human-computer interaction design in cockpit[D]. Nanjing: Southeast University, 2016. (in Chinese)
[21]	KALYUGA S. Cognitive load theory:implications for affective computing[C]//Proceedings of the Twenty-Fourth International FLAIRS Conference. Palm Beach, FL, US:AAAI, 2011:105-110.
[22]	SANTANGELO V, SPENCE C. Multisensory cues capture spatial attention regardless of perceptual load[J]. Journal of Experimental Psychology:Human Perception and Performance, 2007, 33(6):1311-1321. doi: 10.1037/0096-1523.33.6.1311 URL
[23]	MARUCCI M, DI FLUMERI G, BORGHINI G, et al. The impact of multisensory integration and perceptual load in virtual reality settings on performance, workload and presence[J]. Scientific Reports, 2021, 11(1):1-15. doi: 10.1038/s41598-020-79139-8
[24]	GILSON R D, REDDEN E S, ELLIOTT L R. Remote tactile displays for future soldiers: ARL-SR-0152[R]. Aberdeen Proving Ground, MD, US: U.S.Army Research Laboratory, 2007:17-39.
[25]	GREEN P, WILLIAMS M. Perspective in orientation/navigation displays: a human factors test[C]//Proceedings of the Third International Conference on Vehicle Navigation and Information Systems. Piscataway, NJ, US:IEEE, 1992:221-226.
[26]	ELLIOTT L R, REDDEN E S. Reducing workload: a multisensory approach[M]//Designing Soldier Systems. London, UK: CRC Press, 2018: 69-96.
[27]	WHITE T L, HANCOCK P A. Specifying advantages of multi-modal cueing: Quantifying improvements with augmented tactile information[J]. Applied Ergonomics, 2020, 88:103146. doi: 10.1016/j.apergo.2020.103146 URL
[28]	VAN ERP J B F, WERKHOVEN P, WERKHOVEN P. Validation of principles for tactile navigation displays[C]// Proceedings of the Human Factors and Ergonomics Society Annual Meeting. Los Angeles, CA, US: SAGE Publications, 2006, 50(16):1687-1691.
[29]	WHITE T L, KRAUSMAN A S, HAAS E C. Tactile displays in Army operational environments[M]//MARTIN J, ALLENDER L, SAVAGE-KNEPSHIELDP, et al. Designing Soldier Systems. London, UK: CRC Press, 2018:97-116.
[30]	LOOMIS J M, LIPPA Y, KLATZKY R L, et al. Spatial updating of locations specified by 3-D sound and spatial language[J]. Journal of Experimental Psychology:Learning, Memory, and Cognition, 2002, 28(2):335-345. doi: 10.1037/0278-7393.28.2.335 URL
[31]	LAVIE N. Distracted and confused?:selective attention under load[J]. Trends in Cognitive Sciences, 2005, 9(2): 75-82. doi: 10.1016/j.tics.2004.12.004 URL
[32]	SPENCE C. Crossmodal spatial attention[J]. Annals of the New York Academy of Sciences, 2010, 1191(1):182-200. doi: 10.1111/nyas.2010.1191.issue-1 URL

因素1	因素2
因素1	高认知负荷	低认知负荷
V	视觉告警倒数计数次任务	视觉告警无倒数计数次任务
V+A	视觉告警听觉告警倒数计数次任务	视觉告警听觉告警无倒数计数次任务
V+T	视觉告警触觉告警倒数计数次任务	视觉告警触觉告警无倒数计数次任务
V+A+T	视觉告警听觉告警触觉告警倒数计数次任务	视觉告警听觉告警触觉告警无倒数计数次任务

因素1	因素2
因素1	高认知负荷	低认知负荷
V	视觉告警倒数计数次任务	视觉告警无倒数计数次任务
V+A	视觉告警听觉告警倒数计数次任务	视觉告警听觉告警无倒数计数次任务
V+T	视觉告警触觉告警倒数计数次任务	视觉告警触觉告警无倒数计数次任务
V+A+T	视觉告警听觉告警触觉告警倒数计数次任务	视觉告警听觉告警触觉告警无倒数计数次任务

等级	评价内容
等级	信号理解	知觉匹配
2	能够快速并正确理解告警信息	十分符合自身认知状态的感觉
1	能够理解告警信息	比较符合自身认知状态的感觉
0	一定条件下可以理解告警信息	一定条件下符合自身认知状态
-1	会对告警信息产生误解	与自身认知状态感觉不同
-2	完全无法理解告警信息	与自身认知状态感觉完全不同

等级	评价内容
等级	信号理解	知觉匹配
2	能够快速并正确理解告警信息	十分符合自身认知状态的感觉
1	能够理解告警信息	比较符合自身认知状态的感觉
0	一定条件下可以理解告警信息	一定条件下符合自身认知状态
-1	会对告警信息产生误解	与自身认知状态感觉不同
-2	完全无法理解告警信息	与自身认知状态感觉完全不同

等级	舒适度
2	完全没有烦扰感
1	没有烦扰感
0	有轻微烦扰感
-1	烦扰感较高,产生不快情绪
-2	烦扰感很高,产生不适和厌恶