Effects of Multimodal Warning and Cognitive Load on the Response of Armored Vehicle Occupants

doi:10.12382/bgxb.2022.0018

Abstract

Abstract:

To explore the effects of multimodal warning on the response of armored vehicle occupants regarding the direction of enemy vehicles under different cognitive load levels, 20 adult males were recruited to take two-factor ergonomic experiments on the warning type and cognitive load level. The experiments were based on a virtual simulated armored vehicle occupant task platform designed for a hypothetical operational task for armored vehicle occupants and warned the occupants under different cognitive load levels about enemy vehicle directions. Results show that multimodal warning with haptics was easier for occupants to understand the situation and more in line with their cognitive state than visual warning. This type of warning significantly reduced their reaction time, annihilation time, and response error rate. Occupants with a high cognitive load level showed a significant decrease in information comprehension and a significant increase in reaction time, annihilation time, and response error rate compared to those with a lower load. The findings suggest that multimodal warning can be used to speed up occupant reaction and improve combat performance when the visual channel is overloaded or the cognitive load is high. This study provides a theoretical basis for designing multimodal human-computer interaction warnings in armored vehicle compartments.

Key words: armored vehicle, occupant, warning, multimodal, cognitive load

SUN Xiaodong, JIN Xiaoping, XIE Fang, SUN Houjie, ZHENG Sijuan. Effects of Multimodal Warning and Cognitive Load on the Response of Armored Vehicle Occupants[J]. Acta Armamentarii, 2023, 44(4): 972-981.

Figures/Tables 18

References 32

[1]	MAO M, XIE F, HU J J, et al. Analysis of workload of tankcrew under the conditions of informatization[J]. Defence Technology, 2014, 10(1):17-21. doi: 10.1016/j.dt.2013.12.008 URL
[2]	傅斌贺, 刘维平, 聂俊峰, 等. 考虑认知行为差异的乘员信息作业绩效研究[J]. 兵工学报, 2019, 40(3):659-665. doi: 10.3969/j.issn.1000-1093.2019.03.026
	FU B H, LIU W P, NIE J F, et al. Research on crew’s information operation performance with the difference of cognitive behavior[J]. Acta Armamentarii, 2019, 40(3): 659-665. (in Chinese)
[3]	GLUMM M M, KEHRING K L, WHITE T L. Effects of multimodal displays about threat location on target acquisition and attention to visual and auditory communications: ARL-TR-4074[R]. Aberdeen Proving Ground, MD, US:U.S.Army Research Laboratory, 2007:20-26.
[4]	ELLIOTT L R, SCHMEISSER E T, REDDENE S. Development of tactile and haptic systems for US infantry navigation and communication[C]//SMITH M J, SALVENDY G. Human Interface and the Management of Information. Interacting with Information. Orlando, FL, US:Springer, 2011:399-407.
[5]	GRAY R, SPENCE C, HO C, et al. Efficient multimodal cuing of spatial attention[J]. Proceedings of the IEEE, 2013, 101(9):2113-2122. doi: 10.1109/JPROC.2012.2225811 URL
[6]	ERNST M O, BULTHOOF H H. Merging the senses into a robust percept[J]. Trends in Cognitive Sciences, 2004, 8(4): 162-169. doi: 10.1016/j.tics.2004.02.002 pmid: 15050512
[7]	SPENCE C, DRIVER J. Crossmodal space and crossmodal attention[M]. New York, NY, US: Oxford University Press, 2004:148-156.
[8]	HO C, SPENCE C. The multisensory driver:implications for ergonomic car interface design[M]. London, UK: CRC Press, 2017:18-26.
[9]	WICKENS C D. Multiple resources and performance prediction[J]. Theoretical Issues in Ergonomics Science, 2002, 3(2):159-177. doi: 10.1080/14639220210123806 URL
[10]	WICKENS C D. Multiple resources and mental workload[J]. Human Factors, 2008, 50(3):449-455. pmid: 18689052
[11]	WANG Y W, WU B H, MA S, et al. Effect of mapping characteristic on audiovisual warning:evidence from a simulated driving study[J]. Applied Ergonomics, 2022, 99:103638. doi: 10.1016/j.apergo.2021.103638 URL
[12]	HUANG G J, PITTS B J. The effects of age and physical exercise on multimodal signal responses:implications for semi-autonomous vehicle takeover requests[J]. Applied Ergonomics, 2022, 98:103595. doi: 10.1016/j.apergo.2021.103595 URL
[13]	LUNDQVIST L M, ERIKSSON L. Age, cognitive load, and multimodal effects on driver response to directional warning[J]. Applied Ergonomics, 2019, 76:147-154. doi: 10.1016/j.apergo.2019.01.002 URL
[14]	OSKARSSON P A, ERIKSSON L, CARLANDER O. Enhanced perception and performance by multimodal threat cueing in simulated combat vehicle[J]. Human Factors, 2012, 54(1):122-137. doi: 10.1177/0018720811424895 URL
[15]	VAN ERP J B F, ERIKSSON L, LEVIN B, et al. Tactile cueing effects on performance in simulated aerial combat with high acceleration[J]. Aviation, Space, and Environmental Medicine, 2007, 78(12):1128-1134. pmid: 18064917
[16]	CALHOUN G L, FONTEGON J V, DRAPER M H, et al. Tactile versus aural redundant alert cues for UAV control applications[C]//Proceedings of the Human Factors and Ergonomics Society Annual Meeting. Los Angeles, CA, US: Sage Publications, 2004, 48(1):137-141.
[17]	NGO M K, PIERCE R S, SPENCE C. Using multisensory cues to facilitate air traffic management[J]. Human Factors, 2012, 54(6):1093-1103. pmid: 23397817
[18]	李家文, 成波. 驾驶员疲劳状态适应式复合预警方法的研究[J]. 汽车工程, 2012, 34(3):211-216.
	LI J W, CHENG B. A study on the adaptive multiple warnings scheme for driver drowsiness[J]. Automotive Engineering, 2012, 34(3):211-216. (in Chinese)
[19]	张子健. 人机共驾模式下多模态刺激对驾驶员情景意识唤醒及接管评价研究[D]. 重庆: 重庆大学, 2019.
	ZHANG Z J. Research on driver situation awareness arousal and takeover evaluation by multi-modal stimuli in man-machine shared driving mode[D]. Chongqing: Chongqing University, 2019. (in Chinese)
[20]	陈慧娟. 飞机驾驶舱多通道人机交互设计研究[D]. 南京: 东南大学, 2016.
	CHEN H J. Research on multi-modal human-computer interaction design in cockpit[D]. Nanjing: Southeast University, 2016. (in Chinese)
[21]	KALYUGA S. Cognitive load theory:implications for affective computing[C]//Proceedings of the Twenty-Fourth International FLAIRS Conference. Palm Beach, FL, US:AAAI, 2011:105-110.
[22]	SANTANGELO V, SPENCE C. Multisensory cues capture spatial attention regardless of perceptual load[J]. Journal of Experimental Psychology:Human Perception and Performance, 2007, 33(6):1311-1321. doi: 10.1037/0096-1523.33.6.1311 URL
[23]	MARUCCI M, DI FLUMERI G, BORGHINI G, et al. The impact of multisensory integration and perceptual load in virtual reality settings on performance, workload and presence[J]. Scientific Reports, 2021, 11(1):1-15. doi: 10.1038/s41598-020-79139-8
[24]	GILSON R D, REDDEN E S, ELLIOTT L R. Remote tactile displays for future soldiers: ARL-SR-0152[R]. Aberdeen Proving Ground, MD, US: U.S.Army Research Laboratory, 2007:17-39.
[25]	GREEN P, WILLIAMS M. Perspective in orientation/navigation displays: a human factors test[C]//Proceedings of the Third International Conference on Vehicle Navigation and Information Systems. Piscataway, NJ, US:IEEE, 1992:221-226.
[26]	ELLIOTT L R, REDDEN E S. Reducing workload: a multisensory approach[M]//Designing Soldier Systems. London, UK: CRC Press, 2018: 69-96.
[27]	WHITE T L, HANCOCK P A. Specifying advantages of multi-modal cueing: Quantifying improvements with augmented tactile information[J]. Applied Ergonomics, 2020, 88:103146. doi: 10.1016/j.apergo.2020.103146 URL
[28]	VAN ERP J B F, WERKHOVEN P, WERKHOVEN P. Validation of principles for tactile navigation displays[C]// Proceedings of the Human Factors and Ergonomics Society Annual Meeting. Los Angeles, CA, US: SAGE Publications, 2006, 50(16):1687-1691.
[29]	WHITE T L, KRAUSMAN A S, HAAS E C. Tactile displays in Army operational environments[M]//MARTIN J, ALLENDER L, SAVAGE-KNEPSHIELDP, et al. Designing Soldier Systems. London, UK: CRC Press, 2018:97-116.
[30]	LOOMIS J M, LIPPA Y, KLATZKY R L, et al. Spatial updating of locations specified by 3-D sound and spatial language[J]. Journal of Experimental Psychology:Learning, Memory, and Cognition, 2002, 28(2):335-345. doi: 10.1037/0278-7393.28.2.335 URL
[31]	LAVIE N. Distracted and confused?:selective attention under load[J]. Trends in Cognitive Sciences, 2005, 9(2): 75-82. doi: 10.1016/j.tics.2004.12.004 URL
[32]	SPENCE C. Crossmodal spatial attention[J]. Annals of the New York Academy of Sciences, 2010, 1191(1):182-200. doi: 10.1111/nyas.2010.1191.issue-1 URL

因素1	因素2
因素1	高认知负荷	低认知负荷
V	视觉告警倒数计数次任务	视觉告警无倒数计数次任务
V+A	视觉告警听觉告警倒数计数次任务	视觉告警听觉告警无倒数计数次任务
V+T	视觉告警触觉告警倒数计数次任务	视觉告警触觉告警无倒数计数次任务
V+A+T	视觉告警听觉告警触觉告警倒数计数次任务	视觉告警听觉告警触觉告警无倒数计数次任务

因素1	因素2
因素1	高认知负荷	低认知负荷
V	视觉告警倒数计数次任务	视觉告警无倒数计数次任务
V+A	视觉告警听觉告警倒数计数次任务	视觉告警听觉告警无倒数计数次任务
V+T	视觉告警触觉告警倒数计数次任务	视觉告警触觉告警无倒数计数次任务
V+A+T	视觉告警听觉告警触觉告警倒数计数次任务	视觉告警听觉告警触觉告警无倒数计数次任务

等级	评价内容
等级	信号理解	知觉匹配
2	能够快速并正确理解告警信息	十分符合自身认知状态的感觉
1	能够理解告警信息	比较符合自身认知状态的感觉
0	一定条件下可以理解告警信息	一定条件下符合自身认知状态
-1	会对告警信息产生误解	与自身认知状态感觉不同
-2	完全无法理解告警信息	与自身认知状态感觉完全不同

等级	评价内容
等级	信号理解	知觉匹配
2	能够快速并正确理解告警信息	十分符合自身认知状态的感觉
1	能够理解告警信息	比较符合自身认知状态的感觉
0	一定条件下可以理解告警信息	一定条件下符合自身认知状态
-1	会对告警信息产生误解	与自身认知状态感觉不同
-2	完全无法理解告警信息	与自身认知状态感觉完全不同

等级	舒适度
2	完全没有烦扰感
1	没有烦扰感
0	有轻微烦扰感
-1	烦扰感较高,产生不快情绪
-2	烦扰感很高,产生不适和厌恶