Effects of Armored Vehicle Multi-modal Interactions and Environmental Load on Mission Performance of Occupant

doi:10.12382/bgxb.2024.0079

Abstract

Abstract:

With the progress of information technology,the human-computer interaction modes of armored vehicles have been constantly updated.In order to explore the mission performance of occupant in new multimodal interaction and guide the design of multimodal operation and warning modes of future armored vehicles,a human-computer interaction experimental system which integrates multimodal interaction and adjusts the environmental load is established.Based on this system,20 adult males were recruited to conduct an ergonomics experiment involving operation (input) mode,warning (output) mode and environmental load level. The simulated target hitting experiments under different experimental conditions are performed in a virtual experimental platform,and the mission performances of occupants are analyzed.The experimental results show that the reaction time of occupant under mechanical input is the shortest and the operation error rate is the highest compared with those under touch input or voice input.The mission completion time of occupant under mechanical input and touch input is much shorter than that under voice input.Compared to the single modal visual (V) warning,the visual+tactile (V+T) dual-modal warning superimposed with tactile warning is used to significantly shorten the reaction time of occupant,while the visual+auditory (V+A) dual-modal warning superimposed with auditory warning has no significant difference.V+T or V+A dual-modal warning can significantly shorten the mission completion time of occupant.For the influence of environmental load,the mission performance of crew under low environmental load is significantly higher than that under high environmental load.As the environmental load increases,superimposing A warning in V+T warning will cause warning redundancy.The research can provide a practical basis for the design of multi-modal human-computer interaction in the armored vehicle cabin.

Key words: armored vehicle, occupant, multi-modal interaction, environmental load, mission performance

CLC Number:

TJ811

LUAN Kedi, GUO Baoqiao, WEI Zhongliang, GU Yuansen, WANG Gangting, DENG Lisha, CHEN Pengwan. Effects of Armored Vehicle Multi-modal Interactions and Environmental Load on Mission Performance of Occupant[J]. Acta Armamentarii, 2025, 46(4): 240079-.

Figures/Tables 23

Table 1 Measuring and acquiring equipment and their parameters

设备	参数
噪声发生设备	2.0声道,最大功率40W
可调光强式光源	额定功率14.5W(72×0.2W/LED模块)
数字照度计	量程0.1~99990lx、0.1~999.9lx,分度值0.1lx;1000~99990lx,分度值1lx
数字噪声计	量程30~130dB,分度值0.1dB
视频采集设备	300万像素,帧率30帧/s

Fig.1 Interface of armored vehicle virtual simulation software

Fig.2 Operating principle of vibration bracelet

Fig.3 Armored vehicle multi-modal human-computer interaction test system

Fig.4 Azimuth map of enemy vehicles relative to ours

Table 2 Enemy vehicles location information

序号	方位
1	东偏南45°
2	东偏北45°
3	东偏南10°
4	东偏南50°
5	东偏北10°
6	东偏北50°

Table 3 Setting of different experimental conditions

因素1	因素2	因素3
低环境负荷	机械输入	V
		V+A
		V+T
		V+A+T
	触屏输入	V
		V+A
		V+T
		V+A+T
	语音输入	V
		V+A
		V+T
		V+A+T
高环境负荷	机械输入	V
		V+A
		V+T
		V+A+T
	触屏输入	V
		V+A
		V+T
		V+A+T
	语音输入	V
		V+A
		V+T
		V+A+T

Table 4 Order in which enemy vehicles appear in different modes

实验编号类型	敌方车辆波次
实验编号类型	1	2	3	4
奇数	1	2、5	6	3、4
偶数	2	1、3	4	5、6

Fig.5 Reaction time and annihilation time

Table 5 Descriptive statistical results of reaction time,mission completion time and operation error rate under different experimental conditions

环境负荷	输入方式	输出方式	反应时间/s		任务完成时间/s		操作错误数	操作错误率/%
环境负荷	输入方式	输出方式	平均值	标准差	平均值	标准差	操作错误数	操作错误率/%
低环境负荷	机械输入	V	1.31	0.94	77.88	16.76	20	16.67
		V+A	1.20	0.69	68.77	11.35	14	11.67
		V+T	1.06	0.63	66.15	8.93	9	7.50
		V+A+T	1.05	0.72	69.23	11.30	10	8.33
	触屏输入	V	2.01	0.89	75.18	10.74	1	0.83
		V+A	2.00	0.86	70.54	10.32	2	1.67
		V+T	1.74	0.76	68.21	5.50	1	0.83
		V+A+T	1.65	0.82	68.51	7.07	2	1.67
	语音输入	V	4.58	1.25	122.78	20.86	2	1.67
		V+A	4.22	1.09	112.23	13.61	3	2.50
		V+T	4.20	1.34	105.88	8.84	0	0
		V+A+T	3.93	0.93	118.90	13.09	2	1.67
高环境负荷	机械输入	V	1.58	1.04	92.05	28.45	16	13.33
		V+A	1.42	0.81	80.60	18.50	17	14.17
		V+T	1.13	0.78	75.74	14.23	18	15.00
		V+A+T	1.18	0.87	76.94	10.86	18	15.00
	触屏输入	V	2.35	1.32	89.36	15.18	6	5.00
		V+A	2.29	1.20	87.29	22.95	5	4.17
		V+T	2.05	1.27	79.90	13.30	2	1.67
		V+A+T	1.98	1.19	77.45	10.31	3	2.50
	语音输入	V	5.12	1.51	146.23	28.24	4	3.33
		V+A	4.74	1.23	123.57	17.82	3	2.5
		V+T	4.29	0.97	123.33	12.16	1	0.83
		V+A+T	5.97	1.38	163.92	24.22	0	0

Table 6 Results of intersubjective effects

自变量	反应时间				任务完成时间				操作错误数
自变量	f	F	p	偏η²	f	F	p	偏η²	f	F	p	偏η²
环境负荷	1	92.656	<0.001^**	0.037	1	120.032	<0.001^**	0.208	1	5.963	0.015	0.013
输入方式	2	2227.280	<0.001^**	0.649	2	533.338	<0.001^**	0.701	2	87.917	<0.001^**	0.278
输出方式	3	14.673	<0.001^**	0.018	3	17.621	<0.001^**	0.104	3	2.211	0.086	0.014
环境负荷×输入方式	2	18.753	<0.001^**	0.015	2	8.245	<0.001^**	0.035	2	1.399	0.248	0.006
环境负荷×输出方式	3	10.494	<0.001^**	0.013	3	1.529	0.206	0.010	3	0.357	0.784	0.002
输入方式×输出方式	6	7.723	<0.001^**	0.019	6	8.685	<0.001^**	0.103	6	0.215	0.972	0.003
环境负荷×输入方式×输出方式	6	11.285	<0.001^**	0.027	6	3.801	0.001^*	0.048	6	1.878	0.083	0.024

Fig.6 Reaction times under different input modes

Fig.7 Mission completion times under different input methods

Table 7 Multiple comparison of mission completion times of armored vehicle occupant under different input modes

参考项	比较项	任务完成时间平均值差值/s	p
机械输入	触屏输入	-1.135	0.526
机械输入	语音输入	-51.184^*	<0.001
触屏输入	机械输入	1.135	0.526
触屏输入	语音输入	-50.048^*	<0.001
语音输入	机械输入	51.184^*	<0.001
语音输入	触屏输入	50.048^*	<0.001

Fig.8 Operation error rates under different input modes

Table 8 Multiple comparison of the number of operating errors of armored vehicle occupant under different input modes

参考项	比较项	操作错误数平均值差值	p
机械输入	触屏输入	0.62^*	<0.001
机械输入	语音输入	0.67^*	<0.001
触屏输入	机械输入	-0.62^*	<0.001
触屏输入	语音输入	0.04	0.439
语音输入	机械输入	-0.67^*	<0.001
语音输入	触屏输入	-0.04	0.439

Fig.9 Reaction times under different output modes

Table 9 Multiple comparison of reaction times of armored vehicle occupant under different output modes

参考项	比较项	反应时间平均值差值/s	p
V	V+A	0.116	0.062
	V+T	0.409^*	<0.001
	V+A+T	0.199^**	0.001
V+A	V	-0.116	0.062
	V+T	0.293^*	<0.001
	V+A+T	0.083	0.181
V+T	V	-0.409^*	<0.001
	V+A	-0.293^*	<0.001
	V+A+T	-0.210^**	0.001
V+A+T	V	-0.199^**	0.001
	V+A	-0.083	0.181
	V+T	0.210^**	0.001

Fig.10 Mission completion times under different output modes

Table 10 Multiple comparison of mission ompletion times of armored vehicle occupant under different output modes

参考项	比较项	任务完成时间平均值差值/s	p
V	V+A	10.08^*	0.011
	V+T	14.04^**	<0.001
	V+A+T	4.75	0.230
V+A	V	-10.08^*	0.011
	V+T	3.96	0.316
	V+A+T	-5.33	0.178
V+T	V	-14.0 $4 * *$	<0.001
	V+A	-3.96	0.316
	V+A+T	-9.29^*	0.019
V+A+T	V	-4.75	0.230
	V+A	5.32	0.178
	V+T	9.29^*	0.019

Table 10 Multiple comparison of mission ompletion times of armored vehicle occupant under different output modes

参考项	比较项	任务完成时间平均值差值/s	p
V	V+A	10.08^*	0.011
	V+T	14.04^**	<0.001
	V+A+T	4.75	0.230
V+A	V	-10.08^*	0.011
	V+T	3.96	0.316
	V+A+T	-5.33	0.178
V+T	V	-14.0 $4 * *$	<0.001
	V+A	-3.96	0.316
	V+A+T	-9.29^*	0.019
V+A+T	V	-4.75	0.230
	V+A	5.32	0.178
	V+T	9.29^*	0.019

Fig.11 Reaction times under different environmental loads

Fig.12 Mission completion times under different environmental loads

Table 11 Descriptive statistical results of reaction time and mission completion time under the interaction of input mode and output mode (mean±standard deviation)

输入方式	输出方式	反应时间/s		任务完成时间/s
输入方式	输出方式	平均值	标准差	平均值	标准差
机械输入	V	1.45	1.00	84.96	24.14
	V+A	1.30	0.76	74.69	16.29
	V+T	1.10	0.71	70.95	12.70
	V+A+T	1.11	0.79	73.09	11.61
触屏输入	V	2.19	1.15	82.27	14.83
	V+A	2.15	1.06	78.92	19.50
	V+T	1.90	1.06	74.06	11.66
	V+A+T	1.82	1.04	72.98	9.83
语音输入	V	4.85	1.41	134.50	27.23
	V+A	4.48	1.19	117.90	16.67
	V+T	4.25	1.17	114.61	13.72
	V+A+T	4.91	1.55	141.41	29.82

References 39

[1]	陈挚. 目标的识别与快速准确攻击一体化设计研究[D]. 南京: 南京航空航天大学, 2011.
	CHEN Z. Design research on integration detection identification and rapid accurate attack of target[D]. Nanjing: Nanjing University of Aeronautics and Astronautics, 2011. (in Chinese)
[2]	曹灵芝, 殷复鹏. 人因工程在汽车产品中的应用调查与分析[C]// 中国人类工效学学会成立20周年庆祝大会论文集. 北京: 中国人类工效学学会,2009:409-413.
	CAO L Z, YIN F P. Investigation and analysis of human factors application in chinese car industry[C]// Proceedings of the 20th Anniversary Celebration Conference of the Chinese Ergonomics Society. Beijing: Chinese Ergonomics Society,2009:409-413. (in Chinese)
[3]	张文镝, 周诗婕, 金海亮, 等. 自动驾驶系统中人机协同的认知决策一体化机制研究[J]. 汽车文摘, 2023(9):11-18.
	ZHANG W D, ZHOU S J, JIN H L, et al. Research on cognitive decision integration mechanism of human-machinecollaboration in automated driving system[J]. Automotive Digest, 2023(9):11-18. (in Chinese)
[4]	GRAY R, SPENCE C, HO C, et al. Efficient multimodal cuing of spatial attention[J]. Proceedings of the IEEE, 2013, 101(9):2113-2122.
[5]	钱晓松, 杨新, 张帆. 面向特种车辆的触屏交互系统设计研究[J]. 装饰, 2018(3):82-85.
	QIAN X S, YANG X, ZHANG F. An exploratory design research on touchscreen interaction system in special vehicles[J]. Art & Design, 2018(3):82-85. (in Chinese)
[6]	杨洋. 基于语音交互的手持移动端多通道交互研究[D]. 北京: 北京邮电大学, 2018.
	YANG Y. Reserch on multy-channel interaction of handheld mobile device based on voice interaction[D]. Beijing: Beijing University of Posts and Telecommunications, 2018. (in Chinese)
[7]	范雪乔. 基于手势识别VR系统的交互体验评价研究[D]. 哈尔滨: 哈尔滨工业大学, 2018.
	FAN X Q. Research on interactive experience evaluation based on gesture recognition VR system[D].Harbin: Harbin Institute of Technology, 2018. (in Chinese)
[8]	刘洋. 智能音箱语音交互设计中的可用性研究[D]. 北京: 中国矿业大学, 2022.
	LIU Y. Research on usability of intelligent speaker in voice interaction design[D]. Beijing: China University of Mining and Technology, 2022. (in Chinese)
[9]	高岚岚, 纪坤, 乐剑. 眼动交互在点击任务中的应用研究[J]. 舰船电子工程, 2023, 43(5):63-63,81.
	GAO L L, JI K, LE J. Application research of eye movement interaction in click task[J]. Ship Electronic Engineering, 2023, 43(5):63-66,81. (in Chinese)
[10]	徐源. 眼控—手控双通道交互方式的工效学研究[D]. 杭州: 浙江理工大学, 2017.
	XU Y. The ergonomics research of eye control-hand control dual channel interaction[D]. Hangzhou: Zhejiang Sci-Tech University, 2017. (in Chinese)
[11]	孙晓东, 金晓萍, 解芳, 等. 多模态告警和认知负荷对装甲车辆乘员反应的影响[J]. 兵工学报, 2023, 44(4):972-981.
	SUN X D, JIN X P, XIE F, et al. The effect of multimodal warning and cognitive load on the response of armored vehicle occupants[J]. Acta Armamentarii, 2023, 44(4):972-981. (in Chinese)
[12]	NGO M K, PIERCE R S, SPENCE C. Using multisensory cues to facilitate air traffic management[J]. Human Factors, 2012, 54(6):1093-1103. pmid: 23397817
[13]	WANG Y W, WU B H, MA S, et al. Effect of mapping characteristic on audiovisual warning:evidence from a simulated driving study[J]. Applied Ergonomics, 2022,99:103638.
[14]	LUNDQVIST L M, ERIKSSON L. Age,cognitive load,and multimodal effects on driver response to directional warning[J]. Applied ergonomics, 2019,76:147-154.
[15]	HUANG G, PITTS B J. The effects of age and physical exercise on multimodal signal responses:implications for semi-autonomous vehicle takeover requests[J]. Applied Ergonomics, 2022,98:103595.
[16]	张子健. 人机共驾模式下多模态刺激对驾驶员情景意识唤醒及接管评价研究[D]. 重庆: 重庆大学, 2019.
	ZHANG Z J. Research on driver situation awareness arousal and takeover evaluation by multi-modal stimuli in man-machine shared driving mode[D]. Chongqing: Chongqing University, 2019. (in Chinese)
[17]	OSKARSSON P A, ERIKSSON L, CARLANDER O. Enhanced perception and performance by multimodal threat cueing in simulated combat vehicle[J]. Human Factors, 2012, 54(1):122-137.
[18]	van ERP J B F, ERIKSSON L, LEVIN B, et al. Tactile cueing effects on performance in simulated aerial combat with high acceleration.[J]. Aviation,Space and Environmental Medicine, 2007, 78(12):1128-1134.
[19]	李家文, 成波. 驾驶员疲劳状态适应式复合预警方法的研究[J]. 汽车工程, 2012(3):211-216.
	LI J W, CHENG B. A study on the adaptive multiple warnings scheme for driver drowsiness[J]. Automotive Engineering, 2012(3):211-216. (in Chinese)
[20]	陈慧娟. 飞机驾驶舱多通道人机交互设计研究[D]. 南京: 东南大学, 2016.
	CHEN H J. Research on multi-modal human-computer interaction design in cockpit[D]. Nanjing: Southeast University, 2016. (in Chinese)
[21]	高莹. 基于环境心理学的老年人疗养中心设计方法的研究[D]. 西安: 长安大学, 2015.
	GAO Y. Study of the elderly convalescent center design method based on environmental psychology[D]. Xi’an: Chang’an University, 2015. (in Chinese)
[22]	石程. 基于环境—行为关系理论的老年公寓空间环境设计研究[D]. 兰州: 兰州理工大学, 2018.
	SHI C. Research on space environment design of elderly apartment based onenvironment-behavioral relationship theory[D]. Lanzhou: Lanzhou University of Technology, 2018. (in Chinese)
[23]	HUANG L, ZHU Y X, OUYANG Q, et al. A study on the effects of thermal,luminous,and acoustic environments on indoor environmental comfort in offices[J]. Building and Environment, 2011,49:304-309.
[24]	ŠABIĆE , CHEN J, MACDONALD J A, et al. Toward a better understanding of in-vehicle auditory warnings and background noise[J]. Human Factors, 2021,63(2):312-335.
[25]	KARUNARATHNE B, WANG T, SO R H Y, et al. Adversarial relationship between combined medial olivocochlear (MOC) and middle-ear-muscle (MEM) reflexes and alarm-in-noise detection thresholds under negative signal-to-noise ratios (SNRs)[J]. Hearing Research, 2018,367:124-128.
[26]	ANDERSON S, PARBERY-CLARK A, WHITE-SCHWOCH T, et al. Auditory brainstem response to complex sounds predicts self-reported speech-in-noise performance[J]. Journal of Speech Language and Hearing Research, 2013, 56(1):31-43.
[27]	张永超. 高铁客站候车区热、光、声环境对人体舒适度的交互作用影响研究[D]. 北京: 北京交通大学, 2021.
	ZHANG Y C. Research on interaction effect of thermal,light and acousticenvironment on human comfort in waiting area of high-speedrailway station[D]. Beijing: Beijing Jiaotong University, 2021. (in Chinese)
[28]	NAGANO K, HORIKOSHI T. New comfort index during combined conditions of moderate low ambient temperature and traffic noise[J]. Energy & Buildings, 2005, 37(3):287-294.
[29]	KIM H E, TOKURA H. Influence of two different light intensities from 16:00 to 20:30 hours on evening dressing behavior in the cold[J]. Collegium Antropologicum, 2007, 31(1):145-151.
[30]	FRANSSON N, VAESTFJAELL D, SKOOG J. In search of the comfortable indoor environment:a comparison of the utility of objective and subjective indicators of indoor comfort[J]. Building & Environment, 2007, 42(5):1886-1890.
[31]	YANG W, MOON H J. Cross-modal effects of illuminance and room temperature on indoor environmental perception[J]. Building and environment, 2018,146:280-288.
[32]	YANG W, MOON H J. Combined effects of sound and illuminance on indoor environmental perception[J]. Applied Acoustics, 2018,141:136-143.
[33]	ISHII J, TETSUMI H. The combined effects of illuminance and air temperature on the human physiological and psychological[C]// Proceedings of Meeting Architectural Institute of Japan.Minato-ku,Tokyo, Japan: Architectural Institute of Japan,1996.
[34]	YANG W, MOON H J. Combined effects of acoustic,thermal,and illumination conditions on the comfort of discrete senses and overall indoor environment[J]. Building and environment, 2019,148:623-633.
[35]	国家质量监督检验检疫总局.声环境质量标准:GB 3096—2008 [S]. 北京: 中国环境科学出版社, 2008.
	General Administration of Quality Supervision, Inspection and Quarantine of the People’s Republic of China.Environmental quality standard for noise:GB 3096—2008 [S]. Beijing: China Environmental Science Press, 2008. (in Chinese)
[36]	国家市场监督管理总局.视觉工效学原则.室内工作场所照明:GB/T 13379-2023[S]. 北京: 中国标准出版社, 2023.
	State Administration for Market Regulation.Principles of visual ergonomics-the lighting of indoor work places:GB/T 13379—2023[S]. Beijing: China standard press, 2023. (in Chinese).
[37]	高祖新, 言方荣. 概率论与数理统计[M]. 第2版.: 南京:南京大学出版社, 2020. GAO Z X,YAN F R.Probability and statistics[M]. 2 ed. Nanjing: Nanjing University Press,2020. (in Chinese)
[38]	黄捷. 人机交互在震动环境下的仿真模型设计与实验测试[C]// 第四届和谐人机环境联合学术会议论文集. 昆明: 中国计算机学会,2008:1-7.
	HUANG J. Model design and experimental test of HCI in vibratile environment[C]// Proceedings of the fourth Harmonious Human-Machine Environment Joint Academic Conference. Kunming: China Computer Federation,2008:1-7. (in Chinese)
[39]	毛信良, 周田伟, 陈二喜. 一种用于多点触摸终端的触摸控制方法与设备:CN104076986B[P].2015-12-09.
	MAO X L, ZHOU T W, CHEN E X. A touch control method and device for multi-touch terminal:CN104076986B[P].2015-12-09. (in Chinese)