面向战场环境下的语种识别

doi:10.12382/bgxb.2022.0367

摘要/Abstract

摘要：

为实现语种识别在战场环境下保持较高的识别性能,提出一种基于语谱图灰度变换的语种识别方法。根据语音信息和战场环境下的噪声信息在语谱图上的分布特性,引入带通滤波;根据人耳听觉特性提取对数灰度语谱图;采用自动色阶算法抑制语谱图上的噪声信息,增强语种信息,并采用残差神经网络模型进行训练识别。实验结果表明:在-10dB掠夺者战斗机驾驶舱噪声环境下,相对于线性灰度语谱图特征,识别正确率提升了46%;在其他噪声环境下,识别性能也大幅度提升。

关键词: 语种识别, 对数灰度语谱图, 自动色阶算法, 残差神经网络

Abstract:

To achieve accurate language identification in battlefield environments, a language identification method based on spectrogram gray transformation is proposed. Bandpass filtering is introduced based on the distribution characteristics of speech information and noise information in the spectrogram under battlefield noise conditions. Logarithmic gray spectrogram is extracted in line with human auditory characteristics. An automatic color adjustment algorithm is used to suppress noise information and enhance language information on the spectrogram, and a residual neural network model is used for training and identification. Experimental results show that compared with linear gray spectrogram features, the recognition accuracy is improved by 46% in the -10dB Predator fighter cockpit noise environment. In other noise environments, the recognition performance is also greatly improved.

Key words: language identification, logarithmic grayscale spectrogram, automatic tone scale algorithm, residual neural network

华英杰, 刘晶, 邵玉斌, 朵琳. 面向战场环境下的语种识别[J]. 兵工学报, 2023, 44(7): 2197-2206.

HUA Yingjie, LIU Jing, SHAO Yubin, DUO Lin. Language Identification in Battlefield Environments[J]. Acta Armamentarii, 2023, 44(7): 2197-2206.

图/表 11

参考文献 28

[1]	LI H Z, MA B, LEE K A. Spoken language identification: from fundamentals to practice[J]. Proceedings the IEEE, 2013, 101(5):1136-1159. doi: 10.1109/JPROC.2012.2237151 URL
[2]	DAVIS S, MERMELSTEIN P. Comparison of parametric representations for monosyllabic word recognition in continuously spoken sentences[J]. IEEE Transactions on Acoustics, Speech and Signal Processing, 1980, 28(4):357-366. doi: 10.1109/TASSP.1980.1163420 URL
[3]	TORRES-CARRASQUILLO P A, SINGER E, KOHLER M A, et al. Approaches to language identification using Gaussian mixture models and shifted delta cepstral features[C]//Proceedings of the 7th International Confrence on Spoken Language Processing. Piscataway, NJ, US: IEEE, 2002: 89-92.
[4]	HEMANSK H. Perceptual linear predictive (PLP) analysis of speech[J]. The Journal of the Acoustical Society of America, 1990, 87(4):1738-1752. doi: 10.1121/1.399423 URL
[5]	张卫强, 刘加. 基于听感知特征的语种识别[J]. 清华大学学报(自然科学版), 2009, 49(1):78-81.
	ZHANG W Q, LIU J. Language recognition based on auditory perception characteristics[J]. Journal of Tsinghua University (Natural Science Edition), 2009, 49(1): 78-81. (in Chinese)
[6]	CAMPBELL W M, STURIM DE, REYNODS D A. Support vector machines using GMM supervectors for speaker's verification[J]. IEEE Signal Processing Letters, 2006, 13(5):308-311. doi: 10.1109/LSP.2006.870086 URL
[7]	REYNOLDSD A, QUATIERI T F, DUNN R B. Speaker verification using adapted Gaussian mixture models[J]. Digital Signal Processing, 2000, 10(123):19-41. doi: 10.1006/dspr.1999.0361 URL
[8]	ZISSMAN M A. COMPARISON of four approaches to automatic language identification of telephone speech[J]. IEEE Transactions on Speech and Audio Processing, 1996, 4(1):31-38. doi: 10.1109/TSA.1996.481450 URL
[9]	YAN Y H, BARNARD E. An approach to automatic language identification based on language-dependent phone identification[C]//Proceedings of IEEE InternationalConfrence on Acoustics, Speech and Signal Processing. Piscataway, NJ, US: IEEE, 1995:3511-3514.
[10]	DEHAK N, TORRES P A, RETNOLDS D A, et al. Language Identification via I-Vectors and Dimensionality Reduction[C]// Proceedings of Interspeech, Conference of the International Speech Communication Association. Florence, Italy:DBLP, 2011:857-860.
[11]	JIANG B, SONG Y, WEI S, et al. Performance evaluation of deep bottleneck features for spoken language identification[C]// Proceedings of International Symposium on Chinese Spoken Language Processing. Singapore:IEEE, 2014:143-147.
[12]	MONTAVON G. Deep learning for spoken language identification[J]. NIPS Workshop on Deep Learning for Speech Recognition and Related Applications, 2009, 49(10):911-914.
[13]	LOPEZ I, GONZALEZ J, PLCHOT O, et al. Automatic language identification using deep neural networks[C]//Proceedings of IEEE International Conference on Acoustics. Florence, Italy: IEEE, 2014:5337-5341.
[14]	GARCIA R D, MCCREE A. Stacked long-term TDNN for spoken language identification[C]//Proceedings of the 17th Annual Conf of the Int Speech Communication Association. Baixas, France: International Speech and Communication Association, 2016: 3226-3230.
[15]	HE K, ZHANG X, REN S, et al. Deep residual learning for image recognition[C]//Proceedings of 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Las Vegas,NE, US: IEEE Computer Society, 2016:770-778.
[16]	WANG G, WANG W F, ZHAO Y Y, et al. End-to-end language identification using attention-based recurrent neural networks[C]//Proceedings of the 17th Annual Conf of the Int Speech Communication Association. Baixas, France: International Speech and Communication Association, 2016:2944-2948.
[17]	JIN M, SONG Y, MCLOUGHLIN I, et al. LID-senones and their statistics for language identification[J]. IEEEACM Transactions on Audio, Speech, and Language Processing, 2019, 26(1):171-183.
[18]	CAI W C, CAI Z X, LIU W B, et al. Insights into end-to-end learning scheme for language identification[J]. IEEE Signal Processing Society Sigport, 2018, 28(2):202-210.
[19]	DESHWAL D, SANGWAN P, KUMAR D. Feature extraction methods in language identification: a survey[J]. Wireless Personal Communications, 2019, 107(4): 2071-2103. doi: 10.1007/s11277-019-06373-3
[20]	LI L, LI Z, LIU Y, et al. Deep joint learning for language recognition[J]. Neural Networks: The Official Journal of the International Neural Network Society, 2021, 141(9): 72-86. doi: 10.1016/j.neunet.2021.03.026 URL
[21]	BHANJA C C, LASKAR M A, LASKAR R H. Modelling multi-level prosody and spectral features using deep neural network for an automatic tonal and non-tonal preclassification-based Indian language identification system[J]. Language Resources and Evaluation, 2021, 55(3):689-730. doi: 10.1007/s10579-020-09527-z
[22]	刘威. 单通道语音水印与语音增强算法研究[D]. 南京: 东南大学, 2017.
	LIU W. Research on single channel speech watermarking and speech enhancement algorithm[D]. Nanjing: Southeast University, 2017. (in Chinese)
[23]	TAWHID M N A, SIULY S, WANG H. Diagnosis of autism spectrum disorder from EEG using a time-frequency spectrogram image-based approach[J]. Electronics Letters, 2020, 56(25):1372-1375. doi: 10.1049/ell2.v56.25 URL
[24]	HIMANSHU C. Time-frequency representations: spectrogram, cochleogram and correlogram[J]. Procedia Computer Science, 2020, 16(7):1901-1910.
[25]	马元锋, 陈克安, 马苗, 等. 一种新的可应用于声目标识别的倒谱系数[J]. 兵工学报, 2009, 30(11):1477-1483.
	MA Y F, CHEN K A, MA M, et al. A new cepstral coefficient applicable to sound target recognition[J]. Acta Armamentarii, 2009, 30(11): 1477-1483. (in Chinese)
[26]	冯红波, 李萍, 李波. 基于自动色阶和多尺度Retinex彩色图像增强算法[J]. 无线电工程, 2019, 49(10):911-914.
	FENG H B, LI P, LI B. Based on automatic color scale and multi-scale Retinex color image enhancement algorithm[J]. Radio Engineering, 2019, 49(10): 911-914. (in Chinese)
[27]	ZHU D, HUANG M, YANG J J, et al. Identificationof spoken language from webcast using deep convolutionalrecurrent neural networks[C]//Proceedings of 2019 International Conference on Information Technology. Sanya, China: IEEE, 2019:1147-1152.
[28]	邵玉斌, 刘晶, 龙华, 等. 基于声道频谱参数的语种识别[J]. 北京邮电大学学报, 2021, 44(3):112-119. doi: 10.13190/j.jbupt.2020-228
	SHAOYB, LIU J, LONG H, et al. Language recognition based on vocal tract spectral parameters[J]. Journal of Beijing University of Posts and Telecommunications, 2021, 44(3): 112-119. (in Chinese)

α	β
α	0.30	0.35	0.40
0.50	73.74	74.68	76.18
0.45	72.72	79.62	78.70
0.40	72.92	73.12	73.12

α	β
α	0.30	0.35	0.40
0.50	73.74	74.68	76.18
0.45	72.72	79.62	78.70
0.40	72.92	73.12	73.12

噪声源	识别方法	SNR/dB
噪声源	识别方法	-10	-5	0	5	10
	Fbank	20.0	26.3	32.5	63.6	71.3
	LGSS	20.0	22.8	55.5	68.3	73.7
	DBF	20.7	22.5	68.4	79.7	83.7
WN	FRSCIRF	37.7	64.0	76.7	81.7	85.7
	TGSS	22.5	40.7	64.5	80.0	83.5
	FTGSS	26.6	42.3	69.7	80.1	85.1
	FTGSSE	61.2	79.8	83.2	86.0	87.9
	LGSS	20.0	20.1	24.7	44.9	65.7
DORBN	TGSS	20.4	27.8	47.7	73.2	80.4
	FTGSS	20.7	44.2	72.9	79.8	82.6
	FTGSSE	42.6	76.0	81.6	84.4	86.9
	LGSS	19.5	23.5	36.7	62.3	70.7
MVN	TGSS	22.7	30.1	41.3	67.5	74.1
	FTGSS	27.0	38.0	45.1	70.9	80.5
	FTGSSE	37.8	51.2	77.7	84.9	87.2
	LGSS	18.0	27.9	50.2	64.9	73.9
HFCN	TGSS	20.1	34.9	59.2	72.7	78.3
	FTGSS	21.9	40.2	65.1	79.4	84.5
	FTGSSE	62.2	79.0	81.4	85.1	87.0
	LGSS	19.8	31.3	41.5	68.8	73.2
PN	TGSS	21.6	35.2	44.7	70.9	78.4
	FTGSS	22.8	39.6	47.6	72.1	80.3
	FTGSSE	37.3	47.0	77.0	85.4	86.9
	LGSS	53.4	62.1	68.6	69.5	70.9
VN	TGSS	58.2	65.5	71.9	74.3	75.8
	FTGSS	62.0	69.2	75.4	78.7	79.7
	FTGSSE	48.3	67.8	83.5	86.9	88.6
	LGSS	19.7	22.1	21.4	36.9	66.6
F16CN	TGSS	22.6	28.1	34.6	54.7	70.3
	FTGSS	23.4	31.9	45.6	65.6	78.2
	FTGSSE	35.3	43.8	65.1	83.7	86.4
	LGSS	14.1	32.1	41.9	64.2	79.7
BFCN	TGSS	21.5	36.4	43.6	68.1	80.1
	FTGSS	24.9	40.7	47.3	78.9	80.3
	FTGSSE	60.1	78.1	80.4	85.7	86.7
	LGSS	52.5	62.0	65.9	69.2	71.2
MGN	TGSS	53.6	63.4	66.9	69.7	72.3
	FTGSS	54.2	63.7	66.3	71.3	75.9
	FTGSSE	49.3	59.5	67.6	75.3	83.5

噪声源	识别方法	SNR/dB
噪声源	识别方法	-10	-5	0	5	10
	Fbank	20.0	26.3	32.5	63.6	71.3
	LGSS	20.0	22.8	55.5	68.3	73.7
	DBF	20.7	22.5	68.4	79.7	83.7
WN	FRSCIRF	37.7	64.0	76.7	81.7	85.7
	TGSS	22.5	40.7	64.5	80.0	83.5
	FTGSS	26.6	42.3	69.7	80.1	85.1
	FTGSSE	61.2	79.8	83.2	86.0	87.9
	LGSS	20.0	20.1	24.7	44.9	65.7
DORBN	TGSS	20.4	27.8	47.7	73.2	80.4
	FTGSS	20.7	44.2	72.9	79.8	82.6
	FTGSSE	42.6	76.0	81.6	84.4	86.9
	LGSS	19.5	23.5	36.7	62.3	70.7
MVN	TGSS	22.7	30.1	41.3	67.5	74.1
	FTGSS	27.0	38.0	45.1	70.9	80.5
	FTGSSE	37.8	51.2	77.7	84.9	87.2
	LGSS	18.0	27.9	50.2	64.9	73.9
HFCN	TGSS	20.1	34.9	59.2	72.7	78.3
	FTGSS	21.9	40.2	65.1	79.4	84.5
	FTGSSE	62.2	79.0	81.4	85.1	87.0
	LGSS	19.8	31.3	41.5	68.8	73.2
PN	TGSS	21.6	35.2	44.7	70.9	78.4
	FTGSS	22.8	39.6	47.6	72.1	80.3
	FTGSSE	37.3	47.0	77.0	85.4	86.9
	LGSS	53.4	62.1	68.6	69.5	70.9
VN	TGSS	58.2	65.5	71.9	74.3	75.8
	FTGSS	62.0	69.2	75.4	78.7	79.7
	FTGSSE	48.3	67.8	83.5	86.9	88.6
	LGSS	19.7	22.1	21.4	36.9	66.6
F16CN	TGSS	22.6	28.1	34.6	54.7	70.3
	FTGSS	23.4	31.9	45.6	65.6	78.2
	FTGSSE	35.3	43.8	65.1	83.7	86.4
	LGSS	14.1	32.1	41.9	64.2	79.7
BFCN	TGSS	21.5	36.4	43.6	68.1	80.1
	FTGSS	24.9	40.7	47.3	78.9	80.3
	FTGSSE	60.1	78.1	80.4	85.7	86.7
	LGSS	52.5	62.0	65.9	69.2	71.2
MGN	TGSS	53.6	63.4	66.9	69.7	72.3
	FTGSS	54.2	63.7	66.3	71.3	75.9
	FTGSSE	49.3	59.5	67.6	75.3	83.5

噪声源	识别方法	SNR/dB
噪声源	识别方法	-10	-5	0	5	10
WN	LGSS	0.14	0.18	0.52	0.58	0.63
	FTGSSE	0.55	0.79	0.83	0.86	0.88
DORBN	LGSS	0.13	0.16	0.21	0.39	0.61
	FTGSSE	0.39	0.76	0.81	0.84	0.86
MVN	LGSS	0.15	0.21	0.32	0.58	0.65
	FTGSSE	0.37	0.49	0.77	0.85	0.87
HFCN	LGSS	0.13	0.24	0.47	0.62	0.71
	FTGSSE	0.59	0.78	0.80	0.85	0.87
PN	LGSS	0.17	0.28	0.36	0.64	0.67
	FTGSSE	0.36	0.44	0.77	0.85	0.87
VN	LGSS	0.44	0.58	0.64	0.64	0.67
	FTGSSE	0.47	0.67	0.83	0.87	0.88
F16CN	LGSS	0.16	0.20	0.23	0.35	0.62
	FTGSSE	0.33	0.45	0.67	0.84	0.86
BFCN	LGSS	0.11	0.31	0.39	0.61	0.78
	FTGSSE	0.54	0.80	0.78	0.85	0.86
MGN	LGSS	0.50	0.60	0.63	0.65	0.67
	FTGSSE	0.45	0.59	0.68	0.76	0.84