Smoke Screen Video Detection and Parameter Extraction Based on Convolutional Neural Network and Spatio-temporal Features

doi:10.12382/bgxb.2023.0595

Abstract

Abstract:

In order to enhance the operational effectiveness of jamming bomb on the battlefield, it is imperative to comprehend the fundamental characteristics and extreme deployment conditions of jamming bomb, and amend the smoke dispersion equation associated with jamming bomb. Nonetheless, given the inherent variability in the transparency and texture of jamming bomb during the operational deployment and its diverse appearance in different environmental contexts, there exists a significant challenge in accurately extracting the smoke's contour and motion features. To address this issue, a hybrid approach based on the convolutional neural networks and the spatiotemporal characteristics of smoke is proposed. The proposed method encompasses five distinct phases: Adjustment of contrast in YUV color space; Implementation of a frame difference method to detect the motion regions within the input video image sequence, employing a well-designed convolutional neural networks architecture to identify potential smoke regions within these motion regions; Utilization of the smoke's spatiotemporal characteristics to further discern potential smoke regions within each candidate area; Adoption of a support vector machine (SVM) classifier which employs the extracted features to classify real smoke regions from non-smoke regions; Extraction of smoke feature parameters. The experimental results show that the proposed model can be used to improve the accuracy of smoke recognition to at least 99.94%. Consequently, it effectively meets the requirements for adjusting the smoke dispersion equation during the operational deployment of jamming bombs, providing substantial support for the prototyping experiments and practical utilization of jamming bomb.

Key words: jamming bomb, smoke dispersion equation, convolutional neural network, spatio-temporal feature, parameter extraction

CLC Number:

GUO Aiqiang, LI Tianpeng, ZHU Xi, GUAN Zhichao, LI Men, DONG Hongyu, GAO Xinbao. Smoke Screen Video Detection and Parameter Extraction Based on Convolutional Neural Network and Spatio-temporal Features[J]. Acta Armamentarii, 2024, 45(8): 2478-2486.

Figures/Tables 10

References 33

[1]	李笑楠, 李天鹏, 张开创, 等. 多频谱发烟组件结构设计与遮蔽性能[J]. 兵工学报, 2023, 44(3): 910-918. doi: 10.12382/bgxb/2021.0818
	LI X N, LI T P, ZHANG K C, et al. Structural design and shielding performance of multi-spectral interfering smoke module[J]. Acta Armamentarii, 2023, 44(3): 910-918. (in Chinese) doi: 10.12382/bgxb/2021.0818
[2]	郭爱强, 高欣宝, 李天鹏. 烟幕弹对制导弹药的干扰效能评估现状[J]. 兵器装备工程学报, 2021, 42(12): 64-71.
	GUO A Q, GAO X B, LI T P. Current situation of smoke screen jamming efectiveness evaluation on guided munitions[J]. Journal of Ordnance Equipment Engineering, 2021, 42(12): 64-71. (in Chinese)
[3]	徐路程, 郝雪颖, 肖凯涛, 等. 爆炸型烟幕弹遮蔽效能仿真研究[J]. 兵工学报, 2020, 41(7): 1299-1306. doi: 10.3969/j.issn.1000-1093.2020.07.006
	XU L C, HAO X Y, XIAO K T, et al. Simulation study of screening efficiency of explosive smoke bomb[J]. Acta Armamentarii, 2020, 41(7): 1299-1306. (in Chinese) doi: 10.3969/j.issn.1000-1093.2020.07.006
[4]	顾有林, 陈国龙, 胡以华, 等. 气溶胶沉降扩散研究进展(特邀)[J]. 红外与激光工程, 2022, 51(7):11-21.
	GU Y L, CHEN G L, HU Y H, et al. Research progress on the deposition and diffusion of aerosols (invited)[J]. Infrared and Laser Engineering, 2022 ; 51(7):11-21. (in Chinese)
[5]	XU C Y S, ZHA B T, BAO J Q, et al. Analysis of temporal and spatial distribution characteristics of ammonium chloride smoke particles in confined spaces[J]. Defence Technology, 2022, 18(7): 1269-1280. doi: 10.1016/j.dt.2021.09.017
[6]	郭爱强, 高欣宝, 李天鹏, 等. 五帧差分法提取实测红外烟幕图像的特征参数[J]. 含能材料, 2021, 29(12):1144-1151.
	GUO A Q, GAO X B, LI T P, et. al. Five-frame difference method for extracting characteristic parameters from measured infrared smoke screen images[J]. Chinese Journal of Energetic Materials, 2021, 29(12):1144-1151. (in Chinese)
[7]	夏雪, 袁非牛, 章琳, 等. 从传统到深度:视觉烟雾识别、检测与分割[J]. 中国图象图形学报, 2019, 24(10): 1627-1647.
	XIA X, YUAN F N, ZHANG L, et al. From traditional methods to deep ones: review of visual smoke recognition, detection, and segmentation[J]. Journal of Image and Graphics, 2019, 24(10):1627-1647. (in Chinese)
[8]	HASHEMZADEH M, FARAJZADEH N, HEYDARI M. Smoke detection in video using convolutional neural networks and efficient spatio-temporal features[J]. Applied Soft Computing, 2022, 128: 109496.
[9]	TUNG T X, KIM J M. An effective four-stage smoke-detection algorithm using video images for early fire-alarm systems[J]. Fire Safety Journal, 2011, 46(5): 276-282.
[10]	APPANA D K, ISLAM R, KHAN S A, et al. A video-based smoke detection using smoke flow pattern and spatial-temporal energy analyses for alarm systems[J]. Information Sciences, 2017, 418/419: 91-101.
[11]	ZHAO Y Q, TANG G Z, XU M M. Hierarchical detection of wildfire flame video from pixel level to semantic level[J]. Expert Systems with Applications, 2015, 42(8): 4097-4104.
[12]	YUAN F N, SHI J T, XIA X, et al. High-order local ternary patterns with locality preserving projection for smoke detection and image classification[J]. Information Sciences, 2016, 372: 225-240.
[13]	FILONENKO A, HERNÁNDEZ D C. Fast smoke detection for video surveillance using CUDA[J]. IEEE Transactions on Industrial Informatics, 2018, 14(2):725-733.
[14]	MEHTA R, EGIAZARIAN K. Dominant rotated local binary patterns (DRLBP) for texture classification[J]. Pattern Recognition Letters, 2016,71:16-22.
[15]	YUAN F N, FANG Z J, WU S Q, et al. Real-time image smoke detection using staircase searching-based dual threshold AdaBoost and dynamic analysis[J]. IET Image Process, 2015, 9(10): 849-856.
[16]	BARMPOUTIS P, DIMITROPOULOS K, GRAMMALIDIS N. Smoke detection using spatio-temporal analysis, motion modeling and dynamic texture recognition[C]//Proceedings of the 22nd European Signal Processing Conference. Lisbon, Portugal: IEEE, 2014: 1078-1082.
[17]	石志. 基于深度字典学习的纹理分类研究[D]. 南昌: 南昌大学, 2023: 25-41.
	SHI Z. Research on texture classification based on deep dictionary learning[D]. Nanchang: Nanchang University, 2023: 25-41. (in Chinese)
[18]	YU J, RUI Y, TANG Y Y, et al. High-order distance-based multiview stochastic learning in image classification[J]. IEEE Transactions on Cybernetics, 2014, 44(12): 2431-2442. doi: 10.1109/TCYB.2014.2307862 pmid: 25415948
[19]	SATPATHY A, JIANG X D, ENG H L. LBP-based edge-texture features for object recognition[J]. IEEE Transactions on Image Processing, 2014, 23(5):1953-1964. doi: 10.1109/TIP.2014.2310123 pmid: 24690574
[20]	GU K, XIA Z F, QIAO J F, et al. Deep dual-channel neural network for image-based smoke detection[J] IEEE Transactions on Multimedia, 2020, 22(2): 311-323.
[21]	王国平. 基于深度学习的快速火焰检测[D]. 淮安: 淮阴工学院, 2022.
	WANG G P. Fast flame detection based on deep learning[D]. Huai'an: Huaiyin Institute of Technology, 2022. (in Chinese)
[22]	周兴华, 陈西江, 羊海东, 等. 动态卷积YOLOv5的视频火焰检测算法[J]. 测绘科学, 2023, 48(4):106-118.
	ZHOU X H, CHEN X J, YANG H D, et al. Video flame detection algorithm based on dynamic convolution YOLOv5[J]. Science of Surveying and Mapping, 2023, 48(4):106-118. (in Chinese)
[23]	何惜琴, 陈冬冬. 于YUV颜色模型与导向滤波的图像去雾算法[J]. 液晶与显示, 2021, 36(8):1166-1173.
	HE X Q, CHEN D D. Image defogging algorithm based on YUV color model and guided filtering[J]. Chinese Journal of Liquid Crystals and Displays, 2021, 36(8):1166-1173. (in Chinese)
[24]	GAUR A, SINGH A, KUMAR A, et al. Video flame and smoke based fire detection algorithms: a literature review[J]. Fire Technology, 2020, 56:1943-1980.
[25]	YUAN F N, ZHANG L, XIA X, et al. A wave-shaped deep neural network for smoke density estimation[J]. IEEE Transactions on Image Processing, 2020, 29: 2301-2313.
[26]	陈浩, 高欣宝, 李天鹏, 等. 烟幕初始云团最大半径数值模拟[J]. 含能材料, 2018, 26(10):820-827.
	CHEN H, GAO X B, LI T P, et al. Numerical simulation of maximum radius of initial cloud cluster of smoke screen[J]. Chinese Journal of Energetic Materials, 2018, 26(10): 820-827. (in Chinese)
[27]	SOUSA M J, MOUTINHO A, ALMEIDA M. Wildfire detection using transfer learning on augmented datasets[J]. Expert Systems with Applications, 2020, 142:112975.
[28]	HOSSEINI A, HASHEMZADEH M, FARAJZADEH N. UFS-Net: a unified flame and smoke detection method for early detection of fire in video surveillance applications using CNNs[J]. Journal of Computational Science, 2022, 61: 101638.
[29]	CALDERARA S, PICCININI P, CUCCHIARA R. Vision based smoke detection system using image energy and color information[J]. Machine Vision and Applications, 2011, 22:705-719.
[30]	谭永前, 曾凡菊. 基于凸包计算和小波变换的显著目标检测算法[J]. 兵器装备工程学报, 2023, 44(4):252-261.
	TAN Y Q, ZENG F J. A salient object detection algorithm based on convex hull calculation and wavelet transform[J]. Journal of Ordnance Equipment Engineering, 2023, 44(4):252-261. (in Chinese)
[31]	YUE Z X, LIU J. Some advances in theory and algorithm for SVM[J]. Advances in Applied Mathematics, 2021, 10(12): 4535-4541.
[32]	HASHEMZADEH M, ZADEMEHDI A. Fire detection for video surveillance applications using ICA K-medoids-based color model and efficient spatio-temporal visual features[J]. Expert Systems with Applications, 2019, 130: 60-78.
[33]	KHAN Z A, T HUSSAIN, ULLAH F U M, et al. Randomly initialized CNN with densely connected stacked autoencoder for efficient fire detection[J]. Engineering Applications of Artificial Intelligence, 2022, 116: 105403.

方法	优点	缺点
基于颜色和纹理特征^{[9⇓⇓⇓-13]}	1)实现简单; 2)对旋转变化的鲁棒性。	取决于图像分辨率。
基于运动特征^{[6,14⇓ -16]}	可以忽略场景中像烟幕状的物体。	1)对光线变化敏感; 2)对相机震动敏感。
基于动态纹理和图像能量特征^[17-18]	在高能量图像中,检测精度高。	1)执行复杂; 2)计算复杂度高; 3)对低能量背景敏感。
基于卷积神经网络^{[7,19⇓ -21]}	1)检测精度高; 2)自动特征提取。	1)对烟幕的运动特性缺乏关注; 2)计算复杂度高; 3)网络训练需要较多的数据。

方法	优点	缺点
基于颜色和纹理特征^{[9⇓⇓⇓-13]}	1)实现简单; 2)对旋转变化的鲁棒性。	取决于图像分辨率。
基于运动特征^{[6,14⇓ -16]}	可以忽略场景中像烟幕状的物体。	1)对光线变化敏感; 2)对相机震动敏感。
基于动态纹理和图像能量特征^[17-18]	在高能量图像中,检测精度高。	1)执行复杂; 2)计算复杂度高; 3)对低能量背景敏感。
基于卷积神经网络^{[7,19⇓ -21]}	1)检测精度高; 2)自动特征提取。	1)对烟幕的运动特性缺乏关注; 2)计算复杂度高; 3)网络训练需要较多的数据。

视频片段编号	文件名称	数据集	视频图像尺寸/像素	视频帧数	是否有烟幕
VC1	01_ballistic	训练	320×240	347	是
VC2	02_ballistic	训练	320×240	210	是
VC3	03_ballistic	训练	1080×1920	3005	是
VC4	04_ballistic	训练	1080×1920	1835	是
VC5	05_ballistic	训练	320×288	2953	是
VC6	06_ballistic	训练	1080×1920	630	是
VC7	07_ballistic	训练	1080×1920	1400	是
VC8	08_ballistic	训练	320×240	1726	是
VC9	09_ballistic	预测	320×240	1200	是
VC10	10_ballistic	预测	320×240	517	是
VC11	11_ballistic	预测	1080×1920	640	是
VC12	12_ballistic	预测	320×240	783	是

视频片段编号	文件名称	数据集	视频图像尺寸/像素	视频帧数	是否有烟幕
VC1	01_ballistic	训练	320×240	347	是
VC2	02_ballistic	训练	320×240	210	是
VC3	03_ballistic	训练	1080×1920	3005	是
VC4	04_ballistic	训练	1080×1920	1835	是
VC5	05_ballistic	训练	320×288	2953	是
VC6	06_ballistic	训练	1080×1920	630	是
VC7	07_ballistic	训练	1080×1920	1400	是
VC8	08_ballistic	训练	320×240	1726	是
VC9	09_ballistic	预测	320×240	1200	是
VC10	10_ballistic	预测	320×240	517	是
VC11	11_ballistic	预测	1080×1920	640	是
VC12	12_ballistic	预测	320×240	783	是

视频片段编号	a_TPR	a_TNR	a_FPR	a_Accuracy
VC1	55.53	44.47	0	100
VC2	45.71	54.29	0	100
VC3	41.18	48.50	10.32	89.68
VC4	53.27	46.73	0	100
VC5	82.35	17.65	0	100
VC6	86.69	13.31	0	100
VC7	88.72	11.28	0	100
VC8	63.16	26.80	10.04	89.96
VC9	83.81	16.19	0	100
VC10	81.13	13.60	5.27	94.73
VC11	89.14	10.86	0	100
VC12	83.40	16.60	0	100