基于机器视觉与深度学习的飞机防护栅裂纹检测系统

doi:10.12382/bgxb.2021.0674

摘要/Abstract

摘要：

针对传统飞机防护栅裂纹检测中存在的效率低、可靠性差等问题,基于机器视觉技术设计一种飞机防护栅裂纹检测装置,并结合图像处理技术与深度学习原理提出一种飞机防护栅裂纹检测算法。设计飞机防护栅裂纹检测系统,研究防护栅裂纹图像识别算法。采集并整理飞机防护栅裂缝图像,研究并制作飞机防护栅裂纹检测数据集;分别以ZF-Net、VGG-16和ResNet-101卷积神经网络作为Faster-RCNN特征提取网络,开展飞机防护栅表面裂纹和缺陷裂纹检测研究。实验结果表明:3种模型均达到了良好的检测精度,其检测精度分别为92.79%、95.12%和97.54%,其中ResNet-101网络检测效果最好,相比于现有的防护栅裂纹机器视觉检测方法,漏检率和虚警率分别下降了22.54%和89.28%,检出率提高了22.54%;ResNet-101网络在不同光照条件下仍有较高的检测精度,检测装置和检测算法有效,可为飞机防护栅的检测提供了新方法。

关键词: 飞机防护栅, 裂纹检测, 机器视觉, 深度学习, 卷积神经网络

Abstract:

To address the problems of low efficiency and poor reliability in the crack detection of traditional aircraft protective grill, a crack detection device is designed based on machine vision technology. Combined with image processing technology and deep learning principles, a crack detection and calculation method for aircraft protective grill is proposed. Firstly, a detection system is designed, and the image recognition algorithm of protective grill is studied, and then, the crack images of aircraft protective grill are collected and sorted, and the crack detection data set is studied and made. Secondly, the ZF-Net, VGG-16 and ResNet-101 convolutional neural networks are used as the feature extraction networks of Faster-RCNN to detect surface cracks and defect cracks of the aircraft protective grill. The experimental results show that: the three models can achieve good detection accuracy, which are 92.79%,95.12% and 97.54% respectively; the Resnet-101 network has the best detection effect; compared with the existing machine vision detection method for protective grill cracks, the missed detection rate and false alarm rate are reduced by 22.54% and 89.28% respectively, and the detection rate is improved by 22.54%. Further research shows that the ResNet-101 network still has high detection accuracy under different lighting conditions, which shows the effectiveness of the detection device and detection algorithm. This research provides a new method for crack detection of the aircraft protective grill.

Key words: aircraft protective grid, crack detection, machine vision, deep learning, convolutional neural network

中图分类号:

TP391.4

张良安, 陈洋, 谢胜龙, 刘同鑫. 基于机器视觉与深度学习的飞机防护栅裂纹检测系统[J]. 兵工学报, 2023, 44(2): 507-516.

ZHANG Liang'an, CHEN Yang, XIE Shenglong, LIU Tongxin. Crack Detection System for Aircraft Protective Grill based on Machine Vision and Deep Learning[J]. Acta Armamentarii, 2023, 44(2): 507-516.

图/表 21

图1 飞机防护栅结构示意图

Fig.1 Protective grill structure

图2 检测装置

Fig.2 Detection device

图3 防护栅放置平台

Fig.3 Protective grill placement platform

图4 相机搭载装置

Fig.4 Camera mounting device

表1 设备硬件参数

Table 1 Device hardware parameters

硬件	型号	数量	主要参数	厂商
工业相机	MV-GEA2000M-T	8个	分辨率5488×3672,有效像素2000万	迈德威视
远心镜头	NML044-SR130VI-18C	8个	光学倍率0.44,焦距854.53mm	茉丽特
环形漫射光源	RL74-30	8个	外圈直径ϕ60mm,内圈直径ϕ44mm	迈点
工控机	BOX-T8162-02	5台	I7-6700/8G	特控
交换机	TL-SG1016DT	1个	16口千兆交换机	TPLINK

图5 防护栅栅格示意图(放大10倍)

Fig.5 Schematic diagram of protective grill (10×)

图6 防护栅裂纹检测方法流程

Fig.6 Crack detection process of the protective grill

图7 图像处理前后对比

Fig.7 Comparison before and after image processing

图8 防护栅二值化图

Fig.8 Diagram of protective grill binarization

图9 高级形态学处理前后对比

Fig.9 Comparison before and after advanced morphological processing

图10 粒子过滤前后对比

Fig.10 Comparison before and after particle filtration

图11 区域坐标示意图

Fig.11 Schematic diagram of area coordinates

表2 网络结构参数

Table 2 Network structure parameters

模型	卷积层	通道数量	卷积核	池化层	全连接层
ZF-Net	5	96-256-384 -384 -256	7×7 5×5 3×3	3	4096-4096- 1000
VGG-16	13	64×2-128× 2-256 3-512×3	3×3	5	4096-4096- 1000
ResNet- 101	5	256-512- 1024- 2048	7×7 3×3 1×1	5	2048-2048- 1000

图12 数据扩充

Fig.12 Data expansion

图13 各模型在训练时的损失值

Fig.13 Loss of each model during training

图14 各模型在验证集上的准确率曲线

Fig.14 Accuracy curves of each model on the validation set

表3 不同裂纹检测算法实验结果

Table 3 Experimental results of different crack detection algorithms

方法	漏检率/%	虚警率/%	检出率/%
文献[1]方法	25	91	75
ZF-Net	7.21	4.01	92.79
VGG-16	4.88	3.22	95.12
ResNet-101	2.46	1.72	97.54

图15 4种算法实验结果对比

Fig.15 Comparison of the experimental results of the four algorithms

表4 不同数据集下的实验结果

Table 4 Experimental results under different data sets%

方法	验证集			弱光照			正常光照			强光照
方法	漏检率	虚警率	检出率	漏检率	虚警率	检出率	漏检率	虚警率	检出率	漏检率	虚警率	检出率
ZF-Net	7.21	4.01	92.79	10.00	5.26	90.00	6.00	3.01	94.00	6.00	4.08	94.00
VGG-16	4.88	3.22	95.12	6.00	4.08	94.00	4.00	3.03	96.00	5.00	3.06	95.00
ResNet-101	2.46	1.72	97.54	3.00	2.02	97.00	2.00	1.01	98.00	2.00	2.00	98.00

图16 不同光照下ResNet-101网络的检测效果

Fig.16 Detection results of ResNet-101 network under different illumination conditions

表5 不同裂纹检测方法性能对比

Table 5 Performance comparison of different crack detection methods

方法	检测效率	检测精度	可靠性
人工检查	低	低	低
文献[1]方法	中	中	低
本文检测系统	高	高	高

参考文献 18

[1]	曹强, 曾晓利, 程宗辉, 等. 基于机器视觉的飞机防护栅无损检测系统[J]. 无损检测, 2016, 38(2):24-27,51.
	CAO Q, ZENG X L, CHENG Z H, et al. Nondestructive testing system for airplane protective griling based on machine vision[J]. Nondestructive Testing, 2016, 38(2):24-27,51. (in Chinese)
[2]	李国华. 某型飞机防护栅套管固定螺栓断裂原因分析[J]. 科技创新导报, 2015, 12(13):44-45.
	LI G H. Analysis of the causes of the fracture of the fixing bolt of a certain aircraft protective grid casing[J]. Science and Technology Innovation Herald, 2015, 12(13): 44-45. (in Chinese)
[3]	刘平政, 宋凯, 宁宁, 等. 飞机紧固件孔周裂纹检测远场涡流传感器设计及优化[J]. 仪器仪表学报, 2019, 40(6):1-8.
	LIU P Z, SONG K, NING N, et al. Design and optimization of a far-field eddy current sensor for crack detection around the hole of aircraft fasteners[J]. Chinese Journal of Scientific Instrument, 2019, 40(6):1-8. (in Chinese)
[4]	TAVARES S M O, CASTRO P M S T. An overview of fatigue in aircraft structures[J]. Fatigue & Fracture of Engineering Materials & Structures, 2017, 40(10):1510-1529.
[5]	寇光杰, 杨正伟, 贾庸, 等. 复杂型面叶片裂纹的超声红外热成像检测[J]. 红外与激光工程, 2019, 48(12):101-109.
	KOU G J, YANG Z W, JIA Y, et al. Ultrasonic infrared thermal imaging detection of cracks in complex-shaped blades[J]. Infrared and Laser Engineering, 2019 48(12):101-109. (in Chinese)
[6]	黄凤英. 钢轨表面裂纹涡流检测定量评估方法[J]. 中国铁道科学, 2017, 38(2):28-33.
	HUANG F Y. Quantitative evaluation method for eddy current detection of rail surface cracks[J]. China Railway Science, 2017, 38(2):28-33. (in Chinese)
[7]	UCHANIN V, OSTASH O. Development of electromagnetic NDT methods for structural integrity assessment[J]. Procedia Structural Integrity, 2019, 16:192-197. doi: 10.1016/j.prostr.2019.07.040 URL
[8]	CHA Y J, CHOI W, BÜYÜKÖZTÜRK O. Deep learning-based crack damage detection using convolutional neural networks[J]. Computer-Aided Civil and Infrastructure Engineering, 2017, 32(5):361-378. doi: 10.1111/mice.2017.32.issue-5 URL
[9]	PRIYADUMKOL J, KITTICHAIKARN C, THAINIMIT C. Crack detection on unwashed eggs using image processing[J]. Journal of Food Engineering, 2017, 209: 76-82. doi: 10.1016/j.jfoodeng.2017.04.015 URL
[10]	陈波, 张华, 汪双, 等. 基于全卷积神经网络的坝面裂纹检测方法研究[J]. 水力发电学报, 2020, 39(7):52-60.
	CHEN B, ZHANG H, WANG S, et al. Research on dam surface crack detection method based on full convolutional neural network[J]. Journal of Hydroelectric Engineering, 2020, 39(7):52-60. (in Chinese)
[11]	ZHENG Z, QI H Y, ZHUANG L, et al. Automated rail surface crack analytics using deep data-driven models and transfer learning[J]. Sustainable Cities and Society, 2021, 70: 102898. doi: 10.1016/j.scs.2021.102898 URL
[12]	王森, 伍星, 张印辉, 等. 基于深度学习的全卷积网络图像裂纹检测[J]. 计算机辅助设计与图形学学报, 2018, 30(5):859-867.
	WANG S, WU X, ZHANG Y H, et al. Fully convolutional network image crack detection based on deep learning[J]. Journal of Computer Aided Design and Graphics, 2018, 30(5):859-867. (in Chinese)
[13]	吕帅帅, 杨宇, 王彬文, 等. 基于改进Mask-RCNN的飞行器结构裂纹自动检测方法[J]. 振动.测试与诊断, 2021, 41(3):487-494.
	LÜ S S, YANG Y, WANG B W, et al. Automatic detection method of aircraft structure cracks based on improved Mask-RCNN[J]. Vibration. Test and Diagnosis, 2021, 41(3):487-494. (in Chinese)
[14]	申铉京, 张赫, 陈海鹏, 等. 快速递归多阈值分割算法[J]. 吉林大学学报(工学版), 2016, 46(2):528-534.
	SHEN X J, ZHANG H, CHEN H P, et al. Fast recursive multi-threshold segmentation algorithm[J]. Journal of Jilin University (Engineering and Technology Edition), 2016, 46(2):528-534. (in Chinese)
[15]	张玉燕, 李永保, 温银堂, 等. 基于Faster R-卷积神经网络的金属点阵结构缺陷识别方法[J]. 兵工学报, 2019, 40(11):2329-2335. doi: 10.3969/j.issn.1000-1093.2019.11.018
	ZHANG Y Y, LI Y B, WEN Y T, et al. Internal defect detection of metal three-dimensional multi-layer lattice structure based on Faster R-CNN[J]. Acta Armamentarii, 2019, 40(11):2329-2335. (in Chinese) doi: 10.3969/j.issn.1000-1093.2019.11.018
[16]	孙皓泽, 常天庆, 王全东, 等. 一种基于分层多尺度卷积特征提取的坦克装甲目标图像检测方法[J]. 兵工学报, 2017, 38(9):1681-1691. doi: 10.3969/j.issn.1000-1093.2017.09.003
	SUN H Z, CHANG T Q, WANG Q D, et al. Image detection method for tank and armored targets based on hierarchical multi-scale convolution feature extraction[J]. Acta Armamentarii, 2017, 38(9):1681-1691. (in Chinese)
[17]	王伟平, 王琦, 于洋. 基于注意力机制与深度学习算法的机床主轴系统故障辨识[J]. 兵工学报, 2022, 43(4):861-875. doi: 10.12382/bgxb.2021.0202
	WANG W P, WANG Q, YU Y. Fault identification of machine tool spindle system based on attention mechanism and deep learning algorithm[J]. Acta Armamentarii, 2022, 43(4):861-875. (in Chinese) doi: 10.12382/bgxb.2021.0202
[18]	曹磊, 王强, 史润佳, 等. 基于改进RPN的Faster-RCNN网络SAR图像车辆目标检测方法[J]. 东南大学学报(自然科学版), 2021, 51(1):87-91.
	CAO L, WANG Q, SHI R J, et al. Faster-RCNN network SAR image vehicle target detection method based on improved RPN[J]. Journal of Southeast University (Natural Science Edition), 2021, 51(1):87-91. (in Chinese)