军用车辆广域实时成像算法及系统研究

doi:10.12382/bgxb.2024.0903

摘要/Abstract

摘要：

为解决坦克、装甲车等军用车辆在使用多相机探测系统进行环境感知时,多相机图像重叠区域小、视差大及实际应用场景中特征信息复杂多变导致广域成像困难的问题,提出一种基于棋盘格科尔多瓦大学增强现实(Chessboard Augmented Reality University of Cordoba,ChArUco)平板标定的广域实时成像算法,并设计一款军用车辆广域视觉增强系统,用于驾驶员在视野受限的密闭车厢内对车外环境进行实时成像。该算法利用ChArUco平板在受遮挡情况下仍可准确标定的特点,实现对小重叠多相机系统的精确标定,通过标定参数进行投影变换,避免图像特征信息的干扰,有效应对复杂特征场景,同时通过光流关系融合消除重叠区域的视差,并利用投影查找表和并行加速优化方法,实现广域实时成像。该算法部署于移动计算平台,形成完整的广域视觉增强系统。实验结果显示,该系统及算法能够适应多种场景的广域实时成像需求,提升军用车辆在复杂环境中的视觉感知能力。

关键词: 军用车辆, 图像拼接, 多相机标定, ChArUco平板, 优化加速, 系统设计

Abstract:

The wide-area imaging becomes difficult due to the small overlapping area of multi-camera images,large parallax,and complex and changeable feature information in practical application scenarios when a multi-camera detection system is used to provide environmental perception for military vehicles such as tanks and armored vehicles.A wide-area real-time imaging algorithm based on ChArUco board calibration is proposed,and a wide-area visual enhancement system for military vehicles is designed for drivers to perform real-time imaging of the external environment in a closed compartment with limited vision.This algorithm takes advantage of the characteristic that the ChArUco board can still be accurately calibrated under occlusion conditions to achieve the accurate calibration of a small-overlapping multi-camera system.Projection transformation is carried out through calibration parameters to avoid the interference of image feature information and effectively deal with complex feature scenarios.At the same time,the parallax in the overlapping area is eliminated by the fusion of optical flow relationships,and a wide-area real-time imaging is achieved by using the projection look-up table and parallel acceleration optimization method.This algorithm is deployed on a mobile computing platform to form a complete wide-area visual enhancement system.The experimental results show that this system and algorithm can meet the wide-area real-time imaging requirements of various scenarios and improve the visual perception ability of military vehicles in complex environments.

Key words: military vehicle, image stitching, multi-camera calibration, ChArUco board, optimization acceleration, system design

中图分类号:

TP391.4

吴靖, 林建华, 黄炅, 杨铮, 黄峰. 军用车辆广域实时成像算法及系统研究[J]. 兵工学报, 2025, 46(9): 240903-.

WU Jing, LIN Jianhua, HUANG Jiong, YANG Zheng, HUANG Feng. Research on Wide-area Real-time Imaging Algorithm and System for Military Vehicles[J]. Acta Armamentarii, 2025, 46(9): 240903-.

图/表 17

图1 算法流程

Fig.1 Algorithm flow chart

图2 ChArUco平板检测示例

Fig.2 ChArUco board detection example

图3 相机外参标定模型

Fig.3 Camera extrinsic calibration model

图4 ChArUco角点匹配示意图

Fig.4 ChArUco corner matching illustration

图5 估计光流关系(左为相邻两幅图像的重叠区域,右为相邻两幅图像重叠区域的光流关系)

Fig.5 Estimated optical flow relationship(the left is the overlapping area of two adjacent images,and the right is the optical flow relationship of overlapping area of two adjacent images)

表1 低重叠率场景算法对比

Table 1 Comparison of various algorithms for low overlap rate scenarios

重叠率/%	相机摆放夹角/(°)	基于特征点检测和匹配的拼接算法是否有效	本文算法是否有效
30	49	是	是
20	56	是	是
15	59	否	是
10	63	否	是

图6 重叠率为10%的场景拼接成像

Fig.6 Stitching of scenes with overlap rate of 10%

图7 特征稀疏场景算法验证

Fig.7 Validation of the proposed algorithm for feature sparse scenes

图8 特征分布不均场景算法验证

Fig.8 Verification of algorithms in scene with uneven feature distribution

图9 低照度场景噪声图像算法验证

Fig.9 Validation of the algorithm for noise images in low-light scenes

图10 复杂动态场景算法验证

Fig.10 Verification of algorithm in complex dynamic scenarios

图11 广域视觉增强系统组成

Fig.11 Composition of wide-area visual enhancement system

图12 算法优化加速流程

Fig.12 Optimized and accelerated process of the algorithm

表2 算法优化加速前后耗时对比

Table 2 Comparison of time consumptions before and after algorithm optimization and acceleration

算法步骤	优化前耗时/ms	优化后耗时/ms	加速比
数据传入设备端		3.12
图像投影	17.12	3.23	5.30
光流计算	105.57	14.73	7.17
图像融合	20.41	3.34	6.11
数据传回主机端		2.51
总体耗时	143.10	26.93	5.31

图13 算法及系统道路实验1

Fig.13 Algorithm and system road experiment 1

图14 算法及系统道路实验2

Fig.14 Algorithm and system road experiment 2

表3 各拼接算法的客观数据评价

Table 3 Objective data evaluation of various stitching algorithms

图像区域	AutoStitch		APAP		本文算法
图像区域	SSIM↑	PSNR↑	SSIM↑	PSNR↑	SSIM↑	PSNR↑
①	0.6555	17.9066	0.6124	16.4453	0.6841	19.1727
②	0.7027	19.4580	0.6288	17.9052	0.7704	21.0269
③	0.7224	18.7417	0.8017	23.3309	0.8052	23.3833
④	0.7379	20.4495	0.7294	20.0315	0.7929	23.1795

参考文献 21

[1]	PEI Z H, ZHAO W Y, HU L, et al. Factors affecting the situational awareness of armored vehicle occupants[J]. Sensors, 2024, 24(11):3688.
[2]	曹昭睿, 郝永平, 刘万成, 等. 紧凑折反式仿生复眼及图像快速拼接识别算法[J]. 兵工学报, 2022, 43(8):1845-1857.
	CAO Z R, HAO Y P, LIU W C, et al. Compact catadioptric bionic compound eye and fast image mosaic recognition algorithm[J]. Acta Armamentarii, 2022, 43(8):1845-1857. (in Chinese) doi: 10.12382/bgxb.2021.0393
[3]	CHEN J, YE X, GAO S L, et al. Planar wide-angle-imaging camera enabled by metalens array[J]. Optica, 2022, 9(4):431-437.
[4]	PAN R F, ZHANG Y, XU L, et al. Stitching videos from unstructured camera arrays with rectangular boundaries[J]. IEEE Access, 2024, 12:21777-21786.
[5]	QIAO K, WANG S K, JIAO G C, et al. Research on noise testing and reduction of low illumination imaging module[C]//Proceedings of the Fifth Symposium on Novel Optoelectronic Detection Technology and Application.Xi’an, China:SPIE, 2019, 11023:528-532.
[6]	FU M Y, LIANG H, ZHU C H, et al. Image stitching techniques applied to plane or 3-D models:a review[J]. IEEE Sensors Journal, 2023, 23(8):8060-8079.
[7]	裴红星, 刘金达, 葛佳隆, 等. 图像拼接技术综述[J]. 郑州大学学报(理学版), 2019, 51(4):1-10,29.
	PEI H X, LIU J D, GE J L, et al. Review of image stitching techniques[J]. Journal of Zhengzhou University(Natural Science Edition), 2019, 51(4):1-10,29. (in Chinese)
[8]	任海蕾. 基于多相机标定的全景合成方法研究[D]. 西安: 西安电子科技大学, 2019.
	REN H L. Research on panorama synthesis method based on multi-camera calibration[D]. Xi’an: Xidian University, 2019. (in Chinese)
[9]	郑文玲, 钱宏文, 杨文豪, 等. 基于多相机标定的全景实时拼接算法研究[J]. 电子设计工程, 2021, 29(17):165-169.
	ZHENG W L, QIAN H W, YANG W H, et al. Research on real-time panoramic mosaic algorithm based on multi-camera calibration[J]. Electronic Design Engineering, 2021, 29(17):165-169. (in Chinese)
[10]	CHEN H F, ZHUANG J L, LIU B Y, et al. Camera calibration method based on circular array calibration board[J]. Systems Science & Control Engineering, 2023, 11(1):2233562.
[11]	黄文文, 彭小红, 李丽圆, 等. 相机标定方法及进展研究综述[J]. 激光与光电子学进展, 2023, 60(16):9-19.
	HUANG W W, PENG X H, LI L Y, et al. Review of camera calibration methods and their progress[J]. Laser & Optoelectronics Progress, 2023, 60(16):9-19. (in Chinese)
[12]	段志瑜, 罗哉, 江文松, 等. 基于改进ArUco标签的位姿估计方法[J]. 计量学报, 2024, 45(3):364-371.
	DUAN Z Y, LUO Z, JANG W S, et al. Pose estimation method based on improved ArUco tags[J]. Acta Metrologica Sinica, 2024, 45(3):364-371. (in Chinese)
[13]	LIN W W, LIANG P D, LUO G T, et al. Research of online hand-eye calibration method based on ChArUco board[J]. Sensors, 2022, 22(10):3805.
[14]	ZHANG Z Y. Flexible camera calibration by viewing a plane from unknown orientations[C]//Proceedings of the Seventh IEEE International Conference on Computer Vision.Kerkyra, Greece:IEEE, 1999:666-673.
[15]	FISCHLER M A, BOLLES R C. Random sample consensus:a paradigm for model fitting with applications to image analysis and automated cartography[J]. Communications of the ACM, 1981, 24(6):381-395.
[16]	de JESUS RUBIO J. Stability analysis of the modified levenberg-marquardt algorithm for the artificial neural network training[J]. IEEE Transactions on Neural Networks and Learning Systems, 2020, 32(8):3510-3524.
[17]	夏丹, 周睿. 视差图像配准技术研究综述[J]. 计算机工程与应用, 2021, 57(2):18-27. doi: 10.3778/j.issn.1002-8331.2007-0040
	XIA D, ZHOU R. Survey of parallax image registration technology[J]. Computer Engineering and Applications, 2021, 57(2):18-27. (in Chinese) doi: 10.3778/j.issn.1002-8331.2007-0040
[18]	FARNEBACK G. Two-frame motion estimation based on polynomial expansion[C]//Proceedings of the Image Analysis:13th Scandinavian Conference.Berlin,Heidelberg, Germany:Springer, 2003:363-370.
[19]	BROWN M, LOWE D G. Automatic panoramic image stitching using invariant features[J]. International Journal of Computer Vision, 2007, 74(1):59-73.
[20]	ZARAGOZA J, CHIN T J, BROWN M S, et al. As-projective-as-possible image stitching with moving DLT[C]//Proceedings of 2013 IEEE Conference on Computer Vision and Pattern Recognition.Portland,OR, US:IEEE, 2013:2339-2346.
[21]	刘杰, 游品鸿, 田明, 等. 基于局部投影的视差图像拼接平滑优化[J]. 电子学报, 2022, 50(6):1451-1456. doi: 10.12263/DZXB.20200664
	LIU J, YOU P H, TIAN M, et al. Smooth optimization of parallax images mosaic based on local projection[J]. Acta Electronica Sinica, 2022, 50(6):1451-1456. (in Chinese) doi: 10.12263/DZXB.20200664