融合事件的点线特征法视觉惯性里程计

doi:10.12382/bgxb.2023.1221

摘要/Abstract

摘要：

视觉惯性里程计是机器人实现自主定位的关键技术,事件相机作为一种异步视觉传感器,与传统相机具有互补的特点。针对低光照、光照大幅度变化和高速运动场景,对事件相机的输出和传统图像进行融合,并结合惯性测量单元进行实时点线特征法视觉惯性里程计研究。提出一种从事件流生成事件图像的算法,设计融合事件的点线特征检测方法;基于视觉-惯性紧耦合的思想,设计后端滑动窗口优化算法;进行数据集试验验证和无人机飞行试验验证。在数据集上的试验结果表明:与仅使用传统图像的点线特征法视觉惯性里程计相比,在高速运动的场景下,定位误差平均减少了22%以上;在低光照、光照大幅度变化的场景下,定位误差平均减少了59%以上。

关键词: 事件相机, 点线特征, 视觉惯性里程计, 视觉同时定位与地图构建, 位姿估计

Abstract:

Visual-inertial odometry is a key technology for robots to achieve autonomous localization. As an asynchronous vision sensor, the event cameras have complementary to the traditional cameras. For the scene of low light condition, high dynamic range and high-speed motion, the output of event camera and the traditional image are fused.A real-time visual inertial odometry using point and line features is proposed combined with the inertial measurement unit (IMU). An algorithm for generating an event image from event stream is proposed, a point-line feature detection method combined with events is designed, anda back-end sliding window optimization algorithm is designed based on the idea of visual-inertial tight-coupling. The dataset test and UAV flight test are conducted. The test results on the dataset show that, compared with the visual-inertial odometry using point and line features only on the traditional image, the proposed odometry can reduce the positioning error by more than 22% on average in the scene of high-speed motion, and it can reducethe positioning error by more than 59% on average in the scene of low light condition and high dynamic range.

Key words: event camera, point and line features, visual-inertial odometry, visualsimultaneous localization and mapping, pose estimation

中图分类号:

TP249

刘毓敏, 蔡志浩, 孙家岭, 赵江, 王英勋. 融合事件的点线特征法视觉惯性里程计[J]. 兵工学报, 2024, 45(11): 3926-3937.

LIU Yumin, CAI Zhihao, SUN Jialing, ZHAO Jiang, WANG Yingxun. Event-combined Visual-inertial Odometry Using Point and Line Features[J]. Acta Armamentarii, 2024, 45(11): 3926-3937.

图/表 18

图1 算法整体框架

Fig.1 Overall framework of the proposed algorithm

图2 事件相机的输出机制

Fig.2 Output of event camera

图3 累积事件生成事件图像(白色圆圈:传统图像,黑色圆圈:事件)

Fig.3 Generating the images by aggregating events (white circle: traditional image, black circle: event)

图4 运动补偿示意图(黑色圆圈:事件,黑色方块:IMU数据)

Fig.4 The process of motion compensation (black circle: event, black square: IMU data)

图5 不同场景下的运动补偿效果

Fig.5 Motion compensation in different scenes

图6 特征检测示意图

Fig.6 Examples of feature detection

图7 优化变量因子图(左:仅视觉观测,右:视觉-惯性紧耦合)

Fig.7 Factor graphs (left: visual measurements only, right: tightly-coupled visual-inertial measurements)

表1 参数选取

Table 1 Parameters

参数	数值
累积事件个数	20000
传统图像中的点特征检测个数	50
事件图像中的点特征检测个数	50
点特征之间最小距离阈值	10
图像金字塔层数	2
线特征阈值限制	0.125

表2 不同里程计算法在UZH-RPG DAVIS 240C数据集上的定位均方误差比较

Table 2 Localization RMSEs of different odometries on UZH-RPG DAVIS 240C datasets m

序列名称	VINS-Mono算法	PL-VINS算法	Ultimate_SLAM算法	EVIO算法	本文算法
boxes_6dof	0.163	0.055	0.230	0.210	0.048
boxes_translation	0.162	0.085	0.187	0.176	0.059
poster_6dof	0.290	0.034	0.161	0.171	0.027
poster_translation	0.133	0.041	0.055	0.059	0.031
hdr_boxes	0.249	0.065	0.234	0.204	0.041
hdr_poster	0.342	0.099	0.373	0.172	0.025

图8 算法在poster_6dof上的运行结果(对比PL-VINS算法)

Fig.8 Resultsonposter_6dof (compared with PL-VINS algorithm)

图9 算法在hdr_poster上的运行结果(对比PL-VINS算法)

Fig.9 Resultson hdr_poster (compared with PL-VINS algorithm)

表3 运行耗时

Table 3 Execution time cost

模块	耗时/ms
事件流预处理	12.067
点特征检测与跟踪	3.980
线特征检测与跟踪	10.238
后端非线性优化	12.152

图10 四旋翼无人机飞行平台

Fig.10 Quadrotor flight platform

图11 四旋翼无人机飞行环境

Fig.11 Quadrotor flight environment

表4 不同飞行轨迹中本文算法与PL_VINS算法误差对比

Table 4 Estimated errors of the proposed algorithm and PL_VINS algorithm in different flight trajectories

轨迹形状	光照条件/ Lux	轨迹长度/ m	误差/m
轨迹形状	光照条件/ Lux	轨迹长度/ m	本文算法	PL-VINS
方形	145	26.715	0.088	0.081
八字	5~145	27.286	0.231	0.252
圆形	5~145	25.447	0.604	0.752

图12 方形轨迹

Fig.12 Square trajectory

图13 八字轨迹

Fig.13 8-shaped trajectory

图14 圆形轨迹

Fig.14 Circle trajectory

参考文献 22

[23]	WANG Y X, SHAO B, ZHANG C C, et al. REVIO: range-and event-based visual-inertial odometry for bio-inspired sensors[J]. Biomimetics, 2022, 7(4): 169.
[1]	张福斌, 张炳烁, 杨玉帅. 基于单目/IMU/里程计融合的SLAM算法[J]. 兵工学报, 2022, 43(11):2810-2818.
	ZHANG F B, ZHANG B S, YANG Y S. SLAM algorithm basedonmonocular/IMU/odometer fusion[J]. Acta Armamentarii, 2022, 43(11):2810-2818. (in Chinese)
[2]	GALLEGO G, DELBRUCK T, ORCHARD G, et al. Event-based vision: a survey[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2020, 44(1): 154-180.
[3]	邹浩东, 昌敏, 郭金库, 等. 基于事件的视觉SLAM综述[J]. 无人系统技术, 2023, 6(3):59-80.
	ZOU H D, CHANG M, GUO J K, et al. Review of event-based visual SLAM[J]. Unmanned Systems Technology, 2023, 6(3):59-80. (in Chinese)
[4]	FALANGA D, KLEBER K, SCARAMUZZA D. Dynamic obstacle avoidance for quadrotors with event cameras[J]. Science Robotics, 2020, 5(40): eaaz9712.
[5]	ZHOU Y, GALLEGO G, SHEN S J. Event-based stereo visual odometry[J]. IEEE Transactions on Robotics, 2021, 37(5): 1433-1450.
[6]	HIDALGO J, GALLEGO G, SCARAMUZZA D. Event-aided direct sparse odometry[C]// Proceedings of the 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition. New Orleans, LA,US:IEEE, 2022: 5781-5790.
[7]	GAO L, SU H, GEHRIG D, et al. A 5-point minimal solver for event camera relative motion estimation[C]// Proceedings of the 2023 IEEE/CVF International Conference on Computer Vision. Paris,France:IEEE, 2023: 8049-8059.
[8]	ZUBIC N, GEHRIG D, GEHRIG M, et al. From chaos comes order: ordering event representations for object recognition and detection[C]// Proceedings of the 2023 IEEE/CVF International Conference on Computer Vision. Paris,France:IEEE, 2023: 12846-12856.
[9]	SUN Z, MESSIKOMMER N, GEHRIG D, et al. ESS: learning event-based semantic segmentation from still images[C]// Proceedings of 17th European Conference on Computer Vision. Tel Aviv, Israel: Springer, 2022: 341-357.
[10]	MESSIKOMMER N, FANG C, GEHRIG M, et al. Data-driven feature tracking for event cameras[C]// Proceedings of the 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Vancouver, BC,Canada:IEEE, 2023: 5642-5651.
[11]	CHEN P Y, GUAN W P, LU P. ESVIO: event-based stereo visual inertial odometry[J]. IEEE Robotics and Automation Letters, 2023, 8(6):3661-3668.
[12]	MAHLKNECHT F, GEHRIG D, NASH J, et al. Exploring event camera-based odometry for planetary robots[J]. IEEE Robotics and Automation Letters, 2022, 7(4): 8651-8658.
[13]	FU Q, WANG J L, YU H S, et al. PL-VINS: real-time monocular visual-inertial SLAM with point and line features:arXiv:2009.07462[R/OL]. Ithaca, NY, US: Cornell University, 2020(2020-09-16). https://arxiv.org/abs/2009.07462.
[14]	LIM H, JEON J, MYUNG H. UV-SLAM: unconstrained line-based SLAM using vanishing points for structural mapping[J]. IEEE Robotics and Automation Letters, 2022, 7(2): 1518-1525.
[15]	LI H A, ZHAO J, BAZIN J C, et al. Quasi-globally optimal and near/true real-time vanishing point estimation in Manhattan world[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2020, 44(3):1503-1518.
[16]	XU B, WANG P, HE Y J, et al. Leveraging structural information to improve point line visual-inertial odometry[J]. IEEE Robotics and Automation Letters, 2022, 7(2): 3483-3490.
[17]	LEE J, PARK S Y. PLF-VINS: real-time monocular visual-inertial SLAM with point-line fusion and parallel-line fusion[J]. IEEE Robotics and Automation Letters, 2021, 6(4): 7033-7040.
[18]	QIN T, LI P L, SHEN S J. VINS-Mono: a robust and versatile monocular visual-inertial state estimator[J]. IEEE Transactions on Robotics, 2018, 34(4): 1004-1020.
[19]	FURGALE P, REHDER J, SIEGWART R. Unified temporal and spatial calibration for multi-sensor systems[C]// Proceedings of the 2013 IEEE/RSJ International Conference on Intelligent Robots and Systems. Tokyo, Japan:IEEE, 2013: 1280-1286.
[20]	MUEGGLER E, REBECQ H, GALLEGO G, et al. The event-camera dataset and simulator: event-based data for pose estimation, visual odometry, and SLAM[J]. The International Journal of Robotics Research, 2017, 36(2): 142-149.
[21]	VIDAL A, REBECQ H, HORSTSCHAEFER T. Ultimate SLAM? combining events, images, and IMU for robust visual SLAM in HDR and high-speed scenarios[J]. IEEE Robotics and Automation Letters, 2018, 3(2): 994-1001.
[22]	REBECQ H, HORSTSCHAEFER T, SCARAMUZZA D. Real-time visual-inertial odometry for event cameras using keyframe-based nonlinear optimization[C]// Proceedings of the 28th British Machine Vision Conference. London, UK: BMVA Press, 2017.