Event-combined Visual-inertial Odometry Using Point and Line Features

doi:10.12382/bgxb.2023.1221

Abstract

Abstract:

Visual-inertial odometry is a key technology for robots to achieve autonomous localization. As an asynchronous vision sensor, the event cameras have complementary to the traditional cameras. For the scene of low light condition, high dynamic range and high-speed motion, the output of event camera and the traditional image are fused.A real-time visual inertial odometry using point and line features is proposed combined with the inertial measurement unit (IMU). An algorithm for generating an event image from event stream is proposed, a point-line feature detection method combined with events is designed, anda back-end sliding window optimization algorithm is designed based on the idea of visual-inertial tight-coupling. The dataset test and UAV flight test are conducted. The test results on the dataset show that, compared with the visual-inertial odometry using point and line features only on the traditional image, the proposed odometry can reduce the positioning error by more than 22% on average in the scene of high-speed motion, and it can reducethe positioning error by more than 59% on average in the scene of low light condition and high dynamic range.

Key words: event camera, point and line features, visual-inertial odometry, visualsimultaneous localization and mapping, pose estimation

CLC Number:

TP249

LIU Yumin, CAI Zhihao, SUN Jialing, ZHAO Jiang, WANG Yingxun. Event-combined Visual-inertial Odometry Using Point and Line Features[J]. Acta Armamentarii, 2024, 45(11): 3926-3937.

Figures/Tables 18

Fig.1 Overall framework of the proposed algorithm

Fig.2 Output of event camera

Fig.3 Generating the images by aggregating events (white circle: traditional image, black circle: event)

Fig.4 The process of motion compensation (black circle: event, black square: IMU data)

Fig.5 Motion compensation in different scenes

Fig.6 Examples of feature detection

Fig.7 Factor graphs (left: visual measurements only, right: tightly-coupled visual-inertial measurements)

Table 1 Parameters

参数	数值
累积事件个数	20000
传统图像中的点特征检测个数	50
事件图像中的点特征检测个数	50
点特征之间最小距离阈值	10
图像金字塔层数	2
线特征阈值限制	0.125

Table 2 Localization RMSEs of different odometries on UZH-RPG DAVIS 240C datasets m

序列名称	VINS-Mono算法	PL-VINS算法	Ultimate_SLAM算法	EVIO算法	本文算法
boxes_6dof	0.163	0.055	0.230	0.210	0.048
boxes_translation	0.162	0.085	0.187	0.176	0.059
poster_6dof	0.290	0.034	0.161	0.171	0.027
poster_translation	0.133	0.041	0.055	0.059	0.031
hdr_boxes	0.249	0.065	0.234	0.204	0.041
hdr_poster	0.342	0.099	0.373	0.172	0.025

Fig.8 Resultsonposter_6dof (compared with PL-VINS algorithm)

Fig.9 Resultson hdr_poster (compared with PL-VINS algorithm)

Table 3 Execution time cost

模块	耗时/ms
事件流预处理	12.067
点特征检测与跟踪	3.980
线特征检测与跟踪	10.238
后端非线性优化	12.152

Fig.10 Quadrotor flight platform

Fig.11 Quadrotor flight environment

Table 4 Estimated errors of the proposed algorithm and PL_VINS algorithm in different flight trajectories

轨迹形状	光照条件/ Lux	轨迹长度/ m	误差/m
轨迹形状	光照条件/ Lux	轨迹长度/ m	本文算法	PL-VINS
方形	145	26.715	0.088	0.081
八字	5~145	27.286	0.231	0.252
圆形	5~145	25.447	0.604	0.752

Fig.12 Square trajectory

Fig.13 8-shaped trajectory

Fig.14 Circle trajectory

References 22

[23]	WANG Y X, SHAO B, ZHANG C C, et al. REVIO: range-and event-based visual-inertial odometry for bio-inspired sensors[J]. Biomimetics, 2022, 7(4): 169.
[1]	张福斌, 张炳烁, 杨玉帅. 基于单目/IMU/里程计融合的SLAM算法[J]. 兵工学报, 2022, 43(11):2810-2818.
	ZHANG F B, ZHANG B S, YANG Y S. SLAM algorithm basedonmonocular/IMU/odometer fusion[J]. Acta Armamentarii, 2022, 43(11):2810-2818. (in Chinese)
[2]	GALLEGO G, DELBRUCK T, ORCHARD G, et al. Event-based vision: a survey[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2020, 44(1): 154-180.
[3]	邹浩东, 昌敏, 郭金库, 等. 基于事件的视觉SLAM综述[J]. 无人系统技术, 2023, 6(3):59-80.
	ZOU H D, CHANG M, GUO J K, et al. Review of event-based visual SLAM[J]. Unmanned Systems Technology, 2023, 6(3):59-80. (in Chinese)
[4]	FALANGA D, KLEBER K, SCARAMUZZA D. Dynamic obstacle avoidance for quadrotors with event cameras[J]. Science Robotics, 2020, 5(40): eaaz9712.
[5]	ZHOU Y, GALLEGO G, SHEN S J. Event-based stereo visual odometry[J]. IEEE Transactions on Robotics, 2021, 37(5): 1433-1450.
[6]	HIDALGO J, GALLEGO G, SCARAMUZZA D. Event-aided direct sparse odometry[C]// Proceedings of the 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition. New Orleans, LA,US:IEEE, 2022: 5781-5790.
[7]	GAO L, SU H, GEHRIG D, et al. A 5-point minimal solver for event camera relative motion estimation[C]// Proceedings of the 2023 IEEE/CVF International Conference on Computer Vision. Paris,France:IEEE, 2023: 8049-8059.
[8]	ZUBIC N, GEHRIG D, GEHRIG M, et al. From chaos comes order: ordering event representations for object recognition and detection[C]// Proceedings of the 2023 IEEE/CVF International Conference on Computer Vision. Paris,France:IEEE, 2023: 12846-12856.
[9]	SUN Z, MESSIKOMMER N, GEHRIG D, et al. ESS: learning event-based semantic segmentation from still images[C]// Proceedings of 17th European Conference on Computer Vision. Tel Aviv, Israel: Springer, 2022: 341-357.
[10]	MESSIKOMMER N, FANG C, GEHRIG M, et al. Data-driven feature tracking for event cameras[C]// Proceedings of the 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Vancouver, BC,Canada:IEEE, 2023: 5642-5651.
[11]	CHEN P Y, GUAN W P, LU P. ESVIO: event-based stereo visual inertial odometry[J]. IEEE Robotics and Automation Letters, 2023, 8(6):3661-3668.
[12]	MAHLKNECHT F, GEHRIG D, NASH J, et al. Exploring event camera-based odometry for planetary robots[J]. IEEE Robotics and Automation Letters, 2022, 7(4): 8651-8658.
[13]	FU Q, WANG J L, YU H S, et al. PL-VINS: real-time monocular visual-inertial SLAM with point and line features:arXiv:2009.07462[R/OL]. Ithaca, NY, US: Cornell University, 2020(2020-09-16). https://arxiv.org/abs/2009.07462.
[14]	LIM H, JEON J, MYUNG H. UV-SLAM: unconstrained line-based SLAM using vanishing points for structural mapping[J]. IEEE Robotics and Automation Letters, 2022, 7(2): 1518-1525.
[15]	LI H A, ZHAO J, BAZIN J C, et al. Quasi-globally optimal and near/true real-time vanishing point estimation in Manhattan world[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2020, 44(3):1503-1518.
[16]	XU B, WANG P, HE Y J, et al. Leveraging structural information to improve point line visual-inertial odometry[J]. IEEE Robotics and Automation Letters, 2022, 7(2): 3483-3490.
[17]	LEE J, PARK S Y. PLF-VINS: real-time monocular visual-inertial SLAM with point-line fusion and parallel-line fusion[J]. IEEE Robotics and Automation Letters, 2021, 6(4): 7033-7040.
[18]	QIN T, LI P L, SHEN S J. VINS-Mono: a robust and versatile monocular visual-inertial state estimator[J]. IEEE Transactions on Robotics, 2018, 34(4): 1004-1020.
[19]	FURGALE P, REHDER J, SIEGWART R. Unified temporal and spatial calibration for multi-sensor systems[C]// Proceedings of the 2013 IEEE/RSJ International Conference on Intelligent Robots and Systems. Tokyo, Japan:IEEE, 2013: 1280-1286.
[20]	MUEGGLER E, REBECQ H, GALLEGO G, et al. The event-camera dataset and simulator: event-based data for pose estimation, visual odometry, and SLAM[J]. The International Journal of Robotics Research, 2017, 36(2): 142-149.
[21]	VIDAL A, REBECQ H, HORSTSCHAEFER T. Ultimate SLAM? combining events, images, and IMU for robust visual SLAM in HDR and high-speed scenarios[J]. IEEE Robotics and Automation Letters, 2018, 3(2): 994-1001.
[22]	REBECQ H, HORSTSCHAEFER T, SCARAMUZZA D. Real-time visual-inertial odometry for event cameras using keyframe-based nonlinear optimization[C]// Proceedings of the 28th British Machine Vision Conference. London, UK: BMVA Press, 2017.