基于实例分割与光流的动态环境SLAM

doi:10.12382/bgxb.2023.0568

摘要/Abstract

摘要：

针对传统语义实时定位与建图(Semantic real-time Localization and Mappling,SLAM)算法在动态环境下剔除特征点过多、造成定位精度降低的问题,提出一种基于实例分割与光流的视觉语义SLAM算法。算法使用Mask R-CNN网络对图像中的潜在动态物体进行实例级别的分割,同时在光流线程中对动态物体进行识别并剔除,随后使用剩余的静态光流点与静态特征点联合优化定位,实现语义信息与光流信息的充分融合利用。使用公开数据集测试和地面无人平台实验对所提方法进行验证。实验结果表明,在TUM数据集下,新方法的定位均值误差相比ORB-SLAM2平均提高75%,相比Dyna-SLAM平均提高8.5%。

关键词: 动态环境, 光流法, 实例分割, 语义SLAM

Abstract:

A visual semantic SLAM algorithm based on instance segmentation and optical flow is proposed to address the issue of excessive removal of features by traditional semantic SLAM algorithms in dynamic environments.The proposed algorithm utilizes a Mask R-CNN network to perform the instance-level segmentation of potential dynamic objects in an image, and also identifies and eliminates dynamic objects in the optical flow thread. The remaining static optical flow points and static feature points are then used to optimize the location estimation process, ensuring the optimal utilization of both semantic and optical flow information. The proposed algorithm is validated through testing on open datasets and an unmanned ground platform experiment. The experimental results indicate that the average error of the proposed algorithm is 75% and 8.5% lower than those of ORB-SLAM2 and Dyna-SLAM, respectively, on TUM dataset.

Key words: dynamic environment, optical flow method, instance segmentation, semantic simultaneous localization and mapping

中图分类号:

TP242.3

岳胜哲, 王正杰. 基于实例分割与光流的动态环境SLAM[J]. 兵工学报, 2024, 45(1): 156-165.

YUE Shengzhe, WANG Zhengjie. A SLAM in Dynamic Environment Based on Instance Segmentation and Optical Flow[J]. Acta Armamentarii, 2024, 45(1): 156-165.

图/表 15

图1 算法工作流程

Fig.1 Framework of algorithm

图2 动态物体点剔除流程

Fig.2 Removal process of dynamic object point

图3 动态物体实例

Fig.3 Examples of dynamic objects

图4 光流点选取

Fig.4 Optical flow point selection

图5 因子图

Fig.5 Factor graph

图6 地图补全过程

Fig.6 Map completion process

图7 TUM数据集图例

Fig.7 TUM dataset legend

图8 地图补全效果对比

Fig.8 Comparison of map completion effects

表1 TUM数据集绝对轨迹误差对比

Table 1 Comparison of absolute trajectory errors in TUM datasets

编号	ORB-SLAM2		DYNA-SLAM		本文算法
编号	RMSE	STD	RMSE	STD	RMSE	STD
sitting_static	0.009	0.004	0.006	0.003	0.006	0.003
walking_static	0.328	0.115	0.037	0.043	0.034	0.039
walking_xyz	0.892	0.480	0.091	0.057	0.075	0.050

表2 TUM数据集下相机绝对轨迹误差对比

Table 2 Comparison of camera absolute trajectory errors under TUM dataset

表3 KITTI数据集绝对位姿误差对比

Table 3 Comparison of absolute pose errors in KITTI dataset

序列	ORB-SLAM2				本文算法
序列	RMSE	Mean	Max	Min	RMSE	Mean	Max	Min
01	9.558	8.025	16.98	4.398	6.229	6.074	11.958	1.502
02	6.816	5.621	15.005	0.483	5.006	3.978	10.262	0.251
04	1.154	1.057	1.846	0.506	1.147	1.165	2.149	0.105
06	0.801	0.716	1.290	0.480	0.597	0.741	1.086	0.323

表4 同类型算法绝对位姿误差对比

Table 4 Comparison of absolute pose errors of algorithms of the same type

序列	DYNA-SLAM	本文方法
00	3.505	2.853
01	9.003	8.720
02	5.219	4.723
03	1.299	1.195
04	1.591	1.292
05	1.779	1.499
06	0.824	0.721
07	2.489	2.232
08	3.291	2.965
09	2.564	2.181
10	2.743	2.276

图9 地面无人平台

Fig.9 Ground unmanned platform

图10 室外动态环境轨迹对比

Fig.10 Comparison of outdoor dynamic environment trajectories

图11 低/高动态环境图像对比

Fig.11 Comparison of low and high dynamic environment images

参考文献 27

[1]	CADENA C, CARLONE L, CARRILLO H, et al. Past, present, and future of simultaneous localization and mapping: toward the robust-perception age[J]. IEEE Transactions on Robotics, 2016, 32(6): 1309-1332. doi: 10.1109/TRO.2016.2624754 URL
[2]	TAKETOMI T, UCHIYAMA H, IKEDA S. Visual SLAM algorithms: a survey from 2010 to 2016[J]. IPSJ Transactions on Computer Vision and Applications, 2017, 9(1):16. doi: 10.1186/s41074-017-0027-2
[3]	张福斌, 张炳烁, 杨玉帅. 基于单目/IMU/里程计融合的SLAM算法[J]. 兵工学报, 2022, 43(11):2810-2818.
	ZHANG F B, ZHANG B S, YANG Y S. SLAM algorithm based on monocular/IMU/odometer fusion[J]. Acta Armamentarii, 2022, 43(11): 2810-2818. (in Chinese) doi: 10.12382/bgxb.2022.0240
[4]	MUR-ARTAL R, TARDOS D J. ORB-SLAM2:an open-source SLAM system for monocular, stereo, and RGB-D cameras[J]. IEEE Transactions on Robotics, 2017, 33(5):1255-1262. doi: 10.1109/TRO.2017.2705103 URL
[5]	刘全攀, 王正杰, 王寰. 基于双目视觉-惯性导航的轻型无人机导航算法[J]. 兵工学报, 2020, 41(增刊2):241-248.
	LIU Q P, WANG Z J, WANG H. Navigation algorithm of light UAV based on stereo visual inertial navigation odometer[J]. Acta Armamentarii, 2020, 41(S2): 241-248. (in Chinese)
[6]	ENGEL J, KOLTUN V, CREMERS D. Direct sparse odometry[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2018, 40(3):611-625. doi: 10.1109/TPAMI.2017.2658577 pmid: 28422651
[7]	SUN Y X, LIU M, MENG M Q H. Improving RGB-D SLAM in dynamic environments: a motion removal approach[J]. Robotics and Autonomous Systems, 2016, 89: 110-122. doi: 10.1016/j.robot.2016.11.012 URL
[8]	LI S L, LEE D. RGB-D SLAM in dynamic environments using static point weighting[J]. IEEE Robotics and Automation Letters, 2017, 2(4):2263-2270. doi: 10.1109/LRA.2017.2724759 URL
[9]	WANG Y B, HUANG S D. Towards dense moving object segmentation based robust dense RGB-D SLAM in dynamic scenarios[C]// Proceedings of IEEE International Conference on Control, Automation, Robotics & Vision. Singapore: IEEE, 2014: 1841-1846.
[10]	TAN W, LIU H M, DONG Z L, et al. Robust monocular SLAM in dynamic environments[C]// Proceedings of IEEE International Symposium on Mixed and Augmented Reality. Adelaide, SA, Australia:IEEE, 2013:209-218.
[11]	BESCOS B, FACIL J M, CIVERA J, et al. DynaSLAM: tracking, mapping, and inpainting in dynamic scenes[J]. IEEE Robotics and Automation Letters, 2018, 3(4):4076-4083. doi: 10.1109/LSP.2016. URL
[12]	CHENG J B, WANG Z, ZHOU H Y, et al. DM-SLAM:a feature-based SLAM system for rigid dynamic scenes[J]. ISPRS International Journal of Geo-Information, 2020, 9(4):202. doi: 10.3390/ijgi9040202 URL
[13]	ZHONG F W, WANG S, ZHANG Z Q, et al. Detect-SLAM: making object detection and SLAM mutually beneficial[C]// Proceedings of IEEE Winter Conference on Applications of Computer Vision. Lake Tahoe, NV, US: IEEE, 2018:1001-1010.
[14]	YU C, LIU Z X, LIU X J, et al. DS-SLAM:a semantic visual SLAM towards dynamic environments[C]// Proceedings of IEEE/RSJ International Conference on Intelligent Robots and Systems.Madrid, Spain:IEEE, 2018:1168-1174.
[15]	LIU Y B, MIURA J. RDS-SLAM: real-time dynamic SLAM using semantic segmentation methods[J]. IEEE Access, 2021, 9: 23772-23785. doi: 10.1109/Access.6287639 URL
[16]	ALCANTARILLA P F, YEBES J J, ALMAZAN J, et al. On combining visual SLAM and dense scene flow to increase the robustness of localization and mapping in dynamic environments[C]// Proceedings of IEEE International Conference on Robotics and Automation. Saint Paul, MN, US: IEEE, 2012: 1290-1297.
[17]	刘立涛, 聂亮. 合成孔径光学成像系统与图像复原技术[J]. 自动化技术与应用, 2021, 40(3): 96-101.
	LIU L T, NIE L. Synthetic aperture optical imaging system and image restoration technology[J]. Automation Technology and Applications, 2021, 40(3): 96-101. (in Chinese)
[18]	FISCHLER M, BOLLES R. Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography[J]. Communications of the ACM, 1981, 24(6): 381-395. doi: 10.1145/358669.358692 URL
[19]	HE K M, GKIOXARI G, DOLLAR P, et al. Mask r-cnn[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2020, 42(2):386-397. doi: 10.1109/TPAMI.2018.2844175 pmid: 29994331
[20]	TSUNG-YI L, MICHAEL M, SERGE J B, et al. Microsoft coco: common objects in context[J]. Lecture Notes in Computer Science, 2014, 48: 740-755.
[21]	NISTER D, NARODITSKY O, BERGEN J. Visual odometry for ground vehicle applications[J]. Journal of Field Robotics, 2006, 23(1):3-20. doi: 10.1002/rob.v23:01 URL
[22]	CATANIA L, LUATI A. Robust estimation of a location parameter with the integrated hogg function[J]. Statistics and Probability Letters, 2020, 164: 108812. doi: 10.1016/j.spl.2020.108812 URL
[23]	DELLAERT F. Square root SAM: simultaneous localization and mapping via square root information smoothing[J]. The International Journal of Robotics Research, 2006, 25(12):1181-1203. doi: 10.1177/0278364906072768 URL
[24]	KAESS M, JOHANNSSON H, ROBERTS R, et al. iSAM2: Incremental smoothing and mapping using the Bayes tree[J]. The International Journal of Robotics Research, 2012, 31(2):216-235. doi: 10.1177/0278364911430419 URL
[25]	STURM J, ENGELHARD N, ENDRES F, et al. A benchmark for the evaluation of RGB-D SLAM systems[C]// Proceedings of IEEE/RSJ International Conference on Intelligent Robots and Systems.Vilamoura, Algarve, Portugal: IEEE, 2012: 573-580.
[26]	ZHANG J, HENEIN M, MAHONY R, et al. Robust ego and object 6-DoF motion estimation and tracking[C]// Proceedings of IEEE/RSJ International Conference on Intelligent Robots and Systems. Las Vegas, NV, US: IEEE, 2020: 1875-1881.
[27]	NEWSON A, ALMANSA A, FRADET M, et al. Video inpainting of complex scenes[J]. SIAM Journal on Imaging Sciences, 2014, 7(4):1993-2019. doi: 10.1137/140954933 URL