Visual SLAM Algorithm Based on Dynamic Region Exclusion and Dense Map Construction

doi:10.12382/bgxb.2024.0217

Abstract

Abstract:

The simultaneous localization and mapping (SLAM) algorithm has low positioning accuracy and cannot generate the dense maps in dynamic environment.For the above problems,a visual SLAM algorithm based on dynamic region exclusion and dense mapping is proposed.The algorithm creates a dynamic feature point detection thread into the original ORB-SLAM3 algorithm,and the YOLOX network is used to obtain the semantic information and object detection boxes in dynamic scenes.The algorithm detects the motion state of feature points by combining semantic and geometric constraints.A dynamic feature exclusion algorithm is proposed to accurately remove the dynamic feature points.Subsequently,a dense mapping thread is designed to construct dense point cloud maps based on keyframes and their corresponding poses.The remaining static feature points in the map are used to remove the ghosting caused by dynamic objects,thus achieving the construction of a dense map.The proposed algorithm is verified in the public TUM dataset and real dynamic environment.In the dynamic environment of TUM dataset,the proposed algorithm effectively eliminates the impact of dynamic objects on pose estimation,and improves the positioning and mapping accuracies of SLAM algorithm in dynamic scene.

Key words: dynamic environment, object detection, geometric constraint, dense mapping

CLC Number:

TP242.3

ZHAO Wei, WANG Feng, MA Xingyu, ZHAI Weiguang, MENG Pengshuai. Visual SLAM Algorithm Based on Dynamic Region Exclusion and Dense Map Construction[J]. Acta Armamentarii, 2025, 46(3): 240217-.

Figures/Tables 15

References 26

[1]	张一凡, 缪远东, 陈畅宇, 等. 户外增强现实军事仿真系统[J]. 光学技术, 2019, 45(5):622-628.
	ZHANG Y F, MIAO Y D, CHEN C Y, et al. Outdoor military simulation system based on augmented reality[J]. Optical Technique, 2019, 45(5):622-628. (in Chinese)
[2]	DAVISON A J, REID I D, MOLTON N D, et al. MonoSLAM:real-time single camera SLAM[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2007, 29(6):1052-1067.
[3]	FORSTER C, ZHANG Z C, GASSNER M, et al. SVO:Semidirect visual odometry for monocular and multicamera systems[J]. IEEE Transactions on Robotics, 2016, 33(2):249-265.
[4]	MUR-ARTAL R, TARDÓS J D. ORB-SLAM2:an open-source SLAM system for monocular,stereo,and GRB-D cameras[J]. IEEE Transactions on Robotics, 2017, 33(5):1255-1262.
[5]	CAMPOS C, ELVIRA R, RODRÍGUEZ J J G, et al. Orb-slam3:an accurate open-source library for visual,visual-inertial,and multimap SLAM[J]. IEEE Transactions on Robotics, 2021, 37(6):1874-1890.
[6]	张福斌, 张炳烁, 杨玉帅. 基于单目/IMU/里程计融合的 SLAM 算法[J]. 兵工学报, 2022, 43(11):2810-2818.
	ZHANG F B, ZHANG B S, YANG Y S. SLAM algorithm based on monocular/IMU/odometer fusion[J]. Acta Armamentarii, 2022, 43(11):2810-2818. (in Chinese) doi: 10.12382/bgxb.2022.0240
[7]	ZOU D P, TAN P. Coslam:collaborative visual SLAM in dynamic environments[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2012, 35(2):354-366.
[8]	SUN Y X, LIU M, MENG M Q H. Improving RGB-D SLAM in dynamic environments:a motion removal approach[J]. Robotics and Autonomous Systems, 2017, 89:110-122.
[9]	DAI W C, ZHANG Y, LI P, et al. RGB-D SLAM in dynamic environments using point correlations[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2020, 44(1):373-389.
[10]	罗元, 沈吉祥, 李方宇. 动态环境下基于深度学习的视觉SLAM研究综述[J]. 半导体光电, 2024, 45(1):1-10.
	LUO Y, SHEN J X, LI F Y. Review of visual research based on deep learning in dynamic environments[J]. Semiconductor Optoelectronics, 2024, 45(1):1-10. (in Chinese)
[11]	ZHANG L, WEI L Q, SHEN P Y, et al. Semantic SLAM based on object detection and improved octomap[J]. IEEE Access, 2018, 6:75545-75559.
[12]	REDMON J, FARHADI A. YOLO9000:better,faster,stronger[C]// Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Honolulu,HI,US: IEEE, 2017:7263-7271.
[13]	XIAO L H, WANG J G, QIU X S, et al. Dynamic-SLAM:semantic monocular visual localization and mapping based on deep learning in dynamic environment[J]. Robotics and Autonomous Systems, 2019, 117:1-16.
[14]	LIU Y B, MIURA J. RDS-SLAM:real-time dynamic SLAM using semantic segmentation methods[J]. IEEE Access, 2021, 9:23772-23785.
[15]	YU C, LIU Z X, LIU X J, et al. DS-SLAM:a semantic visual SLAM towards dynamic environments[C]// Proceedings of 2018 IEEE/RSJ International Conference on Intelligent Robots and Systems. Madrid,Spain: IEEE, 2018:1168-1174.
[16]	BADRINARAYANAN V, KENDALL A, CIPOLLA R. Segnet:a deep convolutional encoder-decoder architecture for image segmentation[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2017, 39(12):2481-2495.
[17]	BESCOS B, FÁCIL J M, CIVERA J, et al. DynaSLAM:tracking,mapping,and inpainting in dynamic scenes[J]. IEEE Robotics and Automation Letters, 2018, 3(4):4076-4083.
[18]	HE K M, GKIOXARI G, DOLLÁR P, et al. Mask R-CNN[C]// Proceedings of the IEEE International Conference on Computer Vision. Venice Italy: IEEE, 2017:2961-2969.
[19]	ZHONG F W, WANG S, ZHANG Z Q, et al. Detect-SLAM:making object detection and SLAM mutually beneficial[C]// Proceedings of 2018 IEEE Winter Conference on Applications of Computer Vision. Lake Tahoe,NV,US: IEEE, 2018:1001-1010.
[20]	LIU W, ANGUELOV D, ERHAN D, et al. SSD: single shot multibox detector[C] //Proceedings of Computer Vision ECCV 2016: 14th European Conference.Part I 14.Amsterdam,the Netherlands:Springer,2016:21-37.
[21]	CHANG J F, DONG N, LI D H. A real-time dynamic object segmentation framework for SLAM system in dynamic scenes[J]. IEEE Transactions on Instrumentation and Measurement, 2021, 70:1-9.
[22]	岳胜哲, 王正杰. 基于实例分割与光流的动态环境SLAM[J]. 兵工学报, 2024, 45(1):156-165. doi: 10.12382/bgxb.2023.0568
	YUE S Z, WANG Z J. A SLAM in dynamic environment based on instance segmentation and optical flow[J]. Acta Armamentarii, 2024, 45(1):156-165. (in Chinese) doi: 10.12382/bgxb.2023.0568
[23]	GE Z, LIU S T, WANG F, et al. YOLOX:exceeding YOLO series in 2021:arXiv:2107.08430[R]. Ithaca,NY, US: Cornell University, 2021,2107.08430.
[24]	LIN T Y, MAIRE M, BELONGIE S, et al. Microsoft COC: common objects in context[C] //Proceedings of Computer Vision ECCV 2014: 13 rd European Conference.Part V 13.Zurich,Switzerland: Springer, 2014:740-755.
[25]	ESLAMIAN A, AHMADZADEH M R. Det-SLAM:a semantic visual SLAM for highly dynamic scenes using Detectron2[C]// Proceedings of 2022 8th Iranian Conference on Signal Processing and Intelligent Systems. Mazandaran,Iran: IEEE, 2022:1-5.
[26]	LUO Y, RAO Z R, WU R S. FD-SLAM:a semantic SLAM based on enhanced fast-SCNN dynamic region detection and DeepFillv2-Driven background inpainting[J]. IEEE Access, 2023, 11:110615-110626.

结果	ORB-SLAM3				本文算法
结果	均方根	平均值	中位数	标准差	均方根	平均值	中位数	标准差
fr3_walking_xyz	0.2767	0.2495	0.2360	0.1197	0.0149	0.0128	0.0110	0.0076
fr3_walking_static	0.4242	0.3711	0.3491	0.2055	0.0098	0.0088	0.0073	0.0051
fr3_walking_rpy	0.7765	0.6571	0.6436	0.4137	0.0425	0.0327	0.0263	0.0271
fr3_walking_half	0.3619	0.3443	0.3515	0.1114	0.0234	0.0194	0.0164	0.0131
fr3_sitting_static	0.0107	0.0093	0.0081	0.0052	0.0058	0.0049	0.0042	0.0030
fr3_sitting_xyz	0.0099	0.0088	0.0081	0.0045	0.0134	0.0117	0.0106	0.0066
fr3_sitting_rpy	0.0242	0.0174	0.0124	0.0169	0.0354	0.0278	0.0223	0.0219

结果	ORB-SLAM3				本文算法
结果	均方根	平均值	中位数	标准差	均方根	平均值	中位数	标准差
fr3_walking_xyz	0.2767	0.2495	0.2360	0.1197	0.0149	0.0128	0.0110	0.0076
fr3_walking_static	0.4242	0.3711	0.3491	0.2055	0.0098	0.0088	0.0073	0.0051
fr3_walking_rpy	0.7765	0.6571	0.6436	0.4137	0.0425	0.0327	0.0263	0.0271
fr3_walking_half	0.3619	0.3443	0.3515	0.1114	0.0234	0.0194	0.0164	0.0131
fr3_sitting_static	0.0107	0.0093	0.0081	0.0052	0.0058	0.0049	0.0042	0.0030
fr3_sitting_xyz	0.0099	0.0088	0.0081	0.0045	0.0134	0.0117	0.0106	0.0066
fr3_sitting_rpy	0.0242	0.0174	0.0124	0.0169	0.0354	0.0278	0.0223	0.0219

结果	ORB-SLAM3				本文算法
结果	均方根	平均值	中位数	标准差	均方根	平均值	中位数	标准差
fr3_walking_xyz	0.0307	0.0253	0.0207	0.0174	0.0089	0.0079	0.0077	0.0041
fr3_walking_static	0.0177	0.0126	0.0095	0.0124	0.0080	0.0063	0.0052	0.004
fr3_walking_rpy	0.0340	0.0272	0.0215	0.0203	0.0292	0.0166	0.0121	0.0240
fr3_walking_half	0.0263	0.0201	0.0155	0.0170	0.0173	0.0130	0.0100	0.0115
fr3_sitting_static	0.0056	0.0048	0.0040	0.0029	0.0051	0.0044	0.0039	0.0026
fr3_sitting_xyz	0.0115	0.0102	0.0092	0.0052	0.0106	0.0089	0.0077	0.0057
fr3_sitting_rpy	0.0231	0.0146	0.0111	0.0179	0.0170	0.0117	0.0079	0.0123

结果	ORB-SLAM3				本文算法
结果	均方根	平均值	中位数	标准差	均方根	平均值	中位数	标准差
fr3_walking_xyz	0.0307	0.0253	0.0207	0.0174	0.0089	0.0079	0.0077	0.0041
fr3_walking_static	0.0177	0.0126	0.0095	0.0124	0.0080	0.0063	0.0052	0.004
fr3_walking_rpy	0.0340	0.0272	0.0215	0.0203	0.0292	0.0166	0.0121	0.0240
fr3_walking_half	0.0263	0.0201	0.0155	0.0170	0.0173	0.0130	0.0100	0.0115
fr3_sitting_static	0.0056	0.0048	0.0040	0.0029	0.0051	0.0044	0.0039	0.0026
fr3_sitting_xyz	0.0115	0.0102	0.0092	0.0052	0.0106	0.0089	0.0077	0.0057
fr3_sitting_rpy	0.0231	0.0146	0.0111	0.0179	0.0170	0.0117	0.0079	0.0123

结果	ATE				RPE
结果	Det-SLAM^[25]	RDS^[14]	FD-SLAM^[26]	本文算法	Det-SLAM^[25]	RDS^[14]	FD-SLAM^[26]	本文算法
fr3_walking_xyz	0.0553	0.021	0.016	0.0148	0.0653	0.026	0.0221	0.0079
fr3_walking_static	0.0049	0.081	0.0060	0.0098	0.0100	0.022	0.0116	0.0080
fr3_walking_rpy	0.0386	0.146	0.0412	0.0424	0.0680	0.024	0.0132	0.0291
fr3_walking_half	0.0626	0.025	0.0323	0.0234	0.0167	0.027	0.0235	0.0173
fr3_sitting_static	0.0045		0.0055	0.0057	0.0223		0.0064	0.0050
fr3_sitting_xyz				0.0134				0.0106