基于动态区域剔除与稠密地图构建的视觉SLAM算法

doi:10.12382/bgxb.2024.0217

摘要/Abstract

摘要：

针对同步定位与建图(Simultaneous Localization and Mapping,SLAM)算法在动态场景中存在定位精度低且无法生成稠密地图的问题,提出一种基于动态区域剔除与稠密建图的视觉SLAM算法。在原ORB-SLAM3算法的基础上新建动态特征点检测线程,使用YOLOX网络获取动态场景语义信息及物体检测框,同时结合语义和几何约束检测特征点运动状态,提出动态特征点剔除算法,旨在精准实现动态特征点的剔除。随后设计稠密建图线程,基于关键帧及相应位姿构建稠密点云地图,利用地图中剩余的静态特征点,去除动态物体造成的重影,实现稠密地图的构建。在公开TUM数据集和真实动态环境进行验证,在TUM数据集的动态环境下,新算法有效消除了动态物体对位姿估计的影响,提升了SLAM算法在动态场景中的定位与建图的鲁棒性。

关键词: 动态环境, 目标检测, 几何约束, 稠密建图

Abstract:

The simultaneous localization and mapping (SLAM) algorithm has low positioning accuracy and cannot generate the dense maps in dynamic environment.For the above problems,a visual SLAM algorithm based on dynamic region exclusion and dense mapping is proposed.The algorithm creates a dynamic feature point detection thread into the original ORB-SLAM3 algorithm,and the YOLOX network is used to obtain the semantic information and object detection boxes in dynamic scenes.The algorithm detects the motion state of feature points by combining semantic and geometric constraints.A dynamic feature exclusion algorithm is proposed to accurately remove the dynamic feature points.Subsequently,a dense mapping thread is designed to construct dense point cloud maps based on keyframes and their corresponding poses.The remaining static feature points in the map are used to remove the ghosting caused by dynamic objects,thus achieving the construction of a dense map.The proposed algorithm is verified in the public TUM dataset and real dynamic environment.In the dynamic environment of TUM dataset,the proposed algorithm effectively eliminates the impact of dynamic objects on pose estimation,and improves the positioning and mapping accuracies of SLAM algorithm in dynamic scene.

Key words: dynamic environment, object detection, geometric constraint, dense mapping

中图分类号:

TP242.3

赵薇, 王峰, 马星宇, 翟伟光, 孟鹏帅. 基于动态区域剔除与稠密地图构建的视觉SLAM算法[J]. 兵工学报, 2025, 46(3): 240217-.

ZHAO Wei, WANG Feng, MA Xingyu, ZHAI Weiguang, MENG Pengshuai. Visual SLAM Algorithm Based on Dynamic Region Exclusion and Dense Map Construction[J]. Acta Armamentarii, 2025, 46(3): 240217-.

图/表 15

参考文献 26

[1]	张一凡, 缪远东, 陈畅宇, 等. 户外增强现实军事仿真系统[J]. 光学技术, 2019, 45(5):622-628.
	ZHANG Y F, MIAO Y D, CHEN C Y, et al. Outdoor military simulation system based on augmented reality[J]. Optical Technique, 2019, 45(5):622-628. (in Chinese)
[2]	DAVISON A J, REID I D, MOLTON N D, et al. MonoSLAM:real-time single camera SLAM[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2007, 29(6):1052-1067.
[3]	FORSTER C, ZHANG Z C, GASSNER M, et al. SVO:Semidirect visual odometry for monocular and multicamera systems[J]. IEEE Transactions on Robotics, 2016, 33(2):249-265.
[4]	MUR-ARTAL R, TARDÓS J D. ORB-SLAM2:an open-source SLAM system for monocular,stereo,and GRB-D cameras[J]. IEEE Transactions on Robotics, 2017, 33(5):1255-1262.
[5]	CAMPOS C, ELVIRA R, RODRÍGUEZ J J G, et al. Orb-slam3:an accurate open-source library for visual,visual-inertial,and multimap SLAM[J]. IEEE Transactions on Robotics, 2021, 37(6):1874-1890.
[6]	张福斌, 张炳烁, 杨玉帅. 基于单目/IMU/里程计融合的 SLAM 算法[J]. 兵工学报, 2022, 43(11):2810-2818.
	ZHANG F B, ZHANG B S, YANG Y S. SLAM algorithm based on monocular/IMU/odometer fusion[J]. Acta Armamentarii, 2022, 43(11):2810-2818. (in Chinese) doi: 10.12382/bgxb.2022.0240
[7]	ZOU D P, TAN P. Coslam:collaborative visual SLAM in dynamic environments[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2012, 35(2):354-366.
[8]	SUN Y X, LIU M, MENG M Q H. Improving RGB-D SLAM in dynamic environments:a motion removal approach[J]. Robotics and Autonomous Systems, 2017, 89:110-122.
[9]	DAI W C, ZHANG Y, LI P, et al. RGB-D SLAM in dynamic environments using point correlations[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2020, 44(1):373-389.
[10]	罗元, 沈吉祥, 李方宇. 动态环境下基于深度学习的视觉SLAM研究综述[J]. 半导体光电, 2024, 45(1):1-10.
	LUO Y, SHEN J X, LI F Y. Review of visual research based on deep learning in dynamic environments[J]. Semiconductor Optoelectronics, 2024, 45(1):1-10. (in Chinese)
[11]	ZHANG L, WEI L Q, SHEN P Y, et al. Semantic SLAM based on object detection and improved octomap[J]. IEEE Access, 2018, 6:75545-75559.
[12]	REDMON J, FARHADI A. YOLO9000:better,faster,stronger[C]// Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Honolulu,HI,US: IEEE, 2017:7263-7271.
[13]	XIAO L H, WANG J G, QIU X S, et al. Dynamic-SLAM:semantic monocular visual localization and mapping based on deep learning in dynamic environment[J]. Robotics and Autonomous Systems, 2019, 117:1-16.
[14]	LIU Y B, MIURA J. RDS-SLAM:real-time dynamic SLAM using semantic segmentation methods[J]. IEEE Access, 2021, 9:23772-23785.
[15]	YU C, LIU Z X, LIU X J, et al. DS-SLAM:a semantic visual SLAM towards dynamic environments[C]// Proceedings of 2018 IEEE/RSJ International Conference on Intelligent Robots and Systems. Madrid,Spain: IEEE, 2018:1168-1174.
[16]	BADRINARAYANAN V, KENDALL A, CIPOLLA R. Segnet:a deep convolutional encoder-decoder architecture for image segmentation[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2017, 39(12):2481-2495.
[17]	BESCOS B, FÁCIL J M, CIVERA J, et al. DynaSLAM:tracking,mapping,and inpainting in dynamic scenes[J]. IEEE Robotics and Automation Letters, 2018, 3(4):4076-4083.
[18]	HE K M, GKIOXARI G, DOLLÁR P, et al. Mask R-CNN[C]// Proceedings of the IEEE International Conference on Computer Vision. Venice Italy: IEEE, 2017:2961-2969.
[19]	ZHONG F W, WANG S, ZHANG Z Q, et al. Detect-SLAM:making object detection and SLAM mutually beneficial[C]// Proceedings of 2018 IEEE Winter Conference on Applications of Computer Vision. Lake Tahoe,NV,US: IEEE, 2018:1001-1010.
[20]	LIU W, ANGUELOV D, ERHAN D, et al. SSD: single shot multibox detector[C] //Proceedings of Computer Vision ECCV 2016: 14th European Conference.Part I 14.Amsterdam,the Netherlands:Springer,2016:21-37.
[21]	CHANG J F, DONG N, LI D H. A real-time dynamic object segmentation framework for SLAM system in dynamic scenes[J]. IEEE Transactions on Instrumentation and Measurement, 2021, 70:1-9.
[22]	岳胜哲, 王正杰. 基于实例分割与光流的动态环境SLAM[J]. 兵工学报, 2024, 45(1):156-165. doi: 10.12382/bgxb.2023.0568
	YUE S Z, WANG Z J. A SLAM in dynamic environment based on instance segmentation and optical flow[J]. Acta Armamentarii, 2024, 45(1):156-165. (in Chinese) doi: 10.12382/bgxb.2023.0568
[23]	GE Z, LIU S T, WANG F, et al. YOLOX:exceeding YOLO series in 2021:arXiv:2107.08430[R]. Ithaca,NY, US: Cornell University, 2021,2107.08430.
[24]	LIN T Y, MAIRE M, BELONGIE S, et al. Microsoft COC: common objects in context[C] //Proceedings of Computer Vision ECCV 2014: 13 rd European Conference.Part V 13.Zurich,Switzerland: Springer, 2014:740-755.
[25]	ESLAMIAN A, AHMADZADEH M R. Det-SLAM:a semantic visual SLAM for highly dynamic scenes using Detectron2[C]// Proceedings of 2022 8th Iranian Conference on Signal Processing and Intelligent Systems. Mazandaran,Iran: IEEE, 2022:1-5.
[26]	LUO Y, RAO Z R, WU R S. FD-SLAM:a semantic SLAM based on enhanced fast-SCNN dynamic region detection and DeepFillv2-Driven background inpainting[J]. IEEE Access, 2023, 11:110615-110626.

结果	ORB-SLAM3				本文算法
结果	均方根	平均值	中位数	标准差	均方根	平均值	中位数	标准差
fr3_walking_xyz	0.2767	0.2495	0.2360	0.1197	0.0149	0.0128	0.0110	0.0076
fr3_walking_static	0.4242	0.3711	0.3491	0.2055	0.0098	0.0088	0.0073	0.0051
fr3_walking_rpy	0.7765	0.6571	0.6436	0.4137	0.0425	0.0327	0.0263	0.0271
fr3_walking_half	0.3619	0.3443	0.3515	0.1114	0.0234	0.0194	0.0164	0.0131
fr3_sitting_static	0.0107	0.0093	0.0081	0.0052	0.0058	0.0049	0.0042	0.0030
fr3_sitting_xyz	0.0099	0.0088	0.0081	0.0045	0.0134	0.0117	0.0106	0.0066
fr3_sitting_rpy	0.0242	0.0174	0.0124	0.0169	0.0354	0.0278	0.0223	0.0219

结果	ORB-SLAM3				本文算法
结果	均方根	平均值	中位数	标准差	均方根	平均值	中位数	标准差
fr3_walking_xyz	0.2767	0.2495	0.2360	0.1197	0.0149	0.0128	0.0110	0.0076
fr3_walking_static	0.4242	0.3711	0.3491	0.2055	0.0098	0.0088	0.0073	0.0051
fr3_walking_rpy	0.7765	0.6571	0.6436	0.4137	0.0425	0.0327	0.0263	0.0271
fr3_walking_half	0.3619	0.3443	0.3515	0.1114	0.0234	0.0194	0.0164	0.0131
fr3_sitting_static	0.0107	0.0093	0.0081	0.0052	0.0058	0.0049	0.0042	0.0030
fr3_sitting_xyz	0.0099	0.0088	0.0081	0.0045	0.0134	0.0117	0.0106	0.0066
fr3_sitting_rpy	0.0242	0.0174	0.0124	0.0169	0.0354	0.0278	0.0223	0.0219

结果	ORB-SLAM3				本文算法
结果	均方根	平均值	中位数	标准差	均方根	平均值	中位数	标准差
fr3_walking_xyz	0.0307	0.0253	0.0207	0.0174	0.0089	0.0079	0.0077	0.0041
fr3_walking_static	0.0177	0.0126	0.0095	0.0124	0.0080	0.0063	0.0052	0.004
fr3_walking_rpy	0.0340	0.0272	0.0215	0.0203	0.0292	0.0166	0.0121	0.0240
fr3_walking_half	0.0263	0.0201	0.0155	0.0170	0.0173	0.0130	0.0100	0.0115
fr3_sitting_static	0.0056	0.0048	0.0040	0.0029	0.0051	0.0044	0.0039	0.0026
fr3_sitting_xyz	0.0115	0.0102	0.0092	0.0052	0.0106	0.0089	0.0077	0.0057
fr3_sitting_rpy	0.0231	0.0146	0.0111	0.0179	0.0170	0.0117	0.0079	0.0123

结果	ORB-SLAM3				本文算法
结果	均方根	平均值	中位数	标准差	均方根	平均值	中位数	标准差
fr3_walking_xyz	0.0307	0.0253	0.0207	0.0174	0.0089	0.0079	0.0077	0.0041
fr3_walking_static	0.0177	0.0126	0.0095	0.0124	0.0080	0.0063	0.0052	0.004
fr3_walking_rpy	0.0340	0.0272	0.0215	0.0203	0.0292	0.0166	0.0121	0.0240
fr3_walking_half	0.0263	0.0201	0.0155	0.0170	0.0173	0.0130	0.0100	0.0115
fr3_sitting_static	0.0056	0.0048	0.0040	0.0029	0.0051	0.0044	0.0039	0.0026
fr3_sitting_xyz	0.0115	0.0102	0.0092	0.0052	0.0106	0.0089	0.0077	0.0057
fr3_sitting_rpy	0.0231	0.0146	0.0111	0.0179	0.0170	0.0117	0.0079	0.0123

结果	ATE				RPE
结果	Det-SLAM^[25]	RDS^[14]	FD-SLAM^[26]	本文算法	Det-SLAM^[25]	RDS^[14]	FD-SLAM^[26]	本文算法
fr3_walking_xyz	0.0553	0.021	0.016	0.0148	0.0653	0.026	0.0221	0.0079
fr3_walking_static	0.0049	0.081	0.0060	0.0098	0.0100	0.022	0.0116	0.0080
fr3_walking_rpy	0.0386	0.146	0.0412	0.0424	0.0680	0.024	0.0132	0.0291
fr3_walking_half	0.0626	0.025	0.0323	0.0234	0.0167	0.027	0.0235	0.0173
fr3_sitting_static	0.0045		0.0055	0.0057	0.0223		0.0064	0.0050
fr3_sitting_xyz				0.0134				0.0106