YSG-SLAM:动态场景下基于YOLACT的实时语义RGB-D SLAM系统

doi:10.12382/bgxb.2024.0443

摘要/Abstract

摘要：

针对动态环境中实时定位与建图(Simultaneous Localization and Mapping,SLAM)算法位姿估计存在的定位漂移、实时性差等问题,提出一个名为YSG-SLAM的实时语义RGB-D SLAM系统。为了提高系统实时性,新增两个并行线程:一个用于获取二维语义信息的语义分割线程,一个语义建图线程。为优化系统在处理动态物体时的准确性和鲁棒性,YSG-SLAM引入快速动态特征剔除算法,并耦合漏检补偿模块来应对基于实时实例分割(You Only Look At Coefficients,YOLACT)算法可能出现的漏检情况,有效地提升了特征点剔除的精确度和系统的整体稳定性。为减少由特征点聚集引起的定位误差从而优化特征点的空间分布,设计自适应角点提取阈值计算方法,使特征分布更加均匀。语义建图线程充分利用二维语义信息与三维点云数据,可选择性构建语义地图和八叉树地图,提高了系统的环境感知能力及机器人在复杂环境下的相关任务执行能力。YSG-SLAM在德国慕尼黑工业大学数据集、Bonn数据集上进行了评估,相较于原ORB-SLAM2,各项定位误差下降达93%。实验结果表明,YSG-SLAM有效提升了系统实时性,定位精度高,且可构建两种地图,具有一定的实用价值。

关键词: 动态环境, 语义分割, 自适应阈值, 漏检补偿, 语义建图

Abstract:

Simultaneous localization and mapping (SLAM) algorithm has the issues such as localization drift and poor real-time performance in attitude estimation within dynamic environment.A real-time semantic RGB-D SLAM system named as YSG-SLAM is proposed.To enhance the system’s real-time performance,two parallel threads,one for semantic segmentation to obtain 2D semantic information and another for semantic mapping, are introduced.To improve the accuracy and robustness of the system in handling the dynamic objects,YSG-SLAM incorporates a fast dynamic feature removal algorithm and couples it with a missed detection compensation module to mitigate potential missed detections from YOLACT segmentation algorithm.This effectively enhances the feature point removal accuracy and overall system stability.To reduce the localization errors caused by feature point clustering and optimize the spatial distribution of feature points,an adaptive corner extraction threshold calculation method is designed to make the distribution of features more uniform.The semantic mapping thread makes full use of 2D semantic information and 3D point cloud data to optionally construct semantic maps and octree maps,thereby improving the environmental perception capability of system and the task execution ability of robots in complex environment.YSG-SLAM has been evaluated on TUM and Bonn datasets,showing a 93% reduction in various localization errors compared to the original ORB-SLAM2.Experimental results indicate that YSG-SLAM significantly improves real-time performance,has high localization accuracy,and can construct two types of maps.

Key words: dynamic environment, semantic segmentation, adaptive threshold, missed detection compensation, semantic mapping

中图分类号:

TP242.3

翟伟光, 王峰, 马星宇, 赵薇, 孟鹏帅. YSG-SLAM:动态场景下基于YOLACT的实时语义RGB-D SLAM系统[J]. 兵工学报, 2025, 46(6): 240443-.

ZHAI Weiguang, WANG Feng, MA Xingyu, ZHAO Wei, MENG Pengshuai. YSG-SLAM:a Real-time Semantic RGB-D SLAM Based on YOLACT in Dynamic Scene[J]. Acta Armamentarii, 2025, 46(6): 240443-.

图/表 17

参考文献 28

[1]	MUR-ARTAL R, TARDÓS J D. Orb-slam2:an open-source slam system for monocular,stereo,and rgb-d cameras[J]. IEEE Transactions on Robotics, 2017, 33(5):1255-1262.
[2]	QIN T, LI P L, SHEN S J. Vins-mono:a robust and versatile monocular visual-inertial state estimator[J]. IEEE Transactions on Robotics, 2018, 34(4):1004-1020.
[3]	DAVISON A J, REID I D, MOLTON N D, et al. MonoSLAM:real-time single camera SLAM[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2007, 29(6):1052-1067.
[4]	CAMPOS C, ELVIRA R, RODRÍGUEZ J J G, et al. Orb-slam3:an accurate open-source library for visual,visual-inertial,and multimap slam[J]. IEEE Transactions on Robotics, 2021, 37(6):1874-1890.
[5]	FORSTER C, ZHANG Z C, GASSNER M, et al. SVO:semidirect visual odometry for monocular and multicamera systems[J]. IEEE Transactions on Robotics, 2016, 33(2):249-265.
[6]	张福斌, 张炳烁, 杨玉帅. 基于单目/IMU/里程计融合的 SLAM 算法[J]. 兵工学报, 2022, 43(11):2810-2818.
	ZHANG F B, ZHANG B S, YANG Y S. SLAM algorithm based on monocular/IMU/odometer fusion[J]. Acta Armamentarii, 2022, 43(11):2810-2818. (in Chinese) doi: 10.12382/bgxb.2022.0240
[7]	ZHAO X Y, LI Q H, WANG C H, et al. Robust depth-aided RGBD-inertial odometry for indoor localization[J]. Measurement, 2023, 209:112487-112498.
[8]	LIU Y, WU Y L, PAN W Z. Dynamic RGB-D SLAM based on static probability and observation number[J]. IEEE Transactions on Instrumentation and Measurement, 2021, 70:1-11.
[9]	BAHRAINI M S, BOZORG M, RAD A B. SLAM in dynamic environments via ML-RANSAC[J]. Mechatronics, 2018, 49:105-118.
[10]	ZHANG T W, ZHANG H Y, LI Y, et al. Flowfusion:dynamic dense rgb-d slam based on optical flow[C]// Proceedings of 2020 IEEE International Conference on Robotics and Automation.Paris, France: IEEE, 2020:7322-7328.
[11]	SUN Y X, LIU M, MENG M Q H. Motion removal for reliable RGB-D SLAM in dynamic environments[J]. Robotics and Autonomous Systems, 2018, 108:115-128.
[12]	ZHANG L, WEI L Q, SHEN P Y, et al. Semantic SLAM based on object detection and improved octomap[J]. IEEE Access, 2018, 6:75545-75559.
[13]	REDMON J, FARHADI A. YOLO9000:better,faster,stronger[C]// Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.Honolulu,HI,US:IEEE, 2017:7263-7271.
[14]	XIAO L H, WANG J G, QIU X S, et al. Dynamic-SLAM:semantic monocular visual localization and mapping based on deep learning in dynamic environment[J]. Robotics and Autonomous Systems, 2019, 117:1-16.
[15]	LIU Y B, MIURA J. RDS-SLAM:real-time dynamic SLAM using semantic segmentation methods[J]. IEEE Access, 2021, 9:23772-23785.
[16]	BESCOS B, FÁCIL J M, CIVERA J, et al. DynaSLAM:tracking,mapping,and inpainting in dynamic scenes[J]. IEEE Robotics and Automation Letters, 2018, 3(4):4076-4083.
[17]	HE K M, GKIOXARI G, DOLLÁR P, et al. Mask R-CNN[C]// Proceedings of the IEEE International Conference on Computer Vision.Venice,Italy:IEEE, 2017:2961-2969.
[18]	YU C, LIU Z X, LIU X J, et al. DS-SLAM:a semantic visual SLAM towards dynamic environments[C]// Proceedings of 2018 IEEE/RSJ International Conference on Intelligent Robots and Systems.Madrid,Spain:IEEE, 2018:1168-1174.
[19]	BADRINARAYANAN V, KENDALL A, CIPOLLA R. Segnet:a deep convolutional encoder-decoder architecture for image segmentation[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2017, 39(12):2481-2495.
[20]	WU W X, GUO L, GAO H L, et al. YOLO-SLAM:a semantic SLAM system towards dynamic environment with geometric constraint[J]. Neural Computing and Applications, 2022, 34(8):6011-6026.
[21]	CHANG J F, DONG N, LI D H. A real-time dynamic object segmentation framework for SLAM system in dynamic scenes[J]. IEEE Transactions on Instrumentation and Measurement, 2021, 70:1-9.
[22]	BOLYA D, ZHOU C, XIAO F Y, et al. Yolact:real-time instance segmentation[C]// Proceedings of the IEEE/CVF International Conference on Computer Vision.Seoul,Korea:IEEE, 2019:9157-9166.
[23]	LUO Y, RAO Z R, WU R S. FD-SLAM:a semantic SLAM based on enhanced fast-SCNN dynamic region detection and DeepFillv2-Driven background inpainting[J]. IEEE Access, 2023, 11:110615-110626.
[24]	KENYE L, KALA R. Improving RGB-D SLAM in dynamic environments using semantic aided segmentation[J]. Robotica, 2022, 40(6):2065-2090.
[25]	YU X Y, ZHAN Y A, ZHU F. Improved ORB feature extraction algorithm based on quadtree encoding[J]. Theoretical Computer Science, 2018, 45(2):232-235.
[26]	孙浩, 王朋. 一种基于区域划分的改进ORB算法[J]. 北京航空航天大学学报, 2020, 46(9):1763-1769.
	SUN H, WANG P. An improved ORB algorithm based on region division[J]. Journal of Beijing University of Aeronautics and Astronautics, 2020, 46(9):1763-1769. (in Chinese)
[27]	GONG H, GONG L, MA T B, et al. AHY-SLAM:toward faster and more accurate visual SLAM in dynamic scenes using homogenized feature extraction and object detection method[J]. Sensors, 2023, 23(9):4241.
[28]	WANG X Q, ZHENG S Y, LIN X H, et al. Improving RGB-D SLAM accuracy in dynamic environments based on semantic and geometric constraints[J]. Measurement, 2023, 217:113084.

图像序列	原始图像	ORB-SLAM2	YSG-SLAM
v_bird
v_coffeehouse
v_pomegranate

图像序列	原始图像	ORB-SLAM2	YSG-SLAM
v_bird
v_coffeehouse
v_pomegranate

两种算法及提升率/%	图像序列
两种算法及提升率/%	v_bird	v_pomegranate	v_underground	v_dogman	v_coffeehouse
ORB-SLAM2	4.65424	4.27598	4.50987	3.98462	3.88937
YSG-SLAM	5.65170	5.84350	5.69493	5.31122	5.69493
提升率/%	21.0	36.7	30.6	33.3	46.4

两种算法及提升率/%	图像序列
两种算法及提升率/%	v_bird	v_pomegranate	v_underground	v_dogman	v_coffeehouse
ORB-SLAM2	4.65424	4.27598	4.50987	3.98462	3.88937
YSG-SLAM	5.65170	5.84350	5.69493	5.31122	5.69493
提升率/%	21.0	36.7	30.6	33.3	46.4

图像序列	ORB-SLAM2				YSG-SLAM				提升率/%
图像序列	均方根	平均值	中位数	标准差	均方根	平均值	中位数	标准差	均方根	平均值	中位数	标准差
fr3_s_static	0.2899	0.2606	0.2484	0.1271	0.1619	0.1376	0.1136	0.0854	44.15	47.20	54.27	32.81
fr3_w_half	7.9219	4.4695	1.2568	6.5406	0.4142	0.3507	0.3022	0.2204	94.77	92.15	75.95	96.63
fr3_w_rpy	6.4220	4.5134	2.2990	4.5685	0.5450	0.4130	0.3233	0.3556	91.51	90.85	85.94	92.22
fr3_w_xyz	7.1415	5.6403	4.6159	4.3804	0.3997	0.2816	0.2298	0.2836	94.40	95.01	95.02	93.53