YSG-SLAM:a Real-time Semantic RGB-D SLAM Based on YOLACT in Dynamic Scene

Weiguang ZHAI; Feng WANG; Xingyu MA; Wei ZHAO; Pengshuai MENG

doi:10.12382/bgxb.2024.0443

您当前的位置：

首页 >

文章列表页 >

YSG-SLAM:a Real-time Semantic RGB-D SLAM Based on YOLACT in Dynamic Scene

更新时间：2025-09-05

- YSG-SLAM:a Real-time Semantic RGB-D SLAM Based on YOLACT in Dynamic Scene
- Acta Armamentarii Vol. 46, Issue 6, Pages: 240443(2025)
- 作者机构：
  
  1. 太原理工大学电子信息工程学院, 山西太原 030002
  2. 太原理工大学电气与动力工程学院, 山西太原 030002
- 作者简介：
- 基金信息：
- DOI：10.12382/bgxb.2024.0443
  CLC：
- Received：05 June 2024，
  
  Published Online：28 June 2025，
  
  Published：10 June 2025
- 稿件说明：
移动端阅览
Weiguang ZHAI, Feng WANG, Xingyu MA, et al. YSG-SLAM:a Real-time Semantic RGB-D SLAM Based on YOLACT in Dynamic Scene[J]. Acta Armamentarii, 2025, 46(6): 240443.
DOI：

Weiguang ZHAI, Feng WANG, Xingyu MA, et al. YSG-SLAM:a Real-time Semantic RGB-D SLAM Based on YOLACT in Dynamic Scene[J]. Acta Armamentarii, 2025, 46(6): 240443. DOI： 10.12382/bgxb.2024.0443.

摘要

针对动态环境中实时定位与建图(Simultaneous Localization and Mapping

SLAM)算法位姿估计存在的定位漂移、实时性差等问题

提出一个名为YSG-SLAM的实时语义RGB-D SLAM系统。为了提高系统实时性

新增两个并行线程:一个用于获取二维语义信息的语义分割线程

一个语义建图线程。为优化系统在处理动态物体时的准确性和鲁棒性

YSG-SLAM引入快速动态特征剔除算法

并耦合漏检补偿模块来应对基于实时实例分割(You Only Look At Coefficients

YOLACT)算法可能出现的漏检情况

有效地提升了特征点剔除的精确度和系统的整体稳定性。为减少由特征点聚集引起的定位误差从而优化特征点的空间分布

设计自适应角点提取阈值计算方法

使特征分布更加均匀。语义建图线程充分利用二维语义信息与三维点云数据

可选择性构建语义地图和八叉树地图

提高了系统的环境感知能力及机器人在复杂环境下的相关任务执行能力。YSG-SLAM在德国慕尼黑工业大学数据集、Bonn数据集上进行了评估

相较于原ORB-SLAM2

各项定位误差下降达93%。实验结果表明

YSG-SLAM有效提升了系统实时性

定位精度高

且可构建两种地图

具有一定的实用价值。

Abstract

Simultaneous localization and mapping (SLAM) algorithm has the issues such as localization drift and poor real-time performance in attitude estimation within dynamic environment.A real-time semantic RGB-D SLAM system named as YSG-SLAM is proposed.To enhance the system’s real-time performance

two parallel threads

one for semantic segmentation to obtain 2D semantic information and another for semantic mapping

are introduced.To improve the accuracy and robustness of the system in handling the dynamic objects

YSG-SLAM incorporates a fast dynamic feature removal algorithm and couples it with a missed detection compensation module to mitigate potential missed detections from YOLACT segmentation algorithm.This effectively enhances the feature point removal accuracy and overall system stability.To reduce the localization errors caused by feature point clustering and optimize the spatial distribution of feature points

an adaptive corner extraction threshold calculation method is designed to make the distribution of features more uniform.The semantic mapping thread makes full use of 2D semantic information and 3D point cloud data to optionally construct semantic maps and octree maps

thereby improving the environmental perception capability of system and the task execution ability of robots in complex environment.YSG-SLAM has been evaluated on TUM and Bonn datasets

showing a 93% reduction in various localization errors compared to the original ORB-SLAM2.Experimental results indicate that YSG-SLAM significantly improves real-time performance

has high localization accuracy

and can construct two types of maps.

关键词

Keywords

references

MUR-ARTAL R , TARDÓS J D . Orb-slam2:an open-source slam system for monocular,stereo,and rgb-d cameras [J ] . IEEE Transactions on Robotics , 2017 , 33 ( 5 ): 1255 - 1262 .

QIN T , LI P L , SHEN S J . Vins-mono:a robust and versatile monocular visual-inertial state estimator [J ] . IEEE Transactions on Robotics , 2018 , 34 ( 4 ): 1004 - 1020 .

DAVISON A J , REID I D , MOLTON N D , et al . MonoSLAM:real-time single camera SLAM [J ] . IEEE Transactions on Pattern Analysis and Machine Intelligence , 2007 , 29 ( 6 ): 1052 - 1067 .

CAMPOS C , ELVIRA R , RODRÍGUEZ J J G , et al . Orb-slam3:an accurate open-source library for visual,visual-inertial,and multimap slam [J ] . IEEE Transactions on Robotics , 2021 , 37 ( 6 ): 1874 - 1890 .

FORSTER C , ZHANG Z C , GASSNER M , et al . SVO:semidirect visual odometry for monocular and multicamera systems [J ] . IEEE Transactions on Robotics , 2016 , 33 ( 2 ): 249 - 265 .

张福斌 , 张炳烁 , 杨玉帅 . 基于单目/IMU/里程计融合的 SLAM 算法 [J ] . 兵工学报 , 2022 , 43 ( 11 ): 2810 - 2818 .

ZHANG F B , ZHANG B S , YANG Y S . SLAM algorithm based on monocular/IMU/odometer fusion [J ] . Acta Armamentarii , 2022 , 43 ( 11 ): 2810 - 2818 . (in Chinese) DOI: 10.12382/bgxb.2022.0240 http://doi.org/10.12382/bgxb.2022.0240 It is common for navigation and positioning accuracy to be reduced when the monocular vision-inertial SLAM algorithm is applied to planar wheeled robots due to additional unobservability. To solve this problem, a tightly-coupled Visual/IMU/Odometer SLAM algorithm is proposed to improve localization accuracy. First, in the visual front-end part, the original image pyramid LK optical flow method is improved, and the rotation information of the gyroscope and the translation information from the odometer are used as priors to optimize the initial optical flow calculation process, thus reducing the calculation amount. Second, IMU/Odometer pre-integral is derived by introducing the wheel odometer information. Finally, odometer constraints are added into the initialization process and back-end nonlinear optimization to realize that vision, IMU, and odometer information are fully integrated. The results of the open-source data set test and car experiment show that the optical flow iteration time of the new algorithm is reduced by about 32.5%, and the average positioning error reduced by about 40% compared with that of VINS-Mono.

ZHAO X Y , LI Q H , WANG C H , et al . Robust depth-aided RGBD-inertial odometry for indoor localization [J ] . Measurement , 2023 , 209 : 112487 - 112498 .

LIU Y , WU Y L , PAN W Z . Dynamic RGB-D SLAM based on static probability and observation number [J ] . IEEE Transactions on Instrumentation and Measurement , 2021 , 70 : 1 - 11 .

BAHRAINI M S , BOZORG M , RAD A B . SLAM in dynamic environments via ML-RANSAC [J ] . Mechatronics , 2018 , 49 : 105 - 118 .

ZHANG T W , ZHANG H Y , LI Y , et al . Flowfusion:dynamic dense rgb-d slam based on optical flow [C ] // Proceedings of 2020 IEEE International Conference on Robotics and Automation.Paris , France : IEEE , 2020 : 7322 - 7328 .

SUN Y X , LIU M , MENG M Q H . Motion removal for reliable RGB-D SLAM in dynamic environments [J ] . Robotics and Autonomous Systems , 2018 , 108 : 115 - 128 .

ZHANG L , WEI L Q , SHEN P Y , et al . Semantic SLAM based on object detection and improved octomap [J ] . IEEE Access , 2018 , 6 : 75545 - 75559 .

REDMON J , FARHADI A . YOLO9000:better,faster,stronger [C ] // Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.Honolulu,HI,US:IEEE , 2017 : 7263 - 7271 .

XIAO L H , WANG J G , QIU X S , et al . Dynamic-SLAM:semantic monocular visual localization and mapping based on deep learning in dynamic environment [J ] . Robotics and Autonomous Systems , 2019 , 117 : 1 - 16 .

LIU Y B , MIURA J . RDS-SLAM:real-time dynamic SLAM using semantic segmentation methods [J ] . IEEE Access , 2021 , 9 : 23772 - 23785 .

BESCOS B , FÁCIL J M , CIVERA J , et al . DynaSLAM:tracking,mapping,and inpainting in dynamic scenes [J ] . IEEE Robotics and Automation Letters , 2018 , 3 ( 4 ): 4076 - 4083 .

HE K M , GKIOXARI G , DOLLÁR P , et al . Mask R-CNN [C ] // Proceedings of the IEEE International Conference on Computer Vision.Venice,Italy:IEEE , 2017 : 2961 - 2969 .

YU C , LIU Z X , LIU X J , et al . DS-SLAM:a semantic visual SLAM towards dynamic environments [C ] // Proceedings of 2018 IEEE/RSJ International Conference on Intelligent Robots and Systems.Madrid,Spain:IEEE , 2018 : 1168 - 1174 .

BADRINARAYANAN V , KENDALL A , CIPOLLA R . Segnet:a deep convolutional encoder-decoder architecture for image segmentation [J ] . IEEE Transactions on Pattern Analysis and Machine Intelligence , 2017 , 39 ( 12 ): 2481 - 2495 .

WU W X , GUO L , GAO H L , et al . YOLO-SLAM:a semantic SLAM system towards dynamic environment with geometric constraint [J ] . Neural Computing and Applications , 2022 , 34 ( 8 ): 6011 - 6026 .

CHANG J F , DONG N , LI D H . A real-time dynamic object segmentation framework for SLAM system in dynamic scenes [J ] . IEEE Transactions on Instrumentation and Measurement , 2021 , 70 : 1 - 9 .

BOLYA D , ZHOU C , XIAO F Y , et al . Yolact:real-time instance segmentation [C ] // Proceedings of the IEEE/CVF International Conference on Computer Vision.Seoul,Korea:IEEE , 2019 : 9157 - 9166 .

LUO Y , RAO Z R , WU R S . FD-SLAM:a semantic SLAM based on enhanced fast-SCNN dynamic region detection and DeepFillv2-Driven background inpainting [J ] . IEEE Access , 2023 , 11 : 110615 - 110626 .

KENYE L , KALA R . Improving RGB-D SLAM in dynamic environments using semantic aided segmentation [J ] . Robotica , 2022 , 40 ( 6 ): 2065 - 2090 .

YU X Y , ZHAN Y A , ZHU F . Improved ORB feature extraction algorithm based on quadtree encoding [J ] . Theoretical Computer Science , 2018 , 45 ( 2 ): 232 - 235 .

孙浩 , 王朋 . 一种基于区域划分的改进ORB算法 [J ] . 北京航空航天大学学报 , 2020 , 46 ( 9 ): 1763 - 1769 .

SUN H , WANG P . An improved ORB algorithm based on region division [J ] . Journal of Beijing University of Aeronautics and Astronautics , 2020 , 46 ( 9 ): 1763 - 1769 . (in Chinese)

GONG H , GONG L , MA T B , et al . AHY-SLAM:toward faster and more accurate visual SLAM in dynamic scenes using homogenized feature extraction and object detection method [J ] . Sensors , 2023 , 23 ( 9 ): 4241 .

WANG X Q , ZHENG S Y , LIN X H , et al . Improving RGB-D SLAM accuracy in dynamic environments based on semantic and geometric constraints [J ] . Measurement , 2023 , 217 : 113084 .

Views

405

下载量

CNKI被引量

Alert me when the article has been cited

提交

Tools

Publicity Resources

Obstacle Recognition and Traversability Analysis of Tracked Vehicles in Off-road Environment

Autonomous UAV Location Selection Technique Based on Data Fusion

Visual SLAM Algorithm Based on Dynamic Region Exclusion and Dense Map Construction

A SLAM in Dynamic Environment Based on Instance Segmentation and Optical Flow

An Infrared Small Target Detection Method via Dual Network Collaboration

Related Author

Yujian LIU

Yunlu TAN

Liang YU

Man CHEN

Biao MA

Haodong WANG

Hong NIE

Xiaohui WEI

Related Institution

School of Mechanical Engineering, Beijing Institute of Technology

Intelligent Systems Headquarters, China North Artificial Intelligence & Innovation Research Institute

College of Aerospace Engineering, Nanjing University of Aeronautics and Astronautics

Key Laboratory of Fundamental Science for National Defense-Advanced Design Technology of Flight Vehicle, Nanjing University of Aeronautics and Astronautics

National Key Laboratory of Helicopter Aeromechanics, Nanjing University of Aeronautics and Astronautics

Postal code：100089
Tel：010-68963060/68962718 Fax：010-68963025 Email：bgxb@cos.org.cn
Technical support is provided by Beijing Founder electronics co., LTD 京ICP备05059581号-4 京公网安备11010802024360号
It is recommended to read the content of this site in Chrome&IE9+. Please switch to extreme mode in browser 360.
Cookies We use cookies to help provide and enhance our service and tailor content. By continuing, you agree to the use of cookies.
Total Visits：44524 Visits Today：504

⁰