Unmanned Swarm Collaborative Visual SLAM Algorithm Based on Semi-direct Method

doi:10.12382/bgxb.2023.0547

Abstract

Abstract:

The collaborative positioning and environmental awareness technologies are the cornerstone of autonomous navigation of unmanned swarm. However, due to the limitations of computing, load, bandwidth and other resources of small-sized individual platforms in large-scale unmanned swarm systems, many related technologies are difficultly deployed and applied in practice. In order to achieve the accurate positioning and environmental awareness of large-scale unmanned swarm under resource constraints, a lightweight collaborative visual SLAM algorithm based on semi-direct method is proposed, and a semi-direct feature point tracking method that combines the optical flow method and the direct method is designed. The centralized two-way communication strategy is used to make the large-scale unmanned swarm system have a high fault-tolerant rate in the face of communication interference and delay, and the unmanned swarm system. The comparative experiments were conducted on the algorithm based on the EuRoC dataset and actual physical environment. The results show that the real-time performance of the proposed algorithm is improved by an average of 60%, which is significantly better than other feature-based collaborative visual SLAM algorithms. In a low-quality communication environment where the packet loss rate is less than 40% and the communication delay is less than 0.1 second, the proposed algorithm has higher positioning accuracy and stronger robustness.

Key words: unmanned swarm, collaborative visual SLAM, semi-direct visual odometry, lightweight

CLC Number:

CAO Haozhe, LIU Quanpan. Unmanned Swarm Collaborative Visual SLAM Algorithm Based on Semi-direct Method[J]. Acta Armamentarii, 2023, 44(11): 3345-3358.

Figures/Tables 33

Fig.1 Tracking of keyframes and non-keyframes

Fig.2 Direct image alignment

Fig.3 Alignment of non-keyframes with local map

Fig.4 Marginalization strategy

Fig.5 Framework of collaborative SLAM algorithm based on semi-direct method

Fig.6 Interactive information of unmanned platform and ground station

Fig.7 Schematic diagram of three unmanned vehicles running at the same time

Fig.8 Factor graph representation of three unmanned vehicles running at the same time

Fig.9 Comparison of trajectories and point cloud maps before and after loop closure detection triggering

Fig.10 Map matching

Fig.11 Comparison of trajectories and point cloud maps before and after loop closure detection triggering

Fig.12 Communication controller

Fig.13 Communication bandwidth from the unmanned platforms to the ground station under communication delay

Fig.14 Communication bandwidth from the ground station to the unmanned platforms under communication delay

Fig.15 Communication bandwidth from the unmanned platforms to the ground station under communication interference

Fig.16 Communication bandwidth from the ground station to the unmanned platforms under communication interference

Fig.17 Effect of communication delay on positioning accuracy

Fig.18 Effect of communication interference on positioning accuracy

Fig.19 Effect of communication delay on positioning accuracy (communication interference: 40%)

Fig.20 Effect of communication interference on positioning accuracy (communication delay: 0.3s)

Fig.21 Point cloud maps of EuRoC dataset

Fig.22 Trajectories of EuRoC dataset

Fig.23 Point cloud maps and trajectories of EuRoC dataset

Table 1 Comparison of positioning accuracy m

算法		数据集
算法		MH_01	MH_02	MH_03	V2_01	V2_02
DSO		0.057	0.092	0.198	0.077	0.095
COVINS		0.041	0.060	0.056	0.042	0.053
CCM-SLAM		0.106	0.092	0.156	0.202	0.098
	Agent1	0.035			0.056
无协同	Agent2		0.032			0.087
	Agent3			0.029
协同		0.028	0.030	0.027	0.047	0.062

Table 2 Average time spent tracking a frame of image ms

算法	数据集
算法	MH_01	MH_02	MH_03	V2_01	V2_02
CCM-SLAM	37.82	35.31	34.20	38.57	31.13
COVINS	17.67	13.60	12.28	13.83	13.04
本文算法	6.72	6.58	7.87	7.51	10.22

Fig.24 Comparison of keyframes and non-keyframes in number and tracking time consumption

Fig.25 Real environment experiment

Fig.26 SLAM algorithm test system for ground unmanned platform

Fig.27 Trajectories and point cloud maps constructed by two unmanned vehicles in an indoor environment

Fig.28 Trajectories and point cloud maps constructed by three unmanned vehicles in an indoor environment

Fig.29 Trajectories and point cloud maps constructed by four unmanned vehicles in an indoor environment

Fig.30 Communication bandwidth from the unmanned platforms to the ground station when four unmanned vehicles are running at the same time

Fig.31 Communication bandwidth from the ground station to the unmanned platforms when four unmanned vehicles are running at the same time

References 21

[1]	刘延旭. 面向无人机集群的多源融合定位关键技术研究[D]. 北京: 北京邮电大学, 2022.
	LIU Y X. Research on multi-source fusion positioning towards UAV swarm[D]. Beijing: Beijing University of Posts and Telecommunications, 2022. (in Chinese)
[2]	陈超. 无人集群任务分配方法研究[D]. 长沙: 国防科学技术大学, 2023.
	CHEN C. Research on task assignment algorithms in unmanned swarm[D]. Changsha: National University of Defense Technology, 2023. (in Chinese)
[3]	GENEVA P, ECKENHOFF K, LEE W, et al. Openvins: a research platform for visual inertial estimation[C]// Proceedings of IEEE International Conference on Robotics and Automation.Washington,D.C.,US:IEEE, 2020: 4666-4672.
[4]	张福斌, 张炳烁, 杨玉帅. 基于单目/IMU/里程计融合的SLAM算法[J]. 兵工学报, 2022, 43(11):2810-2818.
	ZHANG F B, ZHANG B S, YANG Y S. SLAM algorithm based on monocular/IMU/odometer fusion[J]. Acta Armamentarii, 2022, 43(11): 2810-2818. (in Chinese) doi: 10.12382/bgxb.2022.0240
[5]	唐铭. 基于非线性贝叶斯滤波的SLAM算法研究[D]. 大连: 大连理工大学, 2022.
	TANG M. Research on nonlinear Bayesian filtering based SLAM algorithms[D]. Dalian: Dalian University of Technology, 2022. (in Chinese)
[6]	范迎春. 动态环境下的视觉SLAM地图构建研究[D]. 西安: 西安电子科技大学, 2021.
	FAN Y C. Research on visual SLAM map construction in dynamic environments[D]. Xi’an: Xidian University, 2021. (in Chinese)
[7]	USENKO V, DEMMEL N, SCHUBERT D, et al. Visual-inertial mapping with non-linear factor recovery[J]. IEEE Robotics and Automation Letters, 2020, 5(2): 422-429. doi: 10.1109/LSP.2016. URL
[8]	夏琳琳, 张晶晶, 初妍, 等. 融合天空偏振光的视觉SLAM研究进展与展望[J/OL]. 兵工学报:1-15[2023-06-16]. http://kns.cnki.net/kcms/detail/11.2176.TJ.20220831.1417.002.html.
	XIA L L, ZHANG J J, CHU Y, et al. Progresses and prospects of polarized skylight fused visual SLAM[J/OL]. Acta Armamentarii: 1-15[2023-06-16]. http://kns.cnki.net/kcms/detail/11.2176.TJ.20220831.1417.002.html. (in Chinese)
[9]	MUR-ARTAL R, MONTIEL J M M, TARDOS J D. ORB-SLAM: a versatile and accurate monocular SLAM system[J]. IEEE Transactions on Robotics, 2015, 31(5): 1147-1163. doi: 10.1109/TRO.2015.2463671 URL
[10]	ELVIRA R, TARDOS J D, MONTIEL J M M. ORBSLAM-atlas:a robust and accurate multi-map system:arXiv:1908.11585[R]. Ithaca,NY,US:Cornell University, 2019:1908.11585.
[11]	ENGEL J, KOLTUN V, CREMERS D. Direct sparse odometry[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2018, 40 (3): 611-625. doi: 10.1109/TPAMI.2017.2658577 pmid: 28422651
[12]	LEE S H, CIVERA J. Loosely coupled semi-direct monocular SLAM[J]. IEEE Robotics and Automation Letters, 2019, 4(2): 399-406. doi: 10.1109/LRA.2018.2889156 URL
[13]	BONIN-FONT F, BURGUERA A. Towards multi-robot visual graph-SLAM for autonomous marine vehicles[J]. Journal of Marine Science and Engineering, 2020, 8(6): 437. doi: 10.3390/jmse8060437 URL
[14]	BARTOLOMEI L, KARRER M, CHLI M. Multi-robot coordination with agent-server architecture for autonomous navigation in partially unknown environments[C] // Proceedings of 2020 IEEE/RSJ International Conference on Intelligent Robots and Systems. Washington,D.C.,US:IEEE, 2020: 1516-1522.
[15]	ZOU D P, TAN P. CoSLAM: Collaborative visual SLAM in dynamic environments[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2012, 35(2): 354-366. doi: 10.1109/TPAMI.2012.104 URL
[16]	RIAZUELO L, CIVERA J, MONTIEL J M M. C2TAM: a cloud framework for cooperative tracking and map-ping[J]. Robotics and Autonomous Systems, 2014, 62(4): 401-413. doi: 10.1016/j.robot.2013.11.007 URL
[17]	FORSTER C, LYNEN S, KNEIP L, et al. Collaborative monocular SLAM with multiple micro aerial vehicles[C]// Proceedings of 2013 IEEE/RSJ International Conference on Intelligent Robots and Systems. Washington,D.C.,US:IEEE, 2013:3962-3970.
[18]	SCHMUCK P, CHLI M. CCM-SLAM: robust and efficient centralized collaborative monocular simultaneous localization and mapping for robotic teams[J]. Journal of Field Robotics, 2019, 36(4): 763-781. doi: 10.1002/rob.2019.36.issue-4 URL
[19]	SCHMUCK P, ZIEGLER T, KARRER M, et al. COVINS: visual-inertial SLAM for centralized collaboration[C]// Proceedings of 2021 IEEE International Symposium on Mixed and Augmented Reality Adjunct, ISMAR-Adjunct. Washington,D.C.,US:IEEE, 2021:171-176.
[20]	LAJOIE P Y, RAMTOULA B, CHANG Y, et al. Door-SLAM: distributed, online, and outlier resilient SLAM for robotic teams[J]. IEEE Robotics and Automation Letters, 2020, 5(2): 1656-1663. doi: 10.1109/LSP.2016. URL
[21]	XU H, ZHANG Y C, ZHOU B Y, et al. Omni-Swarm: a decentralized omnidirectional visual-Inertial-UWB state estimation system for aerial swarms[J]. IEEE Transactions on Robotics, 2022, 38(6): 3374-3394. doi: 10.1109/TRO.2022.3182503 URL