Progress on Environmental Perception Technology of Foreign Unmanned Surface Vehicles

doi:10.12382/bgxb.2024.0860

Abstract

Abstract:

Unmanned surface vehicles (USVs) have high mobility,strong concealment,and extensive operational range,making them highly suitable for performing a wide array of tasks such as reconnaissance,anti-submarine warfare,search and rescue.Environmental perception technology,crucial for the operation of USVs,has attracted considerable attention.This paper conducts a survey on the development status of environmental perception technology for USVs at abroad,and define and analyzed the challenges in USV environmental perception through specific case studies.The current state of research on USV environmental perception technology is analyzed from the perspectives of both unimodal and multimodal perception,considering the sensory equipment utilized by USVs.Finally,the unresolved challenges in USV environmental perception technology are summarized,and its future development is prospected.

Key words: unmanned surface vehicle, perception technology, multi-modal perception, radar

LI Dan, YU Shaozhen, YANG Huadong. Progress on Environmental Perception Technology of Foreign Unmanned Surface Vehicles[J]. Acta Armamentarii, 2024, 45(S2): 97-104.

Figures/Tables 11

Fig.1 Schematic diagram of the Sea Hunter

Table 1 Common Sensors for unmanned surface vehicle

传感器类型	探测距离	信息内容	优点	缺点
可见光传感器	10~5km	可见光视频	分辨率高、信息内容丰富	易受干扰,黑暗环境无法工作
红外传感器	10~500m	红外视频	抗干扰能力强、可在夜间工作	分辨率较低,探测范围有限
扫描雷达	100~10km	雷达点云	范围广、抗干扰能力强	信息内容较少、精度低
激光雷达	1~100m	雷达点云	精度高	探测范围较弱
航海雷达	1~100km	雷达点云	探测范围广	精度低
惯导		船舶姿态、位置	自主导航、结构简单	计算量大、误差累积

Fig.2 Main working environments for unmanned surface vehicles

Fig.3 Image enhancement effect of improved convex optimization model[14]

Fig.4 Result of thermal object detection[19]

Fig.5 Result of USV Object Tracking[22]

Fig.6 Maritime radar SLAM mapping[25]

Fig.7 Obstacle avoidance effect in actual ports using LiDAR[31]

Fig.8 Environmental perception scheme combining visual sensors and radar

Fig.9 Flowchart of environmental perception combining AIS and radar

Fig.10 Radar and AIS under occlusion conditions[43]

References 45

[1]	张韩西子, 倪海参, 石正坤, 等. 水面无人艇发展趋势及关键技术展望[J]. 舰船科学技术, 2024, 46(8):108-111.
	ZHANG H X Z, NI H S, SHI Z K, et al. Development trends and key technology prospects of unmanned surface vehicles[J]. Chinese Journal of Ship Research, 2024, 46(8):108-111. (in Chinese)
[2]	CASOLA K J. System architecture and operational analysis of medium displacement unmanned surface vehicle sea hunter as a surface warfare component of distributed lethality[D]. Monterey,CA,US: Naval Postgraduate School, 2017.
[3]	PANDHARIPANDE A, CHENG C H, DAUWELS J, et al. Sensing and machine learning for automotive perception:a review[J]. IEEE Sensors Journal, 2023, 23(11):11097-11115.
[4]	ZHANG H, HUANG Y Y, QIN H C, et al. USV search mission planning methodology for lost target rescue on sea[J]. Electronics, 2023, 12(22):4584.
[5]	ZHANG J, REN J, CUI Y N, et al. Multi-USV task planning method based on improved deep reinforcement learning[J]. IEEE Internet of Things Journal, 2024, 11(10):18549-18567.
[6]	FAN Y S, SUN Z, WANG G F. A novel intelligent collision avoidance algorithm based on deep reinforcement learning approach for USV[J]. Ocean Engineering, 2023, 287:115649.
[7]	鲍凌志, 杜家辉, 胡辛明, 等. 无人艇集群控制技术研究综述[J]. 自然杂志, 2023, 45(3):207-216.
	BAO L Z, DU J H, HU X M, et al. Research review on swarm control technology for unmanned boats[J]. Nature Magazine, 2023, 45(3):207-216. (in Chinese)
[8]	孙文祺, 王文龙, 姜兆祯, 等. 海洋观测无人艇发展现状与展望[J]. 中国造船, 2022, 63(5):256-271.
	SUN W Q, WANG W L, JIANG Z Z, et al. Current status and prospects of unmanned surface vehicles for marine observation[J]. China Shipbuilding, 2022, 64(5):256-271. (in Chinese)
[9]	HASHALI S D, YANG S, XIANG X. Route planning algorithms for unmanned surface vehicles (USVs):a comprehensive analysis[J]. Journal of Marine Science and Engineering, 2024, 12(3):382.
[10]	孔凡涛. 基于激光雷达与相机信息融合的无人艇环境感知[D]. 大连: 大连海事大学, 2023.
	KONG F T. Environment perception of unmanned boats based on the fusion of lidar and camera information[D]. Dalian: Dalian Maritime University, 2023.. (in Chinese)
[11]	WU D, ZHU Q. The latest research progress of image dehazing[J]. Acta Automatica Sinica, 2015, 41(2):221-239.
[12]	MA Z, WEN J. Single-scale Retinex sea fog removal algorithm fused the edge information[J]. Journal of Computer-Aided Design & Computer Graphics, 2015, 27(2):217-225.
[13]	HUANG H, LI Z Y, NIU M B, et al. A sea fog image defogging method based on the improved convex optimization model[J]. Journal of Marine Science and Engineering, 2023, 11(9):1775.
[14]	QIAN R, TAN R T, YANG W, et al. Attentive generative adversarial network for raindrop removal from a single image[C]// Proceedings of the 35th IEEE Conference on Computer Vision and Pattern Recognition. 2018:2482-2491.
[15]	LI N, WANG M L, YANG G C, et al. DENS-YOLOv6:a small object detection model for garbage detection on water surface[J]. Multimedia Tools and Applications, 2024, 83(18):55751-55771.
[16]	JIANG P, ERGU D, LIU F, et al. A review of Yolo algorithm developments[J]. Procedia Computer Science, 2022, 199:1066-1073.
[17]	LI S Y, TIAN X Q, YANG X, et al. Human vision-inspired detection for infrared small targets under scaly light background on sea surface[J]. Digital Signal Processing, 2023, 134:103894.
[18]	ZHANG L L, ZHANG Y, ZHANG Z, et al. Real-time water surface object detection based on improved faster R-CNN[J]. Sensors, 2019, 19(16):3523.
[19]	WANG H J, WU S G, HE Y Z, et al. UYF-Net:a fusion network for feasible domain recognition and obstacle detection based on infrared thermography[J]. Infrared Physics & Technology, 2024:105430.
[20]	PAL S K, PRAMANIK A, MAITI J, et al. Deep learning in multi-object detection and tracking:state of the art[J]. Applied Intelligence, 2021, 51:6400-6429.
[21]	GLADSTONE R, MOSHE Y, BAREL A, et al. Distance estimation for marine vehicles using a monocular video camera[C]// Proceedings of the 24th European Signal Processing Conference.Budapest,Hungary:IEEE, 2016:2405-2409.
[22]	SINISTERRA A J, DHANAK M R, VON ELLENRIEDER K. Stereovision-based target tracking system for USV operations[J]. Ocean Engineering, 2017, 133:197-214.
[23]	HUANG W, LIU X, GILL E W. Ocean wind and wave measurements using X-band marine radar:a comprehensive review[J]. Remote Sensing, 2017, 9(12):1261.
[24]	ALMEIDA C, FRANCO T, FERREIRA H, et al. Radar based collision detection developments on USV ROAZ II[C]// Proceedings of the 5th Oceans 2009-Europe.Bremen,Germany:IEEE, 2009:1-6.
[25]	HAN J, PARK J, KIM J, et al. GPS-less coastal navigation using marine radar for USV operation[J]. IFAC-PapersOnLine, 2016, 49(23):598-603.
[26]	KIRIAKIDIS K, CROTEAU B, SEVERSON T, et al. Degradable tracking system based on hardware multi-model estimators[C]// Proceedings of the 8th resilience week.National Harbor,MD,US:IEEE, 2022:1-6.
[27]	WANG Y Z, JIANG Z Y, LI Y D, et al. RODNet:a real-time radar object detection network cross-supervised by camera-radar fused object 3D localization[J]. IEEE Journal of Selected Topics in Signal Processing, 2021, 15(4):954-967.
[28]	DANZER A, GRIEBEl T, BACH M, et al. 2d car detection in radar data with pointnets[C]// Proceedings of the 22nd IEEE Intelligent Transportation Systems Conference.Auckland, New Zealand: IEEE, 2019:61-66.
[29]	QI C R, SU H, MO K, et al. Pointnet:deep learning on point sets for 3d classification and segmentation[C]// Proceedings of the 34th IEEE Conference on Computer Vision and Pattern Recognition.Piscataway,NJ,US:IEEE, 2017:652-660.
[30]	PENDLETON S, CHONG Z J, QIN B, et al. Multi-class driverless vehicle cooperation for mobility-on-demand[C]// Proceedings of the 19th Intelligent Transportation Systems World Congress.Piscataway,NJ,US:IEEE, 2014:1-4.
[31]	BANDYOPHADYAY T, SARCIONE L, HOVER F S. A simple reactive obstacle avoidance algorithm and its application in Singapore harbor[C]// Proceedings of the 7th International Conference Field and Service Robotics.Berlin, Heidelberg: Springer Berlin Heidelberg, 2010:455-465.
[32]	CLUNIE T, DEFILIPPO M, SACARNY M, et al. Development of a perception system for an autonomous surface vehicle using monocular camera,lidar,and marine radar[C]// Proceedings of the 21st IEEE International Conference on Robotics and Automation.Xi’an,China:IEEE, 2021:14112-14119.
[33]	FENG M Z, SU J B. RGBT tracking:a comprehensive review[J]. Information Fusion, 2024:102492.
[34]	TANG X H, PEI Z H, YIN S, et al. Practical design and implementation of an autonomous surface vessel prototype:navigation and control[J]. International Journal of Advanced Robotic Systems, 2020, 17(3):872-879.
[35]	ZEWGE N S, KIM Y, KIM J, et al. Millimeter-wave radar and RGB-D camera sensor fusion for real-time people detection and tracking[C]// Proceedings of the 7th International Conference on Robot Intelligence Technology and Applications.Daejeon,Korea:IEEE, 2019:93-98.
[36]	ZHANG W, JIANG F, YANG C F, et al. Research on unmanned surface vehicles environment perception based on the fusion of vision and lidar[J]. IEEE Access, 2021, 9:63107-63121.
[37]	FRANK D, GRAY A, Schwartz E. PropaGator 2015:UF autonomous surface vehicle[J]. AUVSI Foundation’s 7th Annual RoboBoat Competition, 2015:1-10.
[38]	HARATI-MOKHTARI A, WALL A, BROOKS P, et al. Automatic Identification System (AIS):data reliability and human error implications[J]. the Journal of Navigation, 2007, 60(3):373-389.
[39]	YU K, LIANG X F, LI M Z, et al. USV path planning method with velocity variation and global optimisation based on AIS service platform[J]. Ocean Engineering, 2021, 236:109560.
[40]	MA H, SMART E, AHMED A, et al. Radar image-based positioning for USV under GPS denial environment[J]. IEEE Transactions on Intelligent Transportation Systems, 2017, 19(1):72-80.
[41]	HAN J, CHO Y, KIM J. Coastal SLAM with marine radar for USV operation in GPS-restricted situations[J]. IEEE Journal of Oceanic Engineering, 2019, 44(2):300-309.
[42]	BØRS-LIND K S. Prototype of HIL test platform for autonomous USV-simulation and visualization of vessel surroundings[D]. Trondheim,Norway: Norwegian University of Science and Technology, 2017.
[43]	SUN S, LÜ H G, DONG C M. AIS aided marine radar target tracking in a detection occluded environment[J]. Ocean Engineering, 2023, 288:116133.
[44]	ZHU D J, PAN Y J. Optimal motion planning for heterogeneous Multi-USV systems using hexagonal grid-based neural networks and parallelogram law under ocean currents[J]. IEEE Transactions on Systems,Man,and Cybernetics:Systems, 2024:1-8.
[45]	HAN Y, XIAO Y, GAO Y, et al. Dynamic relay selection for USV-aided smart ocean IoT[J]. IEEE Communications Letters, 2024, 28(7):1723-1727.