A Real-time Compression Algorithm of Color Point Cloud Streams for Environmental Scanning

doi:10.12382/bgxb.2023.0856

Abstract

Abstract:

The problems of compressing and encoding the large-scale and high-dimensional point cloud data to improve transmission efficiency arise with the continuous development of digital twin technology. However, most point cloud coding methods have weak real-time compression, low compression efficiency, and high requirement of point cloud format. A real-time color stream Draco (RCS-Draco) compression algorithm based on Google Draco geometric compression library is proposed to solve these problems. By integrating the algorithm into the ROS framework, the point cloud stream is encoded and decoded in real time by means of ROS message flow, which improves the real-time performance of the algorithm. An optimal clipping model is established to clip and filter the point cloud, and remove the drift and outlier point cloud, thus improving the compression efficiency of the algorithm. The RGB color information of point cloud is encoded by establishing a quantitative prediction model, which solves the problem that most point cloud compression algorithms cannot process color information. By adjusting the compression grade and quantization parameters, it is proved that the average compression rate of RCS-Draco algorithm can reach 77%, the average compression and decompression time is less than 0.035s, the average position error is less than 0.05m, and the average attribute error is less than 35. The RCS-Draco algorithm is superior to Draco algorithm in every index through the integration test. The experimental results show that the RCS-Draco compression algorithm performs well in terms of real-time compression, efficiency and point cloud format, and can effectively improve transmission efficiency.

Key words: digital twins, compression coding, color point cloud stream, ROS framework

CLC Number:

TJ301

MA Jingqi, YU Qiwen, HUANG Ping, WANG Wei, LI Youwei. A Real-time Compression Algorithm of Color Point Cloud Streams for Environmental Scanning[J]. Acta Armamentarii, 2023, 44(S2): 167-177.

Figures/Tables 13

Table 1 Comparison of typical point cloud compression characteristics

压缩算法	压缩信息	压/解效率	压/解时间	压/解格式	还原质量
Draco	几何+属性	较差	中等	.pcd	可控
HEVC	几何	较高	慢	.bin	无损
Realtime	几何+深度	较高	较快	.bin	有损
BitPan	几何+颜色	较高	慢	.bin	有损

Fig.1 Point cloud filtering and cropping

Fig.2 Coding order

Fig.3 RCS-Draco coding framework

Fig.4 RCS-Draco decoding framework

Fig.5 Point cloud stream encoding/decoding framework

Table 2 ACR, APE and ACE metrics indicators at different quantization and compression levels in the In-door dataset

量化水平	L=0			L=3			L=7			L=10
量化水平	ACR/%	APE/m	ACE	ACR/%	APE/m	ACE	ACR/%	APE/m	ACE	ACR/%	APE/m	ACE
Q=0	77.003	0.000	6.413	75.003	0.000	6.413	75.003	0.000	6.366	75.003	0.000	6.413
Q=5	1.193	0.048	34.478	1.193	0.048	34.478	1.294	0.048	34.478	4.041	0.048	34.292
Q=10	10.989	0.003	4.087	10.989	0.003	7.087	11.922	0.003	7.087	12.438	0.003	7.087
Q=15	22.058	0.000	6.860	22.058	0.000	6.860	23.643	0.000	6.860	18.477	0.000	6.812

Table 3 ACT and ADT indicators at different quantization and compression levels in the In-door dataset

量化水平	L=0		L=3		L=7		L=10
量化水平	ACT/ms	ADT/ms	ACT/ms	ADT/ms	ACT/ms	ADT/ms	ACT/ms	ADT/ms
Q=0	15.428	16.764	15.753	17.325	15.113	19.951	13.948	17.262
Q=5	28.542	13.344	26.655	12.937	23.130	14.387	21.800	15.562
Q=10	34.387	17.802	32.178	18.327	27.653	17.140	20.956	15.723
Q=15	34.799	19.793	34.840	18.306	28.845	19.598	24.602	17.124

Table 4 Point cloud restoration image sets at different quantization and compression levels in the In-door datase

Fig.6 Comparison chart of RCS-Draco and Draco in ACT indicators in the Out-door dataset

Fig.7 Comparison chart of RCS-Draco and Draco in ADT indicators in the Out-door dataset

Fig.8 Comparison of ACR, APE and ACE indicators of RCS-Draco and Draco in the Out-door dataset

Fig.9 Comparison of restoration clarities of RCS-Draco (lift) and Draco (right) point cloud images in the Out-door dataset

References 38

[1]	王珣, 高亮, 魏永勇, 等. 基于任务级-交战级体系仿真平台的并行实验研究[J]. 兵工学报, 2022, 43(增刊1): 183-188.
	WANG X, GAO L, WEI Y Y, et al. Research on parallel experiment based on the simulation platform of mission-engagement level system[J] Acta Armamentarii, 2022, 43(S1): 183-188. (in Chinese) doi: 10.12382/bgxb.2022.A015
[2]	纪广, 郝建国, 张振伟. 面向无人机作战的虚拟孪生系统设计方案[J]. 兵工学报, 2022, 43(8): 1902-1912.
	JI G, HAO J G, ZHANG Z W. Design scheme of virtual twin system for UAV combat[J]. Acta Armamentarii, 2022, 43(8): 1902-1912. (in Chinese) doi: 10.12382/bgxb.2021.0408
[3]	赵继东, 盖振伟, 李晶, 等. 车辆通信设备集成化体系结构[J]. 兵工学报, 2022, 43(增刊1): 21-25.
	ZHAO J D, GAI Z W, LI J, et al. Integrated structure of communication equipment for military vehicle[J]. Acta Armamentarii, 2022, 43(S1): 21-25. (in Chinese) doi: 10.12382/bgxb.2022.A002
[4]	邵玉斌, 刘晶, 龙华, 等. 面向战场环境下的语音传输与重构[J]. 兵工学报, 2022, 43(11): 2827-2835.
	SHAO Y B, LIU J, LONG H, et al. Voice transmission and reconstruction on the battlefield[J]. Acta Armamentarii, 2022, 43(11): 2827-2835. (in Chinese) doi: 10.12382/bgxb.2021.0549
[5]	周可歆, 丁旭辉, 吕德东, 等. 天基信息传输高效低复杂度LDPC译码算法研究[J]. 兵工学报, 2023(2023-04-10)[2023-05-07]. http://www.co-journal.com/CN/10.12382/bgxb.2022.1044.
	ZHOU K X, DING X H, LÜ D D, et al. Research on LDPC decoding algorithm with high efficiency and low complexity for space-based information transmission[J]. Acta Armamentarii, 2023(2023-04-10)[2023-04-10]. http://www.co-journal.com/CN/10.12382/bgxb.2022.1044. (in Chinese)
[6]	SUN X B, WANG S K, WANG M H, et al. An advanced LiDAR point cloud sequence coding scheme for autonomous driving[C]// Proceedings of the 28th ACM International Conference on Multimedia. Seattle, WA, US: Association for Computing Machinery, 2020: 2793-2801.
[7]	YU F, LIU S S, ZHU Y H. Real-time spatio-temporal lidar point cloud compression[C]//Proceedings of the 2020 IEEE/RSJ International Conference on Intelligent Robots and Systems.Las Vegas, NV, US:IEEE, 2020: 10766-10773.
[8]	GAO P, LUO S Z, PAUL M. Rate-distortion modeling for bit rate constrained point cloud compression[J]. IEEE Transactions on Circuits and Systems for Video Technology, 2022, 33(5):2424-2438. doi: 10.1109/TCSVT.2022.3223898 URL
[9]	黄炜, 朱映韬, 陈冬杰, 等. 基于稀疏卷积的非对称特征聚合点云压缩算法[J]. 电视技术, 2022, 46(12):67-71,76.
	HUANG W, ZHU Y T, CHEN D J, et al. Sparse convolution based asymmetric feature aggregation algorithm for point cloud compression[J]. Video Engineering, 2022, 46(12) : 67-71,76. (in Chinese)
[10]	赵志, 许可, 马燕新, 等. 三维点云深度模型压缩算法[J]. 国防科技大学学报, 2023, 45(5):193-201.
	ZHAO Z, XU K, MA Y X, et al. Compression method for three-dimensional point cloud deep model[J]. Journal of National University of Defense Technology, 2023, 45(5):193-201. (in Chinese)
[11]	杜雨欣. 激光雷达点云数据压缩算法研究[D]. 西安: 西安电子科技大学, 2023.
	DU Y X. Research on LiDAR point cloud data compression algorithm[D]. Xi’an: Xidian University, 2023. (in Chinese)
[12]	DUAN C J, CHEN S H, KOVACEVIC J. 3D point cloud denoising via deep neural network based local surface estimation[C]// Proceedings of 2019 IEEE International Conference on Acoustics, Speech and Signal Processing. Brighton, UK: IEEE, 2019: 8553-8557.
[13]	SUN X B MAH, LIU M. A novel point cloud compression algorithm based on clustering[J]. IEEE Robotics and Automation Letters, 2019, 4(2): 2132-2139. doi: 10.1109/LRA.2019.2900747
[14]	RUSU R B, COUSINSS. 3D is here: Point cloud library (PCL)//Proceedings of 2011 IEEE International Conference on Robotics and Automation. Shanghai, China: IEEE, 2011.
[15]	黄源, 达飞鹏, 唐林. 基于改进八叉树的三维点云压缩算法[J]. 光学学报, 2017, 37(12): 1210003.
	HUANG Y, DA F P, TANG L. Three-dimensional point cloud compression algorithm based on improved octree[J]. Acta Optica Sinica, 2017, 37(12): 1210003. (in Chinese)
[16]	WANG Q, JIANG LY, SUN XB, et al. An efficient LiDAR point cloud map coding scheme based on segmentation and frame-inserting network[J]. Sensors, 2022, 22(14): 5108. doi: 10.3390/s22145108 URL
[17]	CHRISTENSEN A, LEHOTTSKY D, POULSEN M K, et al. Presenting a novel pipeline for performance comparison of V-PCC and G-PCC point cloud compression methods on datasets with varying properties[C]// Proceedings of the 17th International Joint Conference on Computer Vision, Imagingand Computer Graphics Theory and Applications.Red Hook,NY, US: Curran Associates,Inc., 2022: 387-393.
[18]	曾志贤, 曹建军, 翁年凤, 等. 结合关键帧提取的视频-文本跨模态实体分辨双重编码方法[J]. 兵工学报, 2022, 43(5): 1107-1116. doi: 10.12382/bgxb.2021.0262
	ZENG Z X, CAO J J, WENG N F, et al. Dual encoding integratingkeyframeextraction for video-text cross-modal entity resolution[J]. Acta Armamentarii, 2022, 43(5): 1107-1116. (in Chinese)
[19]	KAMMERL J, BLODOW N, RUSU R B, et al. Real-time compression of point cloud streams[C]// Proceedings of the 2012 IEEE International Conference on Robotics and Automation. Washington,DC, US: IEEE, 2012: 778-785.
[20]	吴晓军, 刘伟军, 王天然. 基于八叉树的三维网格模型体素化方法[J]. 工程图学学报, 2005, 26(4): 1-7.
	WU X J, LIU W J, WANG T R. Octree structure based voxelization of polygonal meshes[J]. Journal of Engineering Graphics, 2005, 26(4): 1-7. (in Chinese)
[21]	ZERMAN E, OZCINAR C, GAO P, et al. Textured mesh vs coloured point cloud: A subjective study for volumetric video compression[C]//Proceedings of the 12nd International Conference on Quality of Multimedia Experience. Athlone, Ireland:IEEE, 2020: 1-6.
[22]	HUANG Y, PENG J L, JAYKOU C C, et al. Octree-based progressive geometry coding of point clouds//Proceedings of Eurographics Symposium on Point-Based Graphics. 2006.DOI:10.2312/SPBG/SPBG06/103-110.
[23]	MAIMONE AFUCHSH. Encumbrance-free telepresence system with real-time 3D capture and display using commodity depth cameras//Proceedings of the 2011 10th IEEE International Symposium on Mixed and Augmented Reality.Basel, Switzerland:IEEE, 2011.
[24]	张卉冉, 董震, 杨必胜, 等. 点云压缩研究进展与趋势[J]. 武汉大学学报 (信息科学版), 2023, 48(2): 192-205.
	ZHANG H R, DONG Z, YANG B S, et al. Progress and perspectives of point cloud compression[J]. Geomatics and Information Science of Wuhan University, 2023, 48(2): 192-205. (in Chinese)
[25]	VAN RENSBURG B J, PUECH W, PEDEBOY J P. The first Draco 3D object crypto-Compression scheme[J]. IEEE Access, 2022, 10: 10566-10574. doi: 10.1109/ACCESS.2022.3144533 URL
[26]	VARISCHIO A, MANDRUZZATO F, BULLO M, et al. Hybrid point cloud semantic compression for automotive sensors: a performance evaluation[C]//Proceedings of IEEE International Conference on Communications. Montreal, QC, Canada:IEEE, 2021.
[27]	SUN X B, WANG S K, WANG M H, et al. A novel coding architecture for lidar point cloud sequence[J]. IEEE Robotics and Automation Letters, 2020, 5(4): 5637-5644. doi: 10.1109/LSP.2016. URL
[28]	FARAMARZI E, JOSHI R, BUDAGVI M. Mesh coding extensions to MPEG-IV-PCC[C]//Proceedings of the 2020 IEEE 22nd International Workshop on Multimedia Signal Processing. Tampere, Finland:IEEE, 2020.
[29]	HUANG T X, LIU Y. 3D point cloud geometry compression on deep learning[C]// Proceedings of the 27th ACM International Conference on Multimedia. New York,NY, US: Association for Computing Machinery, 2019: 890-898.
[30]	LUO G L, HE B Q, XIONG Y B, et al. An optimized convolutional neural network for the 3D point-cloud compression[J]. Sensors, 2023, 23(4): 2250. doi: 10.3390/s23042250 URL
[31]	WIEMANN T, IGELBRINK F, PUTZ S, et al. Compressing ROS sensor and geometry messages with draco[C]//Proceedings of the 2019 IEEE International Symposium on Safety, Security, and Rescue Robotics. Würzburg, Germany:IEEE, 2019: 243-248.
[32]	SUN X B, WANG S K, LIU M. A novel coding architecture for multi-line LiDAR point clouds based on clustering and convolutional LSTM network[J]. IEEE Transactions on Intelligent Transportation Systems, 2020, 23(3): 2190-2201. doi: 10.1109/TITS.2020.3034879 URL
[33]	律帅, 达飞鹏, 黄源. 基于数据类型转换的点云快速有损压缩算法[J]. 图学学报, 2016, 37(2): 199-205. doi: 10.11996/JG.j.2095-302X.2016020199
	LÜ S, DA F P, HUANG Y. A fast and lossy compression algorithm for point-cloud models based on data type conversion[J]. Journal of Graphics, 2016, 37(2): 199-205. (in Chinese)
[34]	DOUMANOGLOU A, DRAKOULIS P, ZIOULIS N, et al. Benchmarking open-source static 3D mesh codecs for immersive media interactive live streaming[J]. IEEE Journal on Emerging and Selected Topics in Circuits and Systems, 2019, 9(1): 190-203.
[35]	RUSU R B, MARTONZC, BLODOWN, et al. Towards 3D point cloud based object maps for household environments[J]. Robotics and Autonomous Systems, 2008, 56(11):927-941. doi: 10.1016/j.robot.2008.08.005 URL
[36]	DEVILLERSO,GANDOIN P M. Geometric compression for interactive transmission[C]// Proceedings Visualization 2000. Salt Lake City, UT, US: IEEE, 2000.
[37]	PENG J L, JAY KOUC C. Octree-based progressive geometry encoder[J]. Proceedongs of SPIE Internet Multimedia Management Systems IV. 2003. 5242.https://doi.org/10.1117/12.510857.
[38]	AHN J K, LEEK Y, SIMJ Y, et al. Large-scale 3D point cloud compression using adaptive radial distance prediction in hybrid coordinate domains[J]. IEEE Journal of Selected Topics in Signal Processing, 2014, 9(3): 422-434. doi: 10.1109/JSTSP.2014.2370752 URL