A Review of Visual Odometry for Intelligent Vehicles

doi:10.3969/j.issn.1000-1093.2012.02.014

Abstract

Abstract: Algorithms of the visual odometry for intelligent vehicles are reviewed. Firstly, the basic principle of the visual odometry is introduced. Secondly, the classification and comparison are made according to types of visual information, the configuration of cameras and the calculation framework. Thirdly, the key techniques including feature selection and matching, robust motion estimation in two-frame visual odometry system are analyzed. Finally the development trends of the visual odometry algorithms are discussed.

Key words: information processing, intelligent vehicles, visual odometry, motion estimation

CLC Number:

TP242.6

JIANG Yan-hua, XIONG Guang-ming, JIANG Yan, GONG Jian-wei, CHEN Hui-yan. A Review of Visual Odometry for Intelligent Vehicles[J]. Acta Armamentarii, 2012, 33(2): 214-220.

References

［1］吴功伟. 立体视觉里程计的关键技术研究[D]. 杭州：浙江大学，2007.
WU Gong-wei. Research on key technologies in stereo visual odometry[D]. Hangzhou: Zhejiang University, 2007. (in Chinese)
[2] Moravec H. Obstacle avoidance and navigation in the real world by a. seeing robot rover[D]. Stanford: Univ. of Stanford, 1980.
[3] Matthies L, Shafer S A. Error modeling in stereo navigation[J]. IEEE J. Robot. Automat., 1987, RA23(3)：239-250.
[4] 彭勃, 周文晖, 刘济林. 基于Harris角点检测的立体视觉里程计[J]. 兵工学报，2007，28(12)：1498-1502.
PENG Bo, ZHOU Wen-hui, LIU Ji-lin. Harris corner detection-based stereo visual odometry[J]. Acta Armanmentarii, 2007, 28(12)：1498-1502. (in Chinese)
[5] Eade E, Drummond T. Scalable monocular SLAM[C]∥Proc. IEEE Computer Society Conference on Computer Vision and Pattern Recognition, New York: IEEE, 2006: 469-476.
[6] Davison A J, Reid I D, Molton N D, et al. Mono SLAM: real-time single camera SLAM[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2007, 29(6): 1052-1067.
[7] Davison A J. Real-time simultaneous localization and mapping with a single camera[C]∥Proc. International Conference on Computer Vision, Nice: IEEE, 2003: 1403-1410.
[8] 马玉娇. 基于立体视觉里程计的移动机器人定位研究[D]. 武汉：武汉科技大学，2009.
MA Yu-jiao. Research on mobile robot localization based on stereo visual odometry[D]. Wuhan: Wuhan University of Science and Technology, 2009. (in Chinese)
[9] Badino H. A robust approach for ego-motion estimation using a mobile stereo platform[J]. Lecture Notes in Computer Science, 2007, 3417: 198-208.
[10] Nistér D, Naroditsky O, Bergen J. Visual odometry[C]∥IEEE Computer Society Conference on Computer Vision and Pattern Recognition, Washington: IEEE, 2004: 652-659.
[11] Mouragnon E, Lhuillier M, Dhome M, et al. Real time localization and 3D reconstruction[C]∥IEEE Computer Society Conference on Computer Vision and Pattern Recognition, New York: IEEE, 2006: 363-370.
[12] Tardif J, Pavlidis Y, Daniilidis K. Monocular visual cdometry in urban environments using an omnidirectional camera[C]∥IEEE/RSJ International Conference on Intelligent Robots and Systems, Nice: IEEE, 2008: 2531-2538.
[13] Corke P, Strelow D, Singh S. Omnidirectional visual odometry for a planetary rover[C]∥IEEE/RSJ International Conference on Intelligent Robots and Systems, IEEE, 2004: 4007-4012.
[14] Agrawal M, Konolige K. Rough terrain visual odometry[C]. International Conference on Advanced Robotics, Menlo Park: 2007.
[15] Hirschmüller H, Innocent P R, Garibaldi J M. Fast, unconstrained camera motion estimation from stereo without tracking and robust statistics[C]∥In International Conference on Control, Automation, Robotics and Vision, Singapore: Nayang Technological Univ, 2002: 1099-1104.
[16] Milella A, Reina G. Vision-based methods for mobile robot localization and wheel sinkage estimation[C]∥ASME Dynamic Systems and Control Conference, Michigan: 2008: 625-621.
[17] Milella A, Siegwart R. Stereo-based ego-motion estimation using pixel-tracking and iterative closest point[C]∥Proc. IEEE International Conference on Computer Vision Systems, New York: IEEE, 2006: 21-27.
[18] Kitt B, Geiger A, Lategahn H. Visual odometry based on stereo image sequences with RANSAC-based outlier rejection scheme[C]∥IEEE Intelligent Vehicle Symposium, San Diego: IEEE, 2010: 486-492.
[19] Howard A. Real-time stereo visual odometry for autonomous ground vehicles[C]∥IEEE/RSJ International Conference on Intelligent Robots and Systems, Nice: IEEE, 2008: 3946-3952.
[20] García-García R, Sotelo M A, Parra I, et al. 3D visual odometry for road vehicles[J]. Journal of Intelligent and Robotic Systems Archive, 2008, 51(1): 113-134.
[21] Sappa A D, Dornaika F, Ponsa D, et al. An efficient approach to onboard stereo vision system POSE estimation[J]. IEEE Trans. Intell. Transp. Syst, 2008, 9(3): 476-490.
[22] Jong Weon Lee, Suya You, Ulrich Neumann. Large motion estimation for omnidirectional vision[C]∥IEEE Workshop on Omnidirectional Vision, Hilton Head Island: IEEE, 2000: 161-168.
[23] Wen Lik Dennis Lui, Ray A. Jarvis. An omnidirectional vision system for outdoor mobile robots[C]∥IEEE Workshop on Omnidirectional Robot Vision, Anchorage: IEEE, 2008: 273-284.
[24] Parra I, Sotelo M A, Llorca D F. Robust visual odometry for vehicle localization in urban environments[J]. Robotica, 2010, 28(3): 441-452.
[25] Parra I, Sotelo M A, Vlacic L. Robust visual odometry for complex urban environments[C]∥IEEE Intelligent Vehicles Symposium, Eindhoven: IEEE, 2008: 440-445.
[26] Triggs B, McLauchlan P F, Hartley R I, et al. Bundle adjustment-a modern synthesis[J]. In Vision Algorithms: Theory and Practice, ser. LNCS. Springer Verlag, 2000, 1883: 298-375.
[27] Shum H, K E Q, Zhang Z. Efficient bundle adjustment with virtual key frames: a hierarchical approach to multi-frame structure from motion[C]∥IEEE Computer Society Conference on Computer Vision and Pattern Recognition, Collins: IEEE, 1999: 538-543.
[28] Jungho Kim,In So Kweon. Vision-based autonomous navigation based on motion estimation[C]∥International Conference on Control, Seoul: Automation and Systems, 2008: 1738-1743.
[29] Royer E, Lhuillier M, Dhome M, et al. Monocular vision for mobile robot localization and autonomous navigation[J]. International Journal of Computer Vision, 2007, 74(3): 237-260.
[30] Levin A, Szeliski R. Visual odometry and map correlation[C]∥IEEE Computer Society Conference on Computer Vision and Pattern Recognition, Washington: IEEE, 2004:1611-1618.
[31] Konolige K, Agrawal M, Sola J. Large scale visual odometry for rough terrain[C]∥In International Symposium on Research in Robotics, Hiroshima: 2007.
[32] Kwolek B. Visual odometry based on gabor filters and sparse bundle adjustment[C]∥IEEE International Conference on Robotics and Automation, Roma: IEEE, 2007: 3573-3578.
[33] Chiuso A, Favaro P, Jin H, et al. Structure from motion causally integrated over time[J]. IEEE Trans. on Pattern Anal. Mach. Intell., 2002, 24(4): 523-535.
[34] Rudolph van der Merwe, Eric A. Wan. Sigma-point kalman filters for integrated navigation[C]∥Proc. Annual Meeting of The Institute of Navigation, Dayton: NSF, 2004: 641-654.
[35] Mclauchlan P, Reid I, Murray D. Recursive affine structure and motion from image sequences[C]∥Proc. European Conf. Comp. Vision, Stockholm: Springer Verlag, 1994: 217-224.
[36] García-García R G, Sotelo M A, Parra I, et al. 3D visual odometry for GPS navigation assistance[C]∥Proc. IEEE Intelligent Vehicles Symposium, Istanbul: IEEE, 2007: 444-449.
[37] Zhu Z, Oskiper T, Samarasekera S, et al. Sawhney. Ten-fold improvement in visual odometry using landmark matching[C]∥IEEE International Conference on Computer Vision, Rio de Janeiro: IEEE, 2007: 1747-1754.
[38] Oskiper T, Zhu Z, Samarasekera S. Visual odometry system using multiple stereo cameras and inertial measurement unit[C]∥IEEE Conference on Computer Vision and Pattern Recognition, Minneapolis: IEEE, 2007: 976-983.
[39] P. H. S . Torr and A. Zisserman. Feature based methods for structure and motion estimation[J]. in Vision Algorithms: Theory and Practice, 2000, 1883: 278-295.
[40] Taylor C J, Kriegman D J. Structure and motion from line segments in multiple images[J]. IEEE Trans. on Pattern Anal. Mach. Intell., 1995, 17(11): 1021-1032.
[41] Wong K Y K, Mendonca P R S, Cipolla R. Structure and motion estimation from apparent contours under circular motion[J]. Image and Vision Computing, 2002, 20: 441-448.
[42] Schmid C, Mohr R, Baukhage C. Evaluation of interest point detectors[J]. International Journal of Computer Vision, 2000, 37(2): 151-172.
[43] Lowe D G. Distinctive image features from scale-invariant keypoints[J]. International Journal of Computer Vision, 2004, 60(2): 91-110.
[44] 李智, 周文晖, 刘济林. 动态场景下基于视差空间的立体视觉里程计[J]. 浙江大学学报（工学版），2008，42(10)：1661-1665.
LI Zhi, ZHOU Wen-hui, LIU Ji-lin. Stereo visual odometry from disparity space in dynamic environments[J]. Journal of Zhejiang University: Engineering Science, 2008, 42(10): 1661-1665. (in Chinese)
[45] Gong P L, Zhang Q F, Zhang A Q. Stereo vision based motion estimation for underwater vehicles[C]∥International Conference on Intelligent Computation Technology and Automation, Changsha: IEEE, 2009: 745-749.
[46] Rodriguez S A, Fremont F V, Bonnifait P. An experiment of a 3D real-time robust visual odometry for intelligent vehicles[C]∥Proc. International Conference on Intelligent Transportation Systems, Shizuoka: IEEE, 2009: 226-231.
[47] Zhang T, Liu X, Kuhnlenz K, et al. Visual odometry for the autonomous city explorer[C]∥IEEE/RSJ International Conference on Intelligent Robots and Systems, St. Louis: IEEE, 2009: 3513-3518.
[48] Gordon I, Lowe D G. What and where: 3D object recognition with accurate pose[C]∥International Symposium on Mixed and Augmented Reality, Santa Barbara: IEEE, 2006: 67-82.
[49] Horn B K P. Relative orientation[J]. International Journal of Computer Vision, 1990, 4: 59-78.
[50] Nistér D. An efficient solution to the five-point relative pose problem[J]. IEEE Trans. on Pattern Anal. Mach. Intell., 2004, 26(6): 756-770.
[51] Fischler M A, Bolles R C. Random sample consensus: a paradigm for model fitting with apphcatlons to image analysis and automated cartography[J]. Graphics and Image Processing, 1981, 24(6): 381-395.
[52] Nister D. Preemptive RANSAC for live structure and motion estimation[C]∥Proc. International Conference on Computer Vision, Nice: 2003: 199-206.
[53] Scaramuzza D, Fraundorfer F, Siegwart R. Real-time monocular visual odometry for on-road vehicles with 1-point RANSAC[C]∥International Conference on Robotics and Automation, Kobe: 2009: 4293-4299.
[54] Nourani-Vatani N, Roberts J, Srinivasan M V. Practical visual odometry for car-like vehicles[C]∥International Conference on Robotics and Automation, Kobe: 2009: 3551-3557.
[55] Haralick R, Lee C, Ottenberg K, et al. Review and analysis of solutions of the three point perspective pose estimation problem[J]. International Journal of Computer Vision, 1994, 13(3): 331-356.
[56] Nistér D. A minimal solution to the generalised 3-Point pose problem[C]∥IEEE Computer Society Conference on Computer Vision and Pattern Recognition, Washington: IEEE, 2004: 560-567.
[57] Horn B K P, Hilden H M, Negahdaripour S. Closed-form solution of absolute orientation using orthonormal matrices[J]. Journal Opt. Soc. Am. A, 1988, 5: 1127-1135.
[58] Horn B K P. Closed-form solution of absolute orientation using unit quaternions[J]. Journal Opt. Soc. Am. A, 1987, 4(4): 629-642.
[59] Stéwenius H, Engels C, Nistér D. Recent developments on direct relative orientation[J]. ISPRS Journal of Photogrammetry and Remote Sensing, 2006, 60: 284-294.
[60] Matthies L, Maimone M, Johnson A, et al. Computer vision on mars[J]. International Journal of Computer Vision, 2007, 75(1): 67-92.
[61] Kitt B, Moosmann F, Stiller C. Moving on to dynamic environments: visual odometry using feature classification[C]∥International Conference on Intelligent Robots and Systems, Taipei: IEEE, 2010: 5551-5556.
[62] Sakai A, Mitsuhashi M, Kuroda Y. Noise model creation for visual odometry with neural-fuzzy model[C]∥IEEE/RSJ International Conference on Intelligent Robots and Systems, Taipei: IEEE, 2010: 5190-5195.
[63] Sáez J M, Escolano F. 6DOF entropy minimization SLAM for stereo-based wearable devices[J]. Computer Vision and Image Understanding, 2011, 115(2): 270-285.

[1]	JIANG Yan, DING Yuyan, ZHANG Xinglong, XU Xin. A Human-machine Collaborative Control Algorithm for Intelligent Vehicles Based on Model Prediction and Policy Learning [J]. Acta Armamentarii, 2023, 44(11): 3465-3477.
[2]	CAO Haozhe, LIU Quanpan. Unmanned Swarm Collaborative Visual SLAM Algorithm Based on Semi-direct Method [J]. Acta Armamentarii, 2023, 44(11): 3345-3358.
[3]	LIU Yong-qing， JIANG Shuo， YU Dong， SHAO Xiao-tian. Effect of Narrowband Interference Suppression on Pseudorandom Code Zero Value [J]. Acta Armamentarii, 2016, 37(2): 293-298.
[4]	GUAN Si-hai， LI Zhi, HUANG Hui, WANG Zhe. Modified Zero-Attracting l₀-NLMS Algorithm [J]. Acta Armamentarii, 2016, 37(11): 2170-2176.
[5]	JIAO Chuan-hai, LI Yong-cheng, XIE Kai, YANG Yun-fu. Wideband Compressed Blind Sensing without Reconstruction Based on Expectation Deviation and Generalized Likelihood RatioTest [J]. Acta Armamentarii, 2016, 37(10): 1837-1843.
[6]	GUO Ye-cai, WU Hua-peng, WANG Hui, ZHANG Miao-qing. DNA Genetic Bat Algorithm Based Fractionally Spaced Multi-modulus Algorithm [J]. Acta Armamentarii, 2015, 36(8): 1502-1507.
[7]	ZHU Ming-qiang， HOU Jian-jun, LIU Ying ， LI Xu, TIAN Hong-juan. An Time Delay Difference Estimation Algorithm Based on Improved Unscented Particle Filter Suitable for Target Tracking inWireless Sensor Network [J]. Acta Armamentarii, 2015, 36(7): 1266-1272.
[8]	LI Zheng-zhou, HOU Qian, DAI Zhen, FU Hong-xia, GE Feng-zeng, JIN Gang. Dim Moving Target Detection Algorithm Based on Spatial-temporal Sparse Representation [J]. Acta Armamentarii, 2015, 36(7): 1273-1279.
[9]	GUO Ye-cai， ZHANG Miao-qing. A Multi-modulus Blind Equalization Algorithm Based on Shuffled Frog Leaping Algorithm [J]. Acta Armamentarii, 2015, 36(7): 1280-1287.
[10]	LIU Kun, MA Wen-ping, LIU Hong-ying. Speckle Suppression of Polarimetric Synthetic Aperture Radar Images Based on Sublook Data Correlation [J]. Acta Armamentarii, 2015, 36(7): 1288-1294.
[11]	ZHU Xia, CHEN Ren-wen, XIA Hua-kang, ZHANG Piao-yan. A Novel Image Registration Based on Cultural Particle Swarm Optimization Algorithm [J]. Acta Armamentarii, 2015, 36(6): 1033-1039.
[12]	DENG Bing, LUAN Jun-bao. Study of Reconnaissance Reception Algorithm of Wide-band Channelization in the Fractional Fourier Domain [J]. Acta Armamentarii, 2015, 36(6): 1040-1045.
[13]	WU Yi-quan， SONG Yu. Background Suppression of Small Infrared Target Image Based on Nonsubsampled Complex Contourlet Transform and GaussianWavelet Support Vector Regression [J]. Acta Armamentarii, 2015, 36(4): 687-695.
[14]	ZHANG Liang-jun, YANG Jie, LU Kai-wang, SUN Ya-dong. Underdetermined Blind Source Separation Based on Time-frequency Method Using Cyclostationary Characteristic [J]. Acta Armamentarii, 2015, 36(4): 703-709.
[15]	JIANG Run-xiang， LIN Chun-sheng， GONG Shen-guang. Electrostatic Electric Field Inversion Method for Ship Based on Point Charge Source Model [J]. Acta Armamentarii, 2015, 36(3): 545-551.