Please wait a minute...
浙江大学学报(工学版)
Journal of ZheJiang University (Engineering Science)  2021, Vol. 55 Issue (8): 1500-1509    DOI: 10.3785/j.issn.1008-973X.2021.08.011
    
Monocular visual odometry based on improved Census transform
Zhi-wei LIN(),Qi-min LI*(),Xian-yu WANG
State Key Laboratory of Mechanical Transmissions, Chongqing University, Chongqing 400044, China
Download: HTML     PDF(1702KB) HTML
Export: BibTeX | EndNote (RIS)      

Abstract  

A monocular visual odometry based on improved Census transform, vector-Census semi-direct monocular visual odometry (VC-SVO), was proposed, in order to solve the failure problem of the direct visual odometry in the scene of illumination changes. Census transform is a kind of non-parametric transformation in the field of stereo vision, which can effectively reduce the impact of illumination change on the image. The Census transform was introduced into the back-end optimization in SLAM, the expression of traditional Census transform was improved, and the improved Census transform was transformed into Euclidean space successfully. At the same time, a new method to measure the degree of difference was designed. The non-plane hypothesis model of the scene was added into SVO algorithm. The SVO algorithm was extended and the improved Census transform was integrated. The Census transform was integrated to obtain more accurate camera pose by minimizing the Census transform error of the map points, at the same time, the environment map was constructed. Image experiment results on EuRoC, New Tsukuba Stereo and TUM dataset demonstrate the effectiveness and accuracy of the algorithm in illumination change situation. The accuracy and robustness were better than that of the open source SVO and large-scale direct monocular simultaneous localization and mapping (LSD-SLAM) algorithm.

Key words： improved Census transform      semi-direct monocular visual odometry (SVO)      simultaneous localization and mapping (SLAM)      semi-direct method      monocular vision     
Received: 29 July 2020      Published: 01 September 2021
CLC:  TP 242  
Fund:  国家自然科学基金资助项目（61773254，U1813217）；上海市科委资助项目（17DZ1205000）
Corresponding Authors: Qi-min LI     E-mail: 201807021013@cqu.edu.cn;qim_li@163.com
Service
E-mail this article
Add to my bookshelf
Add to citation manager
E-mail Alert
RSS
Articles by authors
Zhi-wei LIN
Qi-min LI
Xian-yu WANG
Cite this article:

Zhi-wei LIN,Qi-min LI,Xian-yu WANG. Monocular visual odometry based on improved Census transform. Journal of ZheJiang University (Engineering Science), 2021, 55(8): 1500-1509.

URL:

https://www.zjujournals.com/eng/10.3785/j.issn.1008-973X.2021.08.011     OR     https://www.zjujournals.com/eng/Y2021/V55/I8/1500


基于改进Census变换的单目视觉里程计

针对直接法视觉里程计在光照变化场景下的失效问题，提出基于改进Census变换的单目视觉里程计，向量Census变换半直接单目视觉里程计（VC-SVO）. Census变换是立体视觉领域中非参数变换的一种，可以有效减少光照变化对图像的影响. 将Census变换引入SLAM中的后端优化，改变传统Census变换的形式，转换到欧氏空间中表示，并采用新的误差计算方法. 在SVO算法中增添非平面假设模型，扩展SVO算法并融合改进后的Census变换，通过最小化地图点的Census变换误差来得到更准确的相机位姿，同时构建环境地图. 在EuRoC、New Tsukuba Stereo与TUM公开数据集上的图像实验表明，VC-SVO实现了光照变化情况下的位姿估计，验证了算法的有效性. VC-SVO算法的精度和鲁棒性要优于已开源的SVO和基于直接法的大范围定位和地图构建（LSD-SLAM）算法.


关键词： 改进Census变换,  半直接单目视觉里程计（SVO）,  同时定位与建图（SLAM）,  半直接法,  单目视觉 
Fig.1 Schematic diagram of Census transform
Fig.2 Overall design of VC-SVO algorithm
Fig.3 Effect of homogenizing features
Fig.4 MH_01 sequence homogenization comparison
图像序列 改进前数量 改进后数量
MH_01 84 329
MH_02 77 301
MH_03 65 296
Tab.1 Comparison of number of key frames
算法步骤 耗时/ms 总时间/ms
特征点提取 5.74 90.53
KLT光流追踪 67.88
V-Census变换 6.87
计算位姿 10.04
优化像素位置 1.83 2.01
优化位姿 0.16
优化地图点 0.02
Tab.2 Time cost of each step of algorithm
Fig.5 MH_02 sequence trajectory comparison
Fig.6 Local enlarged figure of MH_02 sequence
算法 RMSE/ms?1 M/ms?1
SVO 0.0285990 0.0238094
LSD-SLAM 0.0275552 0.0232427
VC-SVO 0.0258653 0.0229387
Tab.3 Error of each algorithm under MH_02 sequence
Fig.7 Building of sparse map
Fig.8 Typical image brightness changes
Fig.9 MH_03 sequence trajectory comparison
Fig.10 Local enlarged figure of MH_03 sequence
Fig.11 Dark scene in MH_03 sequence
Fig.12 Illumination enhancement scene in daylight sequence
Fig.13 Daylight sequence trajectory comparison
算法 RMSE/ms?1 M/ms?1
SVO 4.06884 2.21581
LSD-SLAM 3.38588 2.31256
VC-SVO 1.75917 1.18218
Tab.4 Error of each algorithm under daylight sequences
Fig.14 Effect of light on brightness change
Fig.15 Comparison of rotation trajectory of fr1/360 sequence
Fig.16 Local enlarged figure of Fr1/360 sequence
[1]   SMITH R, CHEESEMAN P On the representation and estimation of spatial uncertainty[J]. International Journal of Robotics Research, 1986, 5 (4): 56- 68
doi: 10.1177/027836498600500404
[2]   SMITH R, SELF M, CHESSEMAN P. Estimating uncertain spatial relationships in robotics[C]// Autonomous Robot Vehicles. NewYork: IEEE, 1990: 167–193.
[3]   刘敏浩, 章国锋, 鲍军虎 基于单目视觉的同时定位与地图构建算法综述[J]. 计算机辅设计与图形学学报, 2016, 28 (6): 855- 868
LIU Min-hao, ZHANG Guo-feng, BAO Jun-hu Overview of simultaneous localization and mapping algorithms based on monocular camera[J]. Journal of Computer-Aided Design and Computer Graphics, 2016, 28 (6): 855- 868
[4]   KROMBACH N, DROESCHEL D, BEHNKE S. Combining feature-based and direct methods for semidense real-time stereo visual odometry[C]// International Conferenceon Intelligent Autonomous Systems. Berlin: IEEE, 2016: 855-868.
[5]   ENDRESS F, HESS J, STURM J, et al 3D mapping with an RGB-D camera[J]. IEEE Transactions on Robotics, 2017, 30 (1): 177- 187
[6]   MUR-ARTAL R, MONTIEL J M, TARDOS J D ORB-SLAM: a versatile and accurate monocular SLAM system[J]. IEEE Transactions on Robotics, 2015, 31 (5): 1147- 1163
doi: 10.1109/TRO.2015.2463671
[7]   LOWE D Distinctive image features from scale-invariant keypoints[J]. International Journal of Computer Vision, 2004, 60 (2): 91- 110
doi: 10.1023/B:VISI.0000029664.99615.94
[8]   ENGEL J, SCHOPS T, CREMERS D. LSD-SLAM: Large-scale direct monocular SLAM[C]// European Conference on Computer Vision. Berlin: IEEE, 2014: 834-849.
[9]   ENGEL J, KOLTUN V, CREMERS D Direct sparse odometry[J]. Transactions on Pattern Analysis and Machine Intelligence, 2018, 40 (3): 611- 625
doi: 10.1109/TPAMI.2017.2658577
[10]   FORSTER C, PIZZOLI M, SCARAMUZZA D. SVO: fast semi-direct monocular visual odometry[C]// International Conference on Robotics and Automation. Piscataway: IEEE, 2014: 15-22.
[11]   MURARTAL R, TARDOS J D ORB-SLAM2: an open-source SLAM system for monocular, stereo, and RGB-D cameras[J]. Transactions on Robotics, 2017, 33 (5): 1255- 1262
doi: 10.1109/TRO.2017.2705103
[12]   PARK S, SCHOPS T, POLLEFEYS M. Illumination change robustness in direct visual SLAM[C]// International Conference on Robotics and Automation. Singapore: IEEE, 2017: 4523-4530.
[13]   ZABIH R, WOODFILL J. Non-parametric local transforms for computing visual correspondence[C]// European Conference on Computer Vision. Berlin: Springer, 1994, 801: 151-158.
[14]   PIZZOLI M, FORSTER C, SCARAMUZZA D. REMODE: probabilistic, monocular dense reconstruction in real time[C]// 2014 IEEE International Conference on Robotics and Automation. Hong Kong: IEEE, 2014: 2609–2616.
[15]   BURRI M, NIKOLIC J, GOHL P, et al The EuRoC micro aerial vehicle datasets[J]. International Journal of Robotics Research, 2016, 35 (10): 1157- 1163
doi: 10.1177/0278364915620033
[16]   MARTULL S, MARTIN P, KAZUHIRO F. Realistic CG stereo image dataset with ground truth disparity maps[C]// International Conference on Pattern Recognition Workshop. TrakMark: IEEE, 2012, 111: 117–118.
[17]   OJALA T, PIETIKAINEN R, HARWOOD D A comparative study of texture measures with classification based on featured distributions[J]. Pattern Recognition, 1996, 29 (1): 51- 59
doi: 10.1016/0031-3203(95)00067-4
[18]   VOGEL C, ROTH S, SCHINDLER K. An evaluation of data costs for optical flow[C]// 2013 German Conference on Pattern Recognition. Berlin: Springer, 2013, 8142: 343-353.
[19]   高翔, 张涛, 颜沁睿, 等. 视觉SLAM十四讲: 从理论到实践[M]. 北京: 电子工业出版社, 2017.
[20]   BAKER S, GROSS R, MATTHEWS I Lucas Kanade 20 years on: a unifying framework[J]. International Journal of Computer Vision, 2004, 56 (3): 221- 255
doi: 10.1023/B:VISI.0000011205.11775.fd
[21]   HARTLEY R, ZISSERMAN A. Multiple view geometry in computer vision[M]. London: Cambridge University Press, 2003.
[22]   ROSTEN E, PORTER R, DRUMMOND T Faster and better: a machine learning approach to corner detection[J]. IEEE Transactions on Software Engineering, 2010, 32 (1): 105- 119
[23]   NEUBECK A, GOOLL J. Efficient non-maximum suppression[C]// International Conference on Pattern Recognition. Hong Kong: IEEE, 2006, 479: 850-855.
[1] Xin MA,Xin-wu LIANG,Ji-yuan CAI. Fast visual SLAM method based on point and line features[J]. Journal of ZheJiang University (Engineering Science), 2021, 55(2): 402-409.
[2] WANG Kai, YUE Bo-xuan, FU Jun-wei, LIANG Jun. Image restoration and fault tolerance of stereo SLAM based on generative adversarial net[J]. Journal of ZheJiang University (Engineering Science), 2019, 53(1): 115-125.
[3] ZHAO Li-ke, ZHENG Shun-yi, WANG Xiao-nan, HUANG Xia. Rigid object position and orientation measurement based on monocular sequence[J]. Journal of ZheJiang University (Engineering Science), 2018, 52(12): 2372-2381.
[4] ZHENG Chi, XIANG Zhi-yu, LIU Ji-lin. Monocular vision odometry based on the fusion of optical flow and feature points matching[J]. Journal of ZheJiang University (Engineering Science), 2014, 48(2): 279-284.
[5] CHEN Jia-Gan, HE Yan, JIANG Jing-Ping. Improved FastSLAM algorithm based on importance weight smoothing[J]. Journal of ZheJiang University (Engineering Science), 2010, 44(8): 1454-1459.