Inter frame point clouds registration algorithm for pose optimization of depth camera

doi:10.3785/j.issn.1008-973X.2019.09.014

Journal of ZheJiang University (Engineering Science)

2019, Vol. 53

Issue (9): 1749-1758 DOI: 10.3785/j.issn.1008-973X.2019.09.014

Computer Science and Artificial Intelligence

Inter frame point clouds registration algorithm for pose optimization of depth camera

Xing-dong LI1,3(

),He-wei GAO1,Long SUN2,3,*(

)

1. College of Mechanical and Electrical Engineering, Northeast Forestry University, Harbin 150040, China
2. College of Forestry, Northeast Forestry University, Harbin 150040, China
3. Northern Forest Fire Management Key Laboratory of the State Forestry and Grassland Bureau, Northeast Forestry University, Harbin 150040, China

Download:

HTML

PDF(1788KB) HTML
Export: BibTeX | EndNote (RIS)

Abstract

TOF (time of flight) camera can collect gray and depth images simultaneously to optimize the estimation of camera pose. Graph-based adjustment structure was applied to optimize the poses of the TOF camera in acquiring several frames. Registration between frames is a key operation which determines both the efficiency and effectiveness of the camera pose optimization. Scale invariant features were detected from a pair of images and matched subsequently. After the 2D feature points were extended into 3D space, two point clouds were registered in terms of relative positions between the features and the normal 3D points. Among all of the point clouds participating in the optimization of camera pose, any two point clouds were registered pair by pair using the proposed registering method. Lastly, the graph based algorithm was employed to adjust the camera poses, with inputs of the valid pairs of registered point clouds. Results demonstrated that the proposed method can improve the precision of the optimized camera pose, and the estimating efficiency is guaranteed.

Key words： 3D vision time of flight (TOF) camera pose inter frame registration graph-based structure

Received: 09 August 2018 Published: 12 September 2019

CLC:

TP 242

Corresponding Authors: Long SUN E-mail: lixd@nefu.edu.cn;13945016458@126.com

	Service
	E-mail this article
	Add to my bookshelf
	Add to citation manager
	E-mail Alert
	RSS
	Articles by authors
	Xing-dong LI
	He-wei GAO
	Long SUN

Cite this article:

Xing-dong LI,He-wei GAO,Long SUN. Inter frame point clouds registration algorithm for pose optimization of depth camera. Journal of ZheJiang University (Engineering Science), 2019, 53(9): 1749-1758.

URL:

http://www.zjujournals.com/eng/10.3785/j.issn.1008-973X.2019.09.014 OR http://www.zjujournals.com/eng/Y2019/V53/I9/1749

面向深度相机位姿优化的帧间点云数据配准算法

TOF相机能够同时采集灰度图像和深度图像从而优化相机位姿的估计值. 应用图结构调整框架优化多帧数据采集时的相机位姿，采用帧间配准决定优化的精度和效率. 从2帧图像上提取并匹配尺度不变特征点对，二维特征点被扩展到三维空间后，利用与特征点的空间位置关系将2帧三维点云配准；逐步应用提出的算法配准参与位姿优化的多帧点云中的任意2帧点云；最后将有效配准的点云帧对作为输入数据，采用图结构算法优化位姿. 实验结果表明，提出的帧间配准算法使得位姿估计值精度显著提高，同时保证了估计效率.

关键词： 三维视觉, TOF相机, 位姿, 帧间配准, 图结构

Fig.1 2D image feature and 3D point cloud feature pairs

Fig.2 Pairs of 3D point sets searched with 3D feature points as spherical centers

Tab.1 Searching radius and 3D point sets extracted from two point clouds

Fig.3 Pixel set related to subset of 3D points divided by quadrant method

Fig.4 Flow chart for pose optimization of depth camera

Fig.5 Experimental environment for capturing original data of depth camera

Fig.6 Comparison of average rotation errors for executing optimization algorithm by several times

Fig.7 Comparison of average translation errors for executing optimization algorithm by several times several

Fig.8 Average time for executing graph structure optimization algorithm by once

Fig.9 Multiple original grayscale images for data registration experiment

Fig.10 Comparison Registration for six point clouds from six frames data after pose optimization using different algorithms


[1]	LANGE R. 3D time-of-flight distance measurement with custom solid-state image sensors in CMOS/CCD technology [D]. Siegen: University of Siegen, 2000: 14–56.

[2]	王宇, 朴燕, 孙荣春结合同场景彩色图像的深度图超分辨率重建[J]. 光学学报, 2017, 37 (8): 102- 108 WANG Yu, PIAO Yan, SUN Rong-chun Depth image super-resolution construction combined with high-resolution color image of the same scene[J]. Acta Optica Sinica, 2017, 37 (8): 102- 108

[3]	李诗锐, 李琪, 李海洋, 等基于Kinect v2 的实时精确三维重建系统[J]. 软件学报, 2016, 27 (10): 2519- 2529 LI Shi-rui, LI Qi, LI Hai-yang, et al Real-time accurate 3D reconstruction based on Kinect v2[J]. Journal of Software, 2016, 27 (10): 2519- 2529

[4]	HENRY P, KRAININ M, HERBST E, et al RGB-D mapping: using Kinect-style depth cameras for dense 3D modeling of indoor environments[J]. International Journal of Robotics Research, 2012, 31 (5): 647- 663 doi: 10.1177/0278364911434148

[5]	FOIX S, ALENY G, ANDRADE-CETTO J, et al. Object modeling using a TOF camera under an uncertainty reduction approach [C] // Proceedings of IEEE International Conference on Robotics and Automation. Anchorage: IEEE, 2010: 1306-1312.

[6]	DELLEN B, ALENY G, FOIX S, et al. 3D object reconstruction from swissranger sensor data using a spring-mass model [C] // Proceedings of the Fourth International Conference on Computer Vision Theory and Applications. Lisboa: [s. n.], 2009: 368-372.

[7]	SARKER A, GEPPERTH A, HANDMANN U, et al. Dynamic hand gesture recognition for mobile systems using deep LSTM [C] // Proceedings of the international Conference on Intelligent Human Computer Interaction. Evry: Springer, 2017: 19-31.

[8]	GOKTURK S B, TOMASI C. 3D head tracking based on recognition and interpolation usinga time-of-flight depth sensor [C] // Proceedings of the 18th IEEE Conference on Computer Vision and Pattern Recognition. Washington: IEEE, 2004: 211-217.

[9]	LEONARD J, DURRAN-WHYTE H. Simultaneous map building and localization for an autonomous mobile robot [C] // Proceedings of the IEEE Conference on Intelligent Robots and Systems. Osaka: IEEE, 1991: 1442-1447.

[10]	BESL P J, MCKAY N D A method for registration of 3D shapes[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 1992, 14 (2): 239- 256 doi: 10.1109/34.121791

[11]	DONG J, PENG Y, YING S, et al LieTrICP: an improvement of trimmed iterative closest point algorithm[J]. Neurocomputing, 2014, 140: 67- 76 doi: 10.1016/j.neucom.2014.03.035

[12]	MAY S, DROESCHEL D, FUCHS S. Robust 3D-mapping with time of light cameras [C] // Proceedings of the IEEE Conference on Intelligent Robots and Systems. St. Louis: IEEE, 2009: 1673-1678.

[13]	SMITH R C, CHEESEMAN P On the representation and estimation of spatial uncertainty[J]. International Journal of Robotics Research, 1986, 5 (4): 56- 68 doi: 10.1177/027836498600500404

[14]	侯冰洁, 杨彦甫, 向前, 等基于EKF和LKF级联的频偏和相位估计联合方案[J]. 光学学报, 2018, 38 (1): 85- 90 HOU Bing-jie, YANG Yan-pu, XIANG Qian, et al A joint frequency offset and phase estimation scheme based on cascaded EKF and LKF[J]. Acta Optica Sinica, 2018, 38 (1): 85- 90

[15]	MONTEMERLO M, THRUN S, KOLLER D, et al. FastSLAM: a factored solution to the simultaneous localization and mapping problem [C] // Proceedings of the National Conference on Artificial Intelligence. Alberta: AAAI Press, 2002: 593-598.

[16]	LU F, MILIOS E Globally consistent range scan alignment for environment mapping[J]. Autonomous Robots, 1997, 4 (4): 333- 349 doi: 10.1023/A:1008854305733

[17]	BORRMANN D, ELSEBERG J, LINGEMANN K, et al Globally consistent 3D mapping with scan matching[J]. Robotics and Autonomous Systems, 2008, 56 (2): 130- 142 doi: 10.1016/j.robot.2007.07.002

[18]	舒程珣, 何云涛, 孙庆科基于卷积神经网络的点云配准方法[J]. 激光与光电子学进展, 2017, (3): 123- 131 SHU Cheng-xun, HE Yun-tao, SUN Qing-ke Point cloud registration based on convolutional neural network[J]. Laser and Opto Electronics Progress, 2017, (3): 123- 131

[19]	林桂潮, 唐昀超, 邹湘军, 等融合高斯混合模型和点到面距离的点云配准[J]. 计算机辅助设计与图形学学报, 2018, (4): 642- 650 LIN Gui-chao, TANG Yun-chao, ZOU Xiang-jun, et al Point cloud registration algorithm combined gaussian mixture model and point-to-plane metric[J]. Journal of Computer-Aided Design and Computer Graphics, 2018, (4): 642- 650

[20]	LOWE D G Distinctive image features from scale invariant key points[J]. International Journal of Computer Vision, 2004, 60 (2): 91- 110 doi: 10.1023/B:VISI.0000029664.99615.94

[21]	KAEHLER A, BRADSKI G. Learning OpenCV 3: Computer Vision in C++ with the OpenCV Library [M]. Boston: O’ Reilly Media, Inc. 2016: 450-505.

[22]	HORN B K P Closed-form solution of absolute orientation using unit quaternion[J]. Journal of the Optical Society of America A, 1987, 4 (4): 629- 642 doi: 10.1364/JOSAA.4.000629

[23]	李兴东, 郭伟, 李满天, 等一种估计深度相机位姿精度的闭式算法[J]. 机器人, 2014, 36 (2): 194- 202 LI Xing-dong, GUO Wei, LI Man-tian, et al A closed-form solution for estimating the accuracy of depth camera’s relative pose[J]. Robot, 2014, 36 (2): 194- 202

[1]	Yan-wei ZHAO,Jian ZHANG,Xian-ming ZHOU,Geng-yu WU. Dynamic tracking and precise landing of UAV based on visual magnetic guidance[J]. Journal of ZheJiang University (Engineering Science), 2021, 55(1): 96-108.

[2]	Da-zhao DONG,Guan-hua XU,Ji-liang GAO,Yue-tong XU,Jian-zhong FU. Online correction algorithm for posture by robot assembly based on machine vision[J]. Journal of ZheJiang University (Engineering Science), 2021, 55(1): 145-152.

[3]	Wen-jin HUANG,Miao-hua HUANG. Double-layer fusion of lidar and roadside camera for cooperative localization[J]. Journal of ZheJiang University (Engineering Science), 2020, 54(7): 1369-1379.

[4]	Hua HUANG,Wen-qiang DENG,Yuan LI,Run-lan GUO. Mass matching design of machine tool parts based on spatial dynamics optimization[J]. Journal of ZheJiang University (Engineering Science), 2020, 54(10): 2009-2017.

[5]	Wen-tao WANG,Jia-tian LI,Hua-jing WU,Peng GAO,Xiao-hui A,Zhi-hao ZHU. Camera active pose cooperation for obtaining face frontal images[J]. Journal of ZheJiang University (Engineering Science), 2020, 54(10): 1936-1944.

[6]	ZHANG Zhen-jie, LI Jian-sheng, ZHAO Man-dan, ZHANG Xiao-dong. Camera pose estimation based on three-view geometry constraint[J]. Journal of ZheJiang University (Engineering Science), 2018, 52(1): 151-159.

[7]	ZHOU Yong gang, LI Pei, AO Xiang, ZHAO Hong. High temperature corrosion inhibition for opposed firing boiler based on combustion distribution evenness[J]. Journal of ZheJiang University (Engineering Science), 2015, 49(9): 1768-1775.

[8]	JIANG Shen-yu, CHEN Kuo, XU Zhi-hai, FENG Hua-jun, LI Qi, CHEN Yue-ting. Multi-exposure image fusion based on well-exposedness assessment[J]. Journal of ZheJiang University (Engineering Science), 2015, 49(3): 470-475.

[9]	XIA Mei-meng, GUAN Fu-ling. Improved PSO based pretension optimum of cable net in antenna[J]. Journal of ZheJiang University (Engineering Science), 2013, 47(3): 480-487.

[10]	YUAN Xing-fei, YU Jun-Wei. Wind-induced response analysis of wind turbine tower installed on building roof[J]. Journal of ZheJiang University (Engineering Science), 2013, 47(11): 1911-1916.

[11]	YANG Ke, LUO Qiong, SHI Jiao-Ying. Application of graphics processors to database technologies[J]. Journal of ZheJiang University (Engineering Science), 2009, 43(8): 1349-1360.

[12]	CHEN Shen, TU Ling. Effect of preconditioned matrix on PCGM used for identification of dynamic axle loads on bridge[J]. Journal of ZheJiang University (Engineering Science), 2009, 43(7): 1293-1296.

[13]	DIAO Gang-Jiang, CHEN Ling, CHEN Gen-Cai. Threedimensional head pose tracking using registration and multiscale appearance model[J]. Journal of ZheJiang University (Engineering Science), 2009, 43(09): 1597-1603.

Viewed

Full text

Abstract

Cited

Shared

Discussed