Inter frame point clouds registration algorithm for pose optimization of depth camera
Xing-dong LI1,3(),He-wei GAO1,Long SUN2,3,*()
1. College of Mechanical and Electrical Engineering, Northeast Forestry University, Harbin 150040, China 2. College of Forestry, Northeast Forestry University, Harbin 150040, China 3. Northern Forest Fire Management Key Laboratory of the State Forestry and Grassland Bureau, Northeast Forestry University, Harbin 150040, China
TOF (time of flight) camera can collect gray and depth images simultaneously to optimize the estimation of camera pose. Graph-based adjustment structure was applied to optimize the poses of the TOF camera in acquiring several frames. Registration between frames is a key operation which determines both the efficiency and effectiveness of the camera pose optimization. Scale invariant features were detected from a pair of images and matched subsequently. After the 2D feature points were extended into 3D space, two point clouds were registered in terms of relative positions between the features and the normal 3D points. Among all of the point clouds participating in the optimization of camera pose, any two point clouds were registered pair by pair using the proposed registering method. Lastly, the graph based algorithm was employed to adjust the camera poses, with inputs of the valid pairs of registered point clouds. Results demonstrated that the proposed method can improve the precision of the optimized camera pose, and the estimating efficiency is guaranteed.
Xing-dong LI,He-wei GAO,Long SUN. Inter frame point clouds registration algorithm for pose optimization of depth camera. Journal of ZheJiang University (Engineering Science), 2019, 53(9): 1749-1758.
Fig.12D image feature and 3D point cloud feature pairs
Fig.2Pairs of 3D point sets searched with 3D feature points as spherical centers
k
rk /m
$\left| {{{P}}_{{a_k}}^i} \right|$
$\left| {{{P}}_{{a_k}}^j} \right|$
1
0.56
2 439
2 551
2
2.36
1 215
1 185
3
0.47
1 284
1 263
4
0.89
2 626
2 465
Tab.1Searching radius and 3D point sets extracted from two point clouds
Fig.3Pixel set related to subset of 3D points divided by quadrant method
Fig.4Flow chart for pose optimization of depth camera
Fig.5Experimental environment for capturing original data of depth camera
Fig.6Comparison of average rotation errors for executing optimization algorithm by several times
Fig.7Comparison of average translation errors for executing optimization algorithm by several times several
Fig.8Average time for executing graph structure optimization algorithm by once
Fig.9Multiple original grayscale images for data registration experiment
Fig.10Comparison Registration for six point clouds from six frames data after pose optimization using different algorithms
[1]
LANGE R. 3D time-of-flight distance measurement with custom solid-state image sensors in CMOS/CCD technology [D]. Siegen: University of Siegen, 2000: 14–56.
[2]
王宇, 朴燕, 孙荣春 结合同场景彩色图像的深度图超分辨率重建[J]. 光学学报, 2017, 37 (8): 102- 108 WANG Yu, PIAO Yan, SUN Rong-chun Depth image super-resolution construction combined with high-resolution color image of the same scene[J]. Acta Optica Sinica, 2017, 37 (8): 102- 108
[3]
李诗锐, 李琪, 李海洋, 等 基于Kinect v2 的实时精确三维重建系统[J]. 软件学报, 2016, 27 (10): 2519- 2529 LI Shi-rui, LI Qi, LI Hai-yang, et al Real-time accurate 3D reconstruction based on Kinect v2[J]. Journal of Software, 2016, 27 (10): 2519- 2529
[4]
HENRY P, KRAININ M, HERBST E, et al RGB-D mapping: using Kinect-style depth cameras for dense 3D modeling of indoor environments[J]. International Journal of Robotics Research, 2012, 31 (5): 647- 663
doi: 10.1177/0278364911434148
[5]
FOIX S, ALENY G, ANDRADE-CETTO J, et al. Object modeling using a TOF camera under an uncertainty reduction approach [C] // Proceedings of IEEE International Conference on Robotics and Automation. Anchorage: IEEE, 2010: 1306-1312.
[6]
DELLEN B, ALENY G, FOIX S, et al. 3D object reconstruction from swissranger sensor data using a spring-mass model [C] // Proceedings of the Fourth International Conference on Computer Vision Theory and Applications. Lisboa: [s. n.], 2009: 368-372.
[7]
SARKER A, GEPPERTH A, HANDMANN U, et al. Dynamic hand gesture recognition for mobile systems using deep LSTM [C] // Proceedings of the international Conference on Intelligent Human Computer Interaction. Evry: Springer, 2017: 19-31.
[8]
GOKTURK S B, TOMASI C. 3D head tracking based on recognition and interpolation usinga time-of-flight depth sensor [C] // Proceedings of the 18th IEEE Conference on Computer Vision and Pattern Recognition. Washington: IEEE, 2004: 211-217.
[9]
LEONARD J, DURRAN-WHYTE H. Simultaneous map building and localization for an autonomous mobile robot [C] // Proceedings of the IEEE Conference on Intelligent Robots and Systems. Osaka: IEEE, 1991: 1442-1447.
[10]
BESL P J, MCKAY N D A method for registration of 3D shapes[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 1992, 14 (2): 239- 256
doi: 10.1109/34.121791
[11]
DONG J, PENG Y, YING S, et al LieTrICP: an improvement of trimmed iterative closest point algorithm[J]. Neurocomputing, 2014, 140: 67- 76
doi: 10.1016/j.neucom.2014.03.035
[12]
MAY S, DROESCHEL D, FUCHS S. Robust 3D-mapping with time of light cameras [C] // Proceedings of the IEEE Conference on Intelligent Robots and Systems. St. Louis: IEEE, 2009: 1673-1678.
[13]
SMITH R C, CHEESEMAN P On the representation and estimation of spatial uncertainty[J]. International Journal of Robotics Research, 1986, 5 (4): 56- 68
doi: 10.1177/027836498600500404
[14]
侯冰洁, 杨彦甫, 向前, 等 基于EKF和LKF级联的频偏和相位估计联合方案[J]. 光学学报, 2018, 38 (1): 85- 90 HOU Bing-jie, YANG Yan-pu, XIANG Qian, et al A joint frequency offset and phase estimation scheme based on cascaded EKF and LKF[J]. Acta Optica Sinica, 2018, 38 (1): 85- 90
[15]
MONTEMERLO M, THRUN S, KOLLER D, et al. FastSLAM: a factored solution to the simultaneous localization and mapping problem [C] // Proceedings of the National Conference on Artificial Intelligence. Alberta: AAAI Press, 2002: 593-598.
[16]
LU F, MILIOS E Globally consistent range scan alignment for environment mapping[J]. Autonomous Robots, 1997, 4 (4): 333- 349
doi: 10.1023/A:1008854305733
[17]
BORRMANN D, ELSEBERG J, LINGEMANN K, et al Globally consistent 3D mapping with scan matching[J]. Robotics and Autonomous Systems, 2008, 56 (2): 130- 142
doi: 10.1016/j.robot.2007.07.002
[18]
舒程珣, 何云涛, 孙庆科 基于卷积神经网络的点云配准方法[J]. 激光与光电子学进展, 2017, (3): 123- 131 SHU Cheng-xun, HE Yun-tao, SUN Qing-ke Point cloud registration based on convolutional neural network[J]. Laser and Opto Electronics Progress, 2017, (3): 123- 131
[19]
林桂潮, 唐昀超, 邹湘军, 等 融合高斯混合模型和点到面距离的点云配准[J]. 计算机辅助设计与图形学学报, 2018, (4): 642- 650 LIN Gui-chao, TANG Yun-chao, ZOU Xiang-jun, et al Point cloud registration algorithm combined gaussian mixture model and point-to-plane metric[J]. Journal of Computer-Aided Design and Computer Graphics, 2018, (4): 642- 650
[20]
LOWE D G Distinctive image features from scale invariant key points[J]. International Journal of Computer Vision, 2004, 60 (2): 91- 110
doi: 10.1023/B:VISI.0000029664.99615.94
[21]
KAEHLER A, BRADSKI G. Learning OpenCV 3: Computer Vision in C++ with the OpenCV Library [M]. Boston: O’ Reilly Media, Inc. 2016: 450-505.
[22]
HORN B K P Closed-form solution of absolute orientation using unit quaternion[J]. Journal of the Optical Society of America A, 1987, 4 (4): 629- 642
doi: 10.1364/JOSAA.4.000629
[23]
李兴东, 郭伟, 李满天, 等 一种估计深度相机位姿精度的闭式算法[J]. 机器人, 2014, 36 (2): 194- 202 LI Xing-dong, GUO Wei, LI Man-tian, et al A closed-form solution for estimating the accuracy of depth camera’s relative pose[J]. Robot, 2014, 36 (2): 194- 202
