Please wait a minute...
浙江大学学报(工学版)
Journal of ZheJiang University (Engineering Science)  2021, Vol. 55 Issue (4): 658-664    DOI: 10.3785/j.issn.1008-973X.2021.04.007
    
Three-dimensional reconstruction algorithm based on fusion of transport of intensity equation and neural network
Hong CHENG(),Jia-jie HU,Yong LIU,Yuan-qing YE
School of Electronics and Information Engineering, Anhui University, Hefei 230601, China
Download: HTML     PDF(1216KB) HTML
Export: BibTeX | EndNote (RIS)      

Abstract  

A three-dimensional reconstruction algorithm based on the fusion of transport of intensity equation and neural network was proposed in order to improve the accuracy of the phase retrieved by the original transport of intensity equation method. The initial phases of different angles of the object were solved by transport of intensity equation and optimized by the neural network algorithm. Then the three-dimensional information was reconstructed according to the final retrieval phases with different angles and the multiplicative technique. The algorithm has the characteristics of high precision, and can provide reference for the application of three-dimensional imaging technology. The phase error obtained by the transport of intensity equation was reduced from 21.40% to 5.26% for the example image in the experiment. The correlation degree between the reconstructed three-dimensional object and the simulated object was significant.

Key words： three-dimensional reconstruction      transport of intensity equation      artificial neural network      multiplicative technique      phase retrieval     
Received: 04 July 2020      Published: 07 May 2021
CLC:  O 436  
Fund:  国家自然科学基金资助项目（61605002）；安徽省高等学校自然科学研究资助项目（KJ2020ZD02，KJ2019ZD04）；安徽省自然科学基金资助项目（2008085MF209）
Service
E-mail this article
Add to my bookshelf
Add to citation manager
E-mail Alert
RSS
Articles by authors
Hong CHENG
Jia-jie HU
Yong LIU
Yuan-qing YE
Cite this article:

Hong CHENG,Jia-jie HU,Yong LIU,Yuan-qing YE. Three-dimensional reconstruction algorithm based on fusion of transport of intensity equation and neural network. Journal of ZheJiang University (Engineering Science), 2021, 55(4): 658-664.

URL:

http://www.zjujournals.com/eng/10.3785/j.issn.1008-973X.2021.04.007     OR     http://www.zjujournals.com/eng/Y2021/V55/I4/658


强度传输方程和神经网络融合的三维重构算法

针对原有强度传输方程法所恢复的相位精度不够精确的缺点，提出强度传输方程和神经网络融合的三维重构算法. 利用强度传输方程求解出物体不同角度的初始相位，利用神经网络算法进行优化，根据不同角度的最终恢复相位结合乘法技术重构出三维体信息. 该算法具有精度高的特点，可以为三维成像技术的应用提供参考. 对于实验中的示例图像，该算法将强度传输方程得到的相位误差从21.40%降低为5.26%，重构三维物体与模拟真实物体的相关程度为显著相关.


关键词： 三维重构,  强度传输方程,  人工神经网络,  乘法技术,  相位恢复 
Fig.1 Schematic diagram of intensity difference
Fig.2 Schematic diagram of fusion of TIE and ANN
Fig.3 Reconstruction process of MT（Take four angles for example）
Fig.4 Simulated objects and their projections with different angles
Fig.5 Initial phase results based on TIE
Fig.6 Optimized phase results based on ANN and corresponding exact phase
Fig.7 Three-dimensional display of exact phase image，initial phase image and optimized phase image
算法 ${ { {E} }_{{\rm{tr}}} }/{\text{%} }$ ${ { {E} }_{{\rm{tes}}} }/{\text{%} }$ ${ { {E} }_{{\rm{exa}}} }/{\text{%} }$ ${ {T} }/{\rm{s}}$
TIE 20.66 20.56 21.40 0.57
TIE+ANN 0.42 4.06 5.26 0.91
Tab.1 Error comparison of two different algorithms
Fig.8 Three-dimensional reconstruction based on MT
相关系数 相关程度
0~0.3 微相关
0.3~0.5 实相关
0.5~0.8 显著相关
0.8~1.0 高度相关
Tab.2 Relationship between correlation coefficient and correlation degree
[1]   SIMPSON J, LOPEZ L, ACAR P, et al Three-dimensional echocardiography in congenital heart disease: an expert consensus document from the European Association of Cardiovascular Imaging and the American Society of Echocardiography[J]. Journal of the American Society of Echocardiography, 2017, 30 (1): 1- 27
doi: 10.1016/j.echo.2016.08.022
[2]   MUSARRA G, LYONS A, CONCA E, et al Non-line-of-sight three-dimensional imaging with a single-pixel camera[J]. Physical Review Applied, 2019, 12 (1): 011002
doi: 10.1103/PhysRevApplied.12.011002
[3]   陈妮, 左超, BYOUNGHO L 基于深度测量的三维成像技术[J]. 红外与激光工程, 2019, 48 (6): 199- 223
CHEN Ni, ZUO Chao, BYOUNGHO L 3D imaging technology based on depth measurement[J]. Infrared and Laser Engineering, 2019, 48 (6): 199- 223
[4]   BORN M, WOLF E. Principles of optics: electromagnetic theory of propagation, interference and diffraction of light [M]. Heading Hill Hall, Oxford OX3 0BW, England: Pergamon Press, 2013.
[5]   GERSHUN A The light field[J]. Journal of Mathematical Physics, 1939, 18: 51151
[6]   LAM E Y Computational photography with plenoptic camera and light field capture: tutorial[J]. Journal of the Optical Society of America, 2015, 32: 2021- 2032
doi: 10.1364/JOSAA.32.002021
[7]   黄坦. 基于强度传输方程的凸优化相位恢复算法研究[D]. 武汉: 华中科技大学, 2019.
HUANG Tan. Study of convex optimization phase retriev- al algorithm based on transport of intensity equation[D]. Wuhan: Huazhong University of Science and Technology, 2019.
[8]   纵榜铭. 基于PIE相位恢复的成像光学元件检测技术研究[D]. 无锡: 江南大学, 2019.
ZONG Bang-ming. Aberration measurement on optical imaging element with ptychographic iterative engine[D]. Wuxi: Jiangnan University, 2019.
[9]   沈成. 基于多图像迭代相位恢复技术的计算成像特性研究[D]. 哈尔滨: 哈尔滨工业大学, 2018.
SHEN Cheng. Computational imaging based on iterative multi-image phase retrieval techniques[D]. Harbin: Harbin Institute of Technology, 2018.
[10]   TEAGUE M R Deterministic phase retrieval: a Green’s function solution[J]. Journal of the Optical Society of America, 1983, 73 (11): 1434- 1441
doi: 10.1364/JOSA.73.001434
[11]   程鸿, 熊帮玲, 王金成, 等 基于配准递进补偿的相位恢复[J]. 光子学报, 2019, 48 (4): 0410002
CHENG Hong, XIONG Bang-ling, WANG Jin-cheng, et al Phase retrieval based on registration progressive compensation algorithm[J]. Acta Optica Sinica, 2019, 48 (4): 0410002
[12]   左超, 陈钱, 孙佳嵩, 等 基于光强传输方程的非干涉相位恢复与定量相位显微成像: 文献综述与最新进展[J]. 中国激光, 2016, 43 (6): 227- 257
ZUO Chao, CHEN Qian, SUN Jia-song, et al Non-interferometric phase retrieval and quantitative phase microscopy based on transport of intensity equation: a review[J]. Chinese Journal of Lasers, 2016, 43 (6): 227- 257
[13]   高要利. 双波长下基于强度传输方程的相位恢复和相位解缠研究[D]. 合肥: 安徽大学, 2019.
GAO Yao-li. Research on phase retrieval and phase unwrapping based on transport of intensity equation at two wave lengths[D]. Hefei: Anhui University, 2019.
[14]   CHENG H, WANG R, YE Y Q, et al Transport of intensity equation method based on edge detection and duty ratio fusion[J]. Journal of Optics, 2020, 22 (4): 045302
doi: 10.1088/2040-8986/ab7ae9
[15]   SUI L, ZHAO X, HUANG C, et al An optical multiple-image authentication based on transport of intensity equation[J]. Optics and Lasers in Engineering, 2019, 116: 116- 124
doi: 10.1016/j.optlaseng.2019.01.006
[16]   HU J, WEI Q, KONG Y, et al Higher order transport of intensity equation methods: comparisons and their hybrid application for noise adaptive phase imaging[J]. IEEE Photonics Journal, 2019, 11 (3): 1- 14
[17]   FAYED M, ELHADARY M, ABDERRAHMANE H A, et al The ability of forecasting flapping frequency of flexible filament by artificial neural network[J]. Alexandria Engineering Journal, 2019, 58 (4): 1367- 1374
doi: 10.1016/j.aej.2019.11.007
[18]   CHENG II H, LIU Y, ZHANG II Q. Phase retrieval algorithm based on the neural network and the GS[C]// Holography, Diffractive Optics, and Applications IX. Hangzhou: ISOP, 2019: 1118809.
[19]   NEHMETALLAH G T, AYLO R, WILLIAMS L. Analog and digital holography with MATLAB [M]. Bellingham: SPIE, 2015.
[20]   左超. 基于光强传输方程的非干涉相位恢复与定量相位显微成像方法研究[D]. 南京: 南京理工大学, 2014.
ZUO Chao. Research on non-interferometric phase retrieval and quantitative phase microscopy based on transport of intensity equation[D]. Nanjing: Nanjing University of Science and Technology, 2014.
[21]   GLOROT X, BENGIO Y. Understanding the difficulty of training deep feedforward neural networks[C]// Proceedings of the 13th International Conference on Artificial Intelligence and Statistics. Sardinia, Italy: PMLR, 2010: 249-256.
[22]   RUDER S. An overview of gradient descent optimization algorithms [DB/OL]. (2017-06-15)[2021-02-19]. https://arxiv.org/abs/1609.04747.
[1] Fang LIU,Zhen WANG,Rui-di LIU,Kai WANG. Short-term forecasting method of wind power generation based on BP neural network with combined loss function[J]. Journal of ZheJiang University (Engineering Science), 2021, 55(3): 594-600.
[2] HUANG Zeng, WANG Rui, ZHAO Yu, WEI Zhen-lei. Susceptibility assessment of landslides triggered by buried fault earthquake[J]. Journal of ZheJiang University (Engineering Science), 2017, 51(11): 2136-2143.
[3] CHIANG Yen ming, ZHANG Jian quan, MING Yan. Flood forecasting by ensemble neural networks[J]. Journal of ZheJiang University (Engineering Science), 2016, 50(8): 1471-1478.
[4] LIU Kai1, ZHANG Li-min1, ZHANG Chao2. New hybrid sparse penalty mechanism of restricted Boltzmann machine[J]. Journal of ZheJiang University (Engineering Science), 2015, 49(6): 1070-1078.
[5] KE Hai-feng, YING Jing. Real-time license character recognition technology based on R-ELM[J]. Journal of ZheJiang University (Engineering Science), 2014, 48(7): 1209-1216.
[6] KE Hai-feng, YING Jing. Real-time license character recognition technology based on R-ELM[J]. Journal of ZheJiang University (Engineering Science), 2014, 48(2): 0-0.
[7] XIE Di, TONG Ruo-feng, TANG Min, FENG Yang. Distinguishable method for video fire detection[J]. Journal of ZheJiang University (Engineering Science), 2012, 46(4): 698-704.
[8] LAI Xiao-bo, ZHU Shi-qiang, FANG Chun-jie. A three-dimensional reconstruction algorithm for complex
background image and its medical applications[J]. Journal of ZheJiang University (Engineering Science), 2012, 46(11): 2061-2067.
[9] DUAN Bin, LIANG Jun, FEI Zheng-shun, YANG Min, HU Bin. Nonlinear semi-parametric modeling mothed based on GA-ANN[J]. Journal of ZheJiang University (Engineering Science), 2011, 45(6): 977-983.
[10] LIN Sha-Pu, CHEN Guo-Jiang. Intelligent hybrid forecasting technique for short-term traffic flow[J]. Journal of ZheJiang University (Engineering Science), 2010, 44(8): 1473-1478.
[11] CHEN Yu-Wen, CHEN Huo. Error analysis for ray-tracing-based visual hull reconstruction algorithms[J]. Journal of ZheJiang University (Engineering Science), 2010, 44(2): 243-247+299.