| CCF CAD/CG 2023 |
|
|
|
|
| Hash encoding empowered IRON for inverse rendering: Geometry and material reconstruction |
Peiquan ZHANG,Weiwei XU( ) |
| College of Computer Science and Technology，Zhejiang University，Hangzhou 310058，China |
|
|
|
Abstract In recent years, the utilization of neural networks to represent 3D scenes for novel view synthesis has emerged as a new research focus in computer graphics, known as neural rendering. Neural networks can also be applied to efficiently represent the geometry and materials of scenes, enabling the reconstruction of high-quality meshes and texture maps under the supervision of 2D photometric images in inverse rendering, thus serving existing graphics pipelines. In this paper, we extend the latest inverse rendering by optimizing neural SDFs and materials from photometric images (IRON) neural rendering model by introducing a multiresolution hash encoding technique and employing strategies such as freezing parameters to enhance the training speed of the original model. Through comparative evaluations on multiple datasets, we achieve approximately 40% improvement in training speed compared to the original model, while producing reconstructions with more details.
|
|
Received: 12 June 2023
Published: 30 November 2023
|
|
|
|
Corresponding Authors:
Weiwei XU
E-mail: xww@cad.zju.edu.cn
|
|
Cite this article:
Peiquan ZHANG,Weiwei XU. Hash encoding empowered IRON for inverse rendering: Geometry and material reconstruction. Journal of Zhejiang University (Science Edition), 2023, 50(6): 754-760.
URL:
https://www.zjujournals.com/sci/EN/Y2023/V50/I6/754
|
哈希编码优化的IRON逆渲染模型：重建几何与材质
将神经网络用于场景几何材质的高效表达，结合逆向渲染在二维光度图的监督下重建高质量的网格和材质贴图，为现有的图形学流水线提供服务——神经渲染已成为近年来计算机图形学新的研究热点。在IRON（inverse rendering by optimizing neural SDFs and materials from photometric images）神经渲染模型基础上，通过引入多分辨率哈希编码，采用冻结训练等方法提高原始模型的训练速度。在多个数据集上的对比实验表明，优化后的IRON逆渲染模型训练速度提升了约40%，且重建结果中包含更多细节。
关键词：
符号距离场,
神经渲染,
哈希编码
|
|
| [1] |
TEWARI A， FRIED O， THIES J， et al. State of the art on neural rendering［J］. Computer Graphic Forum， 2020， 39（2）： 701-727. DOI：10.1111/cgf.14022
doi: 10.1111/cgf.14022
|
|
|
| [2] |
XIE Y， TAKIKAWA T， SAITO S， et al. Neural fields in visual computing and beyond［J］. Computer Graphic Forum， 2022， 41（2）： 641-676. DOI：10. 1111/cgf.14505
doi: 10. 1111/cgf.14505
|
|
|
| [3] |
MILDENHALL B， SRINIVASAN P P， TANCIK M， et al. NeRF： Representing scenes as neural radiance fields for view synthesis［C］// Proceedings of the 2020 European Conference on Computer Vision. Glasgow： Springer， 2020： 405-501. DOI：10.1007/978-3-030-58452-8_24
doi: 10.1007/978-3-030-58452-8_24
|
|
|
| [4] |
WANG P， LIU L， LIU Y， et al. NeuS： Learning Neural Implicit Surfaces by Volume Rendering For Multi-View Reconstruction［Z］. （2021-06-20）. https：//arxiv.org/abs/2106.10689.
|
|
|
| [5] |
ZHANG K， LUAN F， LI Z， et al. IRON： Inverse rendering by optimizing neural SDFs and materials from photometric images［C］// 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition （CVPR）， New Orleans： IEEE， 2022： 5555-5564. DOI：10.1109/CVPR52688.2022.00548
doi: 10.1109/CVPR52688.2022.00548
|
|
|
| [6] |
MÜLLER T， EVANS A， SCHIED C， et al. Instant neural graphics primitives with a multiresolution hash encoding［J］. ACM Transactions on Graphics， 2022， 41（4）： 102. DOI：10.1145/3528223.3530127
doi: 10.1145/3528223.3530127
|
|
|
| [7] |
MÜLLER T. Tiny-CUDA-NN［CP/OL］. （2021-01-07）. .
|
|
|
| [8] |
BARRON J T， MILDENHALL B， TANCIK M， et al. Mip-NeRF： A Multiscale Representation for Anti-Aliasing Neural Radiance Fields［Z］. （2021-03-24）. https：//arxiv.org/abs/2103.13415. doi:10.1109/iccv48922.2021.00580
doi: 10.1109/iccv48922.2021.00580
|
|
|
| [9] |
ZHANG K， RIEGLER G， SNAVELY N， et al. NeRF++： Analyzing and Improving Neural Radiance Fields［Z］. （2020-10-15）. https：//arxiv.org/abs/2010.07492v2.
|
|
|
| [10] |
SRINIVASAN P P， DENG B， ZHANG X， et al. NeRV： Neural reflectance and visibility fields for relighting and view synthesis［C］// 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition （CVPR）. Nashville： IEEE， 2021： 7491-7500. DOI：10.1109/CVPR46437.2021.00741
doi: 10.1109/CVPR46437.2021.00741
|
|
|
| [11] |
NIEMEYER M， MESCHEDER L， OECHSLE M， et al. Differentiable volumetric rendering： Learning implicit 3D representations without 3D supervision［C］// 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition （CVPR）. Seattle： IEEE， 2020： 3501-3512. DOI：10.1109/CVPR42600.2020.00356
doi: 10.1109/CVPR42600.2020.00356
|
|
|
| [12] |
YARIV L， KASTEN Y， MORAN D， et al. Multiview Neural Surface Reconstruction by Disentangling Geometry and Appearance［Z］. （2020-03-20）. https：//arxiv.org/abs/2003.09852.
|
|
|
| [13] |
GROPP A， YARIV L， HAIM N， et al. Implicit geometric regularization for learning shapes［C］// Proceedings of the 37th International Conference on Machine Learning. Vienna： ICML， 2020： 3569-3573.
|
|
|
| [14] |
MUNKBERG J， HASSELGREN J， SHEN T， et al. Extracting triangular 3D models， materials， and lighting from images［C］// Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition （CVPR）. New Orleans： IEEE， 2022： 8270-8280. DOI：10.1109/CVPR52688.2022.00810
doi: 10.1109/CVPR52688.2022.00810
|
|
|
| [15] |
SHEN T， GAO J， YIN K， et al. Deep Marching Tetrahedra： A Hybrid Representation for High-Resolution 3D Shape Synthesis［Z］. （2021-11-08）. https：//arxiv.org/abs/2111.04276v1.
|
|
|
| [16] |
MÜLLER T， ROUSSELLE F， NOVÁ K J， et al. Real-time neural radiance caching for path tracing［J］. ACM Transactions on Graphics， 2021， 40（4）： 1-16. DOI：10.1145/3450626.3459812
doi: 10.1145/3450626.3459812
|
|
|
| [17] |
LI Z， MÜLLER T， EVANS A， et al. Neuralangelo： High-fidelity neural surface reconstruction［C］// 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition （CVPR）. Vancouver： IEEE， 2023： 8456-8465. DOI：10.1109/CVPR52729.2023. 00817
doi: 10.1109/CVPR52729.2023. 00817
|
|
|
| [18] |
MO S， CHO M， SHIN J. Freeze the Discriminator： A Simple Baseline for Fine-Tuning GANs［Z］. （2020-02-25）. https：//arxiv.org/abs/2002.10964v1.
|
|
|
|
Viewed |
|
|
|
Full text
|
|
|
|
|
Abstract
|
|
|
|
|
Cited |
|
|
|
|
| |
Shared |
|
|
|
|
| |
Discussed |
|
|
|
|
