|
|
|
| Reconstruction of three-dimensional human bodies from single image by LeNet-5 |
Hao-can XU1,2( ),Ji-tuo LI1,2,*(),Guo-dong LU1,2 |
1. School of Mechanical Engineering, Zhejiang University, Hangzhou 310027, China
2. Robotics Institute, Zhejiang University, Yuyao 315400, China |
|
|
|
Abstract A novel human body modeling method that can reconstruct three-dimensional (3D) human bodies from single dressed human body image based on LeNet-5 was proposed. The method can reconstruct 3D human bodies accurately and efficiently, and the reconstruction results can be potentially used in some occasions where require precise surface shapes, such as virtual try-on systems. 3D human bodies collected from open datasets were selected and augmented on manifolds with PGA. A dressed human body database was established after dressing these 3D human bodies with virtual garments in various types and sizes. Feature descriptors were extracted from the frontal projected images of dressed human bodies. The corresponding 3D human bodies were constructed through LeNet-5 with the constraints of shape parameters as well as the frontal and lateral contours. The experimental results show that the model can reconstruct a high-precision 3D human body from a single dressed human body image for people wearing different styles of clothing.
|
|
Received: 09 January 2020
Published: 05 January 2021
|
|
|
|
Corresponding Authors:
Ji-tuo LI
E-mail: haocan_xu@zju.edu.cn;jituo_li@zju.edu.cn
|
由LeNet-5从单张着装图像重建三维人体
提出基于LeNet-5的从单张着装图像恢复人体三维形状的方法，建立着装人体正面轮廓和人体形状空间之间的映射模型，实现了高效、精确的三维人体建模，可以应用于对人体表面形状精度要求较高的场合，如虚拟试衣. 基于PGA在流型空间上对公开的三维人体数据集进行数据扩增，给虚拟人体进行着装，构建着装人体数据库. 从着装人体正面投影图像中提取信息，以人体形状参数及正、侧面轮廓信息为约束，基于LeNet-5完成三维人体重建. 实验证明，对于身穿不同款式服装的人，采用的模型通常都能从单张着装图像中重建得到较高精度的三维人体模型.
关键词：
三维人体重建,
虚拟试衣,
数据扩增,
着装人体,
深度学习
|
|
| [1] |
ALLDIECK T, MAGNOR M, XU W, et al. Detailed human avatars from monocular video [C]// International Conference on 3D Vision. Verona: IEEE, 2018: 98-109.
|
|
|
| [2] |
TONG J, ZHOU J, LIU L, et al Scanning 3D full human bodies using kinects[J]. IEEE Transactions on Visualization and Computer Graphics, 2012, 18 (4): 643- 650
doi: 10.1109/TVCG.2012.56
|
|
|
| [3] |
CHEN G, LI J, WANG B, et al Reconstructing 3D human models with a kinect[J]. Computer Animation and Virtual Worlds, 2016, 27 (1): 72- 85
doi: 10.1002/cav.1632
|
|
|
| [4] |
CHEN G, LI J, ZENG J, et al Optimizing human model reconstruction from RGB-D image based on skin detection[J]. Virtual Reality, 2016, 20 (3): 159- 172
doi: 10.1007/s10055-016-0291-y
|
|
|
| [5] |
WEISS A, HIRSHBERG D, BLACL M J. Home 3D body scans from noisy image and range data [C]// International Conference on Computer Vision. Barcelona: IEEE, 2011: 1951-1958.
|
|
|
| [6] |
WANG C C L Parameterization and parametric design of mannequins[J]. Computer-Aided Design, 2005, 37 (1): 83- 98
doi: 10.1016/j.cad.2004.05.001
|
|
|
| [7] |
BEAK S Y, LEE K Parametric human body shape modeling framework for human-centered product design[J]. Computer-Aided Design, 2012, 44 (1): 56- 67
doi: 10.1016/j.cad.2010.12.006
|
|
|
| [8] |
HUANG J, KWOK T H, ZHOU C Parametric design for human body modeling by wireframe-assisted deep learning[J]. Computer-Aided Design, 2019, 108: 19- 29
doi: 10.1016/j.cad.2018.10.004
|
|
|
| [9] |
ANGUELOV D, SRINIVASAN P, KOLLER D, et al SCAPE: shape completion and animation of people[J]. ACM Transactions on Graphics, 2005, 24 (3): 408- 416
doi: 10.1145/1073204.1073207
|
|
|
| [10] |
LOPER M, MAHMOOD N, ROMERO J, et al SMPL: a skinned multi-person linear model[J]. ACM Transactions on Graphics, 2015, 34 (6): 248
|
|
|
| [11] |
POPA A I, ZANFIR M, SMINCHISESCU C. Deep multitask architecture for integrated 2d and 3d human sensing [C]//Conference on Computer Vision and Pattern Recognition. Hawaii: IEEE, 2017: 6289-6298.
|
|
|
| [12] |
PAVLAKOS G, ZHU L, ZHOU X, et al. Learning to estimate 3D human pose and shape from a single color image [C]// Conference on Computer Vision and Pattern Recognition. Salt Lake City: IEEE, 2018: 459-468.
|
|
|
| [13] |
JI Z, QI X, WANG Y, et al Human body shape reconstruction from binary silhouette images[J]. Computer Aided Geometric Design, 2019, 71: 231- 243
doi: 10.1016/j.cagd.2019.04.019
|
|
|
| [14] |
KANAZAWA A, BLACK M J, JACOBS D W, et al. End-to-end recovery of human shape and pose [C]// Conference on Computer Vision and Pattern Recognition. Salt Lake City: IEEE, 2018: 7122-7131.
|
|
|
| [15] |
GUAN P, WEISS A, BALAN A O, et al. Estimating human shape and pose from a single image [C]// International Conference on Computer Vision. Florida: IEEE, 2009: 1381-1388.
|
|
|
| [16] |
OMRAN M, LASSNER C, PONS-MOLL G, et al. Neural body fitting: unifying deep learning and model based human pose and shape estimation [C]// International Conference on 3D Vision. Verona: IEEE, 2018: 484-494.
|
|
|
| [17] |
JOHNSON S, EVERINGHAM M. Clustered pose and nonlinear appearance models for human pose estimation [C]// British Machine Vision Conference. Aberystwyth: BMVA, 2010: 5.
|
|
|
| [18] |
IONESCU C, PAPAVA D, OLARU V, et al Human3.6m: large scale datasets and predictive methods for 3d human sensing in natural environments[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2013, 36 (7): 1325- 1339
|
|
|
| [19] |
LASSNER C, ROMERO J, KIEFEL M, et al. Unite the people: closing the loop between 3d and 2d human representations [C]// Conference on Computer Vision and Pattern Recognition. Hawaii: IEEE, 2017: 6050-6059.
|
|
|
| [20] |
PISHCHULIN L, WUHRER S, HELTEN T, et al Building statistical shape space for 3d human modeling[J]. Patten Recognition, 2017, 67: 276- 286
doi: 10.1016/j.patcog.2017.02.018
|
|
|
| [21] |
LI J, LU G Customizing 3D garments based on volumetric deformation[J]. Computers in Industry, 2011, 62 (7): 693- 707
doi: 10.1016/j.compind.2011.04.002
|
|
|
| [22] |
FREIFELD O, BLACK M J. Lie bodies: a manifold representation of 3D human shape [C]// European Conference on Computer Vision. Berlin: Springer, 2012: 1-14.
|
|
|
| [23] |
FLETCHER P T, LU C, JOSHI S. Statistics of shape via principal geodesic analysis on lie groups [C]// IEEE Computer Society Conference on Computer Vision and Pattern Recognition. Wisconsin: IEEE, 2003: 95-101.
|
|
|
| [24] |
LECUN Y, BOTTOU L, BENGIO Y, et al Gradient-based learning applied to document recognition[J]. Proceedings of the IEEE, 1998, 86 (11): 2278- 2324
doi: 10.1109/5.726791
|
|
|
| [25] |
HINTON G E, SRIVASTAVA N, KRIZHEVSKY A, et al. Improving neural networks by preventing co-adaptation of feature detectors [J]. Computer Science, 2012, 3(4): 212-223.
|
|
|
|
Viewed |
|
|
|
Full text
|
|
|
|
|
Abstract
|
|
|
|
|
Cited |
|
|
|
|
| |
Shared |
|
|
|
|
| |
Discussed |
|
|
|
|
