Bojian Wu
Hui Huang
Skip Abstract Section
Skip Supplemental Material SectionAbstract
Numerous techniques have been proposed for reconstructing 3D models for opaque objects in past decades. However, none of them can be directly applied to transparent objects. This paper presents a fully automatic approach for reconstructing complete 3D shapes of transparent objects. Through positioning an object on a turntable, its silhouettes and light refraction paths under different viewing directions are captured. Then, starting from an initial rough model generated from space carving, our algorithm progressively optimizes the model under three constraints: surface and refraction normal consistency, surface projection and silhouette consistency, and surface smoothness. Experimental results on both synthetic and real objects demonstrate that our method can successfully recover the complex shapes of transparent objects and faithfully reproduce their light refraction properties.
Supplemental Material
- Kfir Aberman, Oren Katzir, Qiang Zhou, Zegang Luo, Andrei Sharf, Chen Greif, Baoquan Chen, and Daniel Cohen-Or. 2017. Dip Transform for 3D Shape Reconstruction. ACM Trans. on Graphics (Proc. of SIGGRAPH) 36, 4 (2017), 79:1--79:11. Google Scholar
Digital Library
- Bradley Atcheson, Ivo Ihrke, Wolfgang Heidrich, Art Tevs, Derek Bradley, Marcus Magnor, and Hans-Peter Seidel. 2008. Time-resolved 3D Capture of Non-stationary Gas Flows. ACM Trans. on Graphics (Proc. of SIGGRAPH Asia) 27, 5 (2008), 132:1--132:9. Google ScholarDigital Library
- Moshe Ben-Ezra and Shree K. Nayar. 2003. What Does Motion Reveal About Transparency? Proc. Int. Conf. on Computer Vision (2003), 1025--1032. Google ScholarDigital Library
- Matthew Berger, Joshua A. Levine, Luis Gustavo Nonato, Gabriel Taubin, and Claudio T. Silva. 2013. A Benchmark for Surface Reconstruction. ACM Trans. on Graphics 32, 2 (2013), 20:1--20:17. Google ScholarDigital Library
- Matthew Berger, Andrea Tagliasacchi, Lee M. Seversky, Pierre Alliez, Joshua A. Levine, Andrei Sharf, and Claudio Silva. 2014. State of the Art in Surface Reconstruction from Point Clouds. Eurographics STAR (2014), 165--185.Google Scholar
- Max Born and Emil Wolf. 2013. Principles of optics: electromagnetic theory of propagation, interference and diffraction of light. Elsevier.Google Scholar
- Tongbo Chen, Hendrik PA Lensch, Christian Fuchs, and Hans-Peter Seidel. 2007. Polarization and Phase-Shifting for 3D Scanning of Translucent Objects. Proc. IEEE Conf. on Computer Vision & Pattern Recognition (2007), 1--8.Google ScholarCross Ref
- Yung-Yu Chuang, Douglas E. Zongker, Joel Hindorff, Brian Curless, David H. Salesin, and Richard Szeliski. 2000. Environment Matting Extensions: Towards Higher Accuracy and Real-time Capture. ACM Trans. on Graphics (Proc. of SIGGRAPH) (2000), 121--130. Google ScholarDigital Library
- Massimiliano Corsini, Paolo Cignoni, and Roberto Scopigno. 2012. Efficient and Flexible Sampling with Blue Noise Properties of Triangular Meshes. IEEE Trans. Visualization & Computer Graphics 18, 6 (2012), 914--924. Google ScholarDigital Library
- Zhaopeng Cui, Jinwei Gu, Boxin Shi, Ping Tan, and Jan Kautz. 2017. Polarimetric Multi-View Stereo. Proc. IEEE Conf. on Computer Vision & Pattern Recognition (2017), 1558--1567.Google ScholarCross Ref
- Qi Duan, Jianfei Cai, and Jianmin Zheng. 2015. Compressive Environment Matting. The Visual Computer 31, 12 (2015), 1587--1600. Google ScholarDigital Library
- Silvano Galliani, Katrin Lasinger, and Konrad Schindler. 2015. Massively Parallel Multiview Stereopsis by Surface Normal Diffusion. Proc. Int. Conf. on Computer Vision (2015), 873--881. Google ScholarDigital Library
- James Gregson, Michael Krimerman, Matthias B. Hullin, and Wolfgang Heidrich. 2012. Stochastic Tomography and Its Applications in 3D Imaging of Mixing Fluids. ACM Trans. on Graphics (Proc. of SIGGRAPH) 31, 4 (2012), 52:1--52:10. Google ScholarDigital Library
- Hui Huang, Dan Li, Hao Zhang, Uri Ascher, and Daniel Cohen-Or. 2009. Consolidation of Unorganized Point Clouds for Surface Reconstruction. ACM Trans. on Graphics (Proc. of SIGGRAPH Asia) 28, 5 (2009), 176:1--176:7. Google ScholarDigital Library
- Hui Huang, Shihao Wu, Minglun Gong, Daniel Cohen-Or, Uri Ascher, and Hao Zhang. 2013. Edge-aware Point Set Resampling. ACM Trans. on Graphics 32, 1 (2013), 9:1--9:12. Google ScholarDigital Library
- Matthias B. Hullin, Martin Fuchs, Ivo Ihrke, Hans-Peter Seidel, and Hendrik P. A. Lensch. 2008. Fluorescent Immersion Range Scanning. ACM Trans. on Graphics (Proc. of SIGGRAPH) 27, 3 (2008), 87:1--87:10. Google ScholarDigital Library
- Cong Phuoc Huynh, Antonio Robles-Kelly, and Edwin Hancock. 2010. Shape and refractive index recovery from single-view polarisation images. Proc. IEEE Conf. on Computer Vision & Pattern Recognition (2010), 1229--1236.Google ScholarCross Ref
- Ivo Ihrke, Kiriakos N. Kutulakos, Hendrik Lensch, Marcus Magnor, and Wolfgang Heidrich. 2010. Transparent and specular object reconstruction. Computer Graphics Forum 29, 8 (2010), 2400--2426.Google ScholarCross Ref
- Ivo Ihrke and Marcus Magnor. 2004. Image-based Tomographic Reconstruction of Flames. Proc. Eurographics Symp. on Computer Animation (2004), 365--373. Google ScholarDigital Library
- Julian Iseringhausen, Bastian Goldlücke, Nina Pesheva, Stanimir Iliev, Alexander Wender, Martin Fuchs, and Matthias B. Hullin. 2017. 4D Imaging Through Spray-on Optics. ACM Trans. on Graphics (Proc. of SIGGRAPH) 36, 4 (2017), 35:1--35:11. Google ScholarDigital Library
- Yu Ji, Jinwei Ye, and Jingyi Yu. 2013. Reconstructing gas flows using light-path approximation. Proc. IEEE Conf. on Computer Vision & Pattern Recognition (2013), 2507--2514. Google ScholarDigital Library
- Michael Kazhdan and Hugues Hoppe. 2013. Screened Poisson Surface Reconstruction. ACM Trans. on Graphics 32, 3 (2013), 29:1--29:13. Google ScholarDigital Library
- Jaewon Kim, Ilya Reshetouski, and Abhijeet Ghosh. 2017. Acquiring Axially-Symmetric Transparent Objects Using Single-View Transmission Imaging. Proc. IEEE Conf. on Computer Vision & Pattern Recognition (2017), 1484--1492.Google ScholarCross Ref
- Kiriakos N. Kutulakos and Steven M. Seitz. 2000. A Theory of Shape by Space Carving. Int. J. Computer Vision 38, 3 (2000), 199--218. Google ScholarDigital Library
- Kiriakos N. Kutulakos and Eron Steger. 2008. A theory of refractive and specular 3D shape by light-path triangulation. Int. J. Computer Vision 76, 1 (2008), 13--29. Google ScholarDigital Library
- Aviad Levis, Yoav Y Schechner, Amit Aides, and Anthony B Davis. 2015. Airborne three-dimensional cloud tomography. Proc. Int. Conf. on Computer Vision (2015), 3379--3387. Google ScholarDigital Library
- Aviad Levis, Yoav Y Schechner, and Anthony B Davis. 2017. Multiple-scattering microphysics tomography. Proc. IEEE Conf. on Computer Vision & Pattern Recognition (2017), 5797--5806.Google ScholarCross Ref
- Yaron Lipman, Daniel Cohen-Or, David Levin, and Hillel Tal-Ezer. 2007. Parameterization-free Projection for Geometry Reconstruction. ACM Trans. on Graphics (Proc. of SIGGRAPH) 26, 3 (2007), 22:1--22:6. Google ScholarDigital Library
- Daisuke Miyazaki and Katsushi Ikeuchi. 2005. Inverse polarization raytracing: estimating surface shapes of transparent objects. Proc. IEEE Conf. on Computer Vision & Pattern Recognition 2 (2005), 910--917. Google ScholarDigital Library
- Nigel JW Morris and Kiriakos N. Kutulakos. 2007. Reconstructing the surface of inhomogeneous transparent scenes by scatter-trace photography. Proc. Int. Conf. on Computer Vision (2007), 1--8.Google Scholar
- Nigel JW Morris and Kiriakos N. Kutulakos. 2011. Dynamic refraction stereo. IEEE Trans. Pattern Analysis & Machine Intelligence 33, 8 (2011), 1518--1531. Google ScholarDigital Library
- Pieter Peers and Philip Dutré. 2003. Wavelet environment matting. Proc. Eurographics Workshop on Rendering (2003), 157--166. Google ScholarDigital Library
- Yiming Qian, Minglun Gong, and Yee Hong Yang. 2016. 3D Reconstruction of Transparent Objects with Position-Normal Consistency. Proc. IEEE Conf. on Computer Vision & Pattern Recognition (2016), 4369--4377.Google ScholarCross Ref
- Yiming Qian, Minglun Gong, and Yee-Hong Yang. 2015. Frequency-based environment matting by compressive sensing. Proc. Int. Conf. on Computer Vision (2015), 3532--3540. Google ScholarDigital Library
- Yiming Qian, Minglun Gong, and Yee-Hong Yang. 2017. Stereo-Based 3D Reconstruction of Dynamic Fluid Surfaces by Global Optimization. Proc. IEEE Conf. on Computer Vision & Pattern Recognition (2017), 6650--6659.Google ScholarCross Ref
- Yuliy Schwartzburg, Romain Testuz, Andrea Tagliasacchi, and Mark Pauly. 2014. High-contrast Computational Caustic Design. ACM Trans. on Graphics (Proc. of SIGGRAPH) 33, 4 (2014), 74:1--74:11. Google ScholarDigital Library
- Qi Shan, Sameer Agarwal, and Brian Curless. 2012. Refractive height fields from single and multiple images. Proc. IEEE Conf. on Computer Vision & Pattern Recognition (2012), 286--293. Google ScholarDigital Library
- Kenichiro Tanaka, Yasuhiro Mukaigawa, Hiroyuki Kubo, Yasuyuki Matsushita, and Yasushi Yagi. 2016. Recovering Transparent Shape from Time-of-Flight Distortion. Proc. IEEE Conf. on Computer Vision & Pattern Recognition (2016), 4387--4395.Google ScholarCross Ref
- Borislav Trifonov, Derek Bradley, and Wolfgang Heidrich. 2006. Tomographic reconstruction of transparent objects. Proc. Eurographics Conf. on Rendering Techniques (2006), 51--60. Google ScholarDigital Library
- Chia-Yin Tsai, Ashok Veeraraghavan, and Aswin C Sankaranarayanan. 2015. What does a single light-ray reveal about a transparent object? Proc. IEEE Int. Conf. on Image Processing (2015), 606--610.Google ScholarDigital Library
- Gordon Wetzstein, David Roodnick, Wolfgang Heidrich, and Ramesh Raskar. 2011. Refractive shape from light field distortion. Proc. Int. Conf. on Computer Vision (2011), 1180--1186. Google ScholarDigital Library
- Yonatan Wexler, Andrew. W. Fitzgibbon, and Andrew. Zisserman. 2002. Image-based Environment Matting. Proc. Eurographics Workshop on Rendering (2002), 279--290. Google ScholarDigital Library
- Shihao Wu, Hui Huang, Minglun Gong, Matthias Zwicker, and Daniel Cohen-Or. 2015a. Deep Points Consolidation. ACM Trans. on Graphics (Proc. of SIGGRAPH Asia) 34, 6 (2015), 176:1--176:13. Google ScholarDigital Library
- Shihao Wu, Wei Sun, Pinxin Long, Hui Huang, Daniel Cohen-Or, Minglun Gong, Oliver Deussen, and Baoquan Chen. 2014. Quality-driven Poisson-guided Autoscanning. ACM Trans. on Graphics (Proc. of SIGGRAPH Asia) 33, 6 (2014), 203:1--203:12. Google ScholarDigital Library
- Zhaohui Wu, Zhong Zhou, Delei Tian, and Wei Wu. 2015b. Reconstruction of Three-dimensional Flame with Color Temperature. The Visual Computer 31, 5 (2015), 613--625. Google ScholarDigital Library
- Sai-Kit Yeung, Tai-Pang Wu, Chi-Keung Tang, Tony F Chan, and Stanley Osher. 2011. Adequate reconstruction of transparent objects on a shoestring budget. Proc. IEEE Conf. on Computer Vision & Pattern Recognition (2011), 2513--2520. Google ScholarDigital Library
- Yonghao Yue, Kei Iwasaki, Bing-Yu Chen, Yoshinori Dobashi, and Tomoyuki Nishita. 2014. Poisson-Based Continuous Surface Generation for Goal-Based Caustics. ACM Trans. on Graphics 33, 3 (2014), 31:1--31:7. Google ScholarDigital Library
- Mingjie Zhang, Xing Lin, Mohit Gupta, Jinli Suo, and Qionghai Dai. 2014. Recovering Scene Geometry under Wavy Fluid via Distortion and Defocus Analysis. Proc. Euro. Conf. on Computer Vision (2014), 234--250.Google ScholarCross Ref
- Zhengyou Zhang. 2000. A Flexible New Technique for Camera Calibration. IEEE Trans. Pattern Analysis & Machine Intelligence 22, 11 (2000), 1330--1334. Google ScholarDigital Library
- Ciyou Zhu, Richard H. Byrd, Peihuang Lu, and Jorge Nocedal. 1997. Algorithm 778: L-BFGS-B: Fortran Subroutines for Large-scale Bound-constrained Optimization. ACM Trans. Mathematical Software 23, 4 (1997), 550--560. Google ScholarDigital Library
- Douglas E. Zongker, Dawn M. Werner, Brian Curless, and David H. Salesin. 1999. Environment Matting and Compositing. ACM Trans. on Graphics (Proc. of SIGGRAPH) (1999), 205--214. Google ScholarDigital Library
- Xinxin Zuo, Chao Du, Sen Wang, Jiangbin Zheng, and Ruigang Yang. 2015. Interactive visual hull refinement for specular and transparent object surface reconstruction. Proc. Int. Conf. on Computer Vision (2015), 2237--2245. Google ScholarDigital Library
Index Terms
- Full 3D reconstruction of transparent objects
Recommendations
Differentiable refraction-tracing for mesh reconstruction of transparent objects
Capturing the 3D geometry of transparent objects is a challenging task, ill-suited for general-purpose scanning and reconstruction techniques, since these cannot handle specular light transport phenomena. Existing state-of-the-art methods, designed ...
Matting and compositing of transparent and refractive objects
This article introduces a new approach for matting and compositing transparent and refractive objects in photographs. The key to our work is an image-based matting model, termed the Attenuation-Refraction Matte (ARM), that encodes plausible refractive ...
3D Reconstruction by Shadow Carving: Theory and Practical Evaluation
Cast shadows are an informative cue to the shape of objects. They are particularly valuable for discovering object's concavities which are not available from other cues such as occluding boundaries. We propose a new method for recovering shape from ...
Comments