ABSTRACT
This paper addresses the challenges in creating good quality composite 3D contents for 3DTV applications and post-production visual-effects. We present a novel content-aware compositing technique that faithfully preserves the salient structures of cloned source and target content, and avoid major conflicting stereopsis cues to maintain a pleasant 3D illusion altogether. Our approach typically learns the appearance layouts of both source and target scene elements. The system extracts object's significance prior maps using classified labels and derive geometric transforms to compensate the 3D perspective mismatches between source and target images using a novel depth image-based rendering procedure. For seamless cloning, we apply a new depth-consistent interpolant technique which utilizes the classified likelihood confidences in weighting the salient or low-significant regions and re-estimating the plausible depth values of the cloned region in accordance with target 3D structure. Further, we adopt a novel content-preserving local warping scheme to reduce the apparent distortions in object shape, size and perspective. Finally, we propose a content-aware mean value cloning technique that seamlessly merges the warped cloned patches with the geometric-appearance context of new background and homogenize vague boundaries with the aid of an object salient map to remove the smudging effects. The overall process is formulated as an energy minimization problem and optimally regularized for large warps, vertical disparities, and stereo baseline changes. Plausible results are demonstrated to show the effectiveness of our approach.
- D. Hoiem, A. A. Efros, M. Hebert. Recovering Surface Layout from an Image. IJCV, 75(1), Oct. 2007. Google ScholarDigital Library
- D. Hoiem, A. A. Efros, M. Hebert. Putting Objects in Perspective. IJCV, 80(1):3–15, Oct. 2008. Google ScholarDigital Library
- S. Wang, H. Lu, F. Yang, M. Yang. Superpixel tracking. ICCV, 1323–1330, Nov. 2011. Google ScholarDigital Library
- M. Sharma, S. Chaudhury, B. Lall, M. S. Venkatesh. A flexible architecture for multi-view 3DTV based on uncalibrated cameras. JVCIR, vol. 25, Issue 4, 2014. Google ScholarDigital Library
- Z. Farbman, G. Hoffer, Y. Lipman, D. Cohen-Or, D. Lischinski. Coordinates for instant image cloning. ACM Trans. on Graphics (TOG), 28(3), Aug. 2009. Google ScholarDigital Library
- S. Luo, I. Shen, B. Chen, W. Cheng, Y. Chuang. Perspective-Aware Warping for Seamless Stereoscopic Image Cloning. ACM Trans. on Graphics (TOG), vol. 31, Issue 6, Article No. 182, Nov. 2012. Google ScholarDigital Library
- A. J. Woods, T. Docherty and R. Koch. Image distortions in stereoscopic video systems. Proc. SPIE 1915, 36, Sep. 1993, doi:10.1117/12.157041.Google Scholar
- J. Jia, J. Sun, C. Tang, H. Shum. Drag-and-Drop Pasting. ACM Trans. on Graphics (TOG), vol. 25, Issue 3, pp. 631–637, July 2006. Google ScholarDigital Library
- W. Lo, J. V. Baar, C. Knaus, M. Zwicker, M. Gross. Stereoscopic 3D copy & paste. ACM Trans. on Graphics (TOG), vol. 29, Issue 6, December 2010. Google ScholarDigital Library
- P. Pãľrez, M. Gangnet, A. Blake. Poisson image editing. ACM Trans. on Graphics (TOG), vol. 22, Issue 3, pp. 313–318, July 2003. Google ScholarDigital Library
- J. Lalonde, D. Hoeim, A. A. Efros, C. Rother, J. Winn and A. Criminisi. Photo Clip Art. ACM Trans. on Graphics, vol 26, no. 3, Aug. 2007. Google ScholarDigital Library
- Y. Jia, M. Han. Category-Independent Object-Level Saliency Detection. ICCV, pp. 1761–1768, 2013. Google ScholarDigital Library
- http://vision.middlebury.edu/stereo/Google Scholar
- International Telecommunication Union. Subjective assessment of stereoscopic television pictures, 2000.Google Scholar
Index Terms
- Content-aware seamless stereoscopic 3D compositing
Recommendations
Stereoscopic 3D copy & paste
With the increase in popularity of stereoscopic 3D imagery for film, TV, and interactive entertainment, an urgent need for editing tools to support stereo content creation has become apparent. In this paper we present an end-to-end system for object ...
Perspective-aware warping for seamless stereoscopic image cloning
This paper presents a novel technique for seamless stereoscopic image cloning, which performs both shape adjustment and color blending such that the stereoscopic composite is seamless in both the perceived depth and color appearance. The core of the ...
Stereoscopic 3D copy & paste
SIGGRAPH ASIA '10: ACM SIGGRAPH Asia 2010 papersWith the increase in popularity of stereoscopic 3D imagery for film, TV, and interactive entertainment, an urgent need for editing tools to support stereo content creation has become apparent. In this paper we present an end-to-end system for object ...
Comments