Rufael Mekuria
Michele Sanna
Pablo Cesar
ABSTRACT
3D Tele-immersion enables participants in remote locations to share, in real-time, an activity. It offers users natural interactivity and immersive experiences, but it challenges current networking solutions. Work in the past has mainly focused on the efficient delivery of image-based 3D videos and on the realistic rendering and reconstruction of geometry-based 3D objects. The contribution of this paper is a complete media pipeline that allows for geometry-based 3D tele-immersion. Unlike previous approaches, that stream videos or video plus depth estimate, our streaming module can transmit the live-reconstructed 3D representations (triangle meshes). Based on a set of comparative experiments, this paper details the architecture and describes a novel component that can efficiently stream geometry in real-time. This component includes both a novel fast local compression algorithm and a rateless packet protection scheme geared towards the requirements imposed by real-time transmission of live-capture mesh geometry. Tests on a large dataset show an encoding and decoding speed-up of over 10 times at similar compression and quality rates, when compared to the high-end MPEG-4 SC3DMC mesh encoder. The implemented rateless code ensures complete packet loss protection of the triangle mesh object and avoids delay introduced by retransmissions. This approach is compared to a streaming mechanism over TCP and outperforms it at packet loss rates over 2% and/or latencies over 9 ms in terms of end-to-end transmission delay. As reported in this paper, the component has been successfully integrated into a larger tele-immersive environment that includes beyond state of the art 3D reconstruction and rendering modules. This resulted in a prototype that can capture, compress transmit and render triangle mesh geometry in real-time over the internet.
- Yang Z., Yu B., Diankov R., Wu W., and Bajcsy W. Collaborative Dancing in Tele-Immersive Environment(2006). ACM International Conference on Multimedia (MM) pp. 723--726 Google Scholar
Digital Library
- Forte M., Kurillo G., Cyberarchaeology - Experimenting with Teleimmersive Archaeology (2010) 16th International Conference on Virtual Systems and Multimedia (VSMM 2010), pp 155--162.Google Scholar
- Kurillo G., Bajcsy R., Kreylos O., Rodriguez R., Tele-immersive environment for remote medical collaboration (2009). In Proceedings of Medicine Meets Virtual Reality (MMVR17), pp. 148--150.Google Scholar
- Nahrstedt K., Bajcsy R., Wymore L., Kurillo G., Mezur K., Sheppard R., Yang Z., and Wu. W. (2007), Symbiosis of Tele-Immersive Environments with Creative Choreography. ACM Workshop on Supporting Creative Acts Beyond Dissemination 2007 (in conjunction with CCC'07).Google Scholar
- Kauff P., Schreer O. An immersive 3D video-conferencing system using shared virtual team user environments (2002). In Proceedings of the 4th international conference on Collaborative virtual environments (CVE '02). ACM, pp 105--112 Google ScholarDigital Library
- Alexiadis D., Kordelas D. S., G. Apostolakis, K. C.; Agapito, J. D.; Vegas, J. M.; Izquierdo, E.; Daras, Reconstruction for 3D immersive virtual environments (2012). 3th International Workshop on Image Analysis for Multimedia Interactive Services (WIAMIS) pp.1--4Google Scholar
- Vasudevan R., Kurillo K., Lobaton E., Bernardin T., Kreylos O., Bajcsy R., Nahrstedt, K. High Quality Visualization for Geographically Distributed 3-D Tele-immersive Applications(2011). IEEE Transactions on Multimedia, Vol. 13, NO 3, June 2011 pp. 573--584 Google ScholarDigital Library
- Wu W., Arefin A., Kurillo G., Argawal P., Nahrstedt K., Bajcsy R., Color-plus-Depth Level-of-Detail in 3D Tele-Immersive Video: A psychophysical Approach (2011). In Proceedings of the 19th ACM international conference on Multimedia (MM '11). ACM, New York, NY, USA, pp. 13--22. Google ScholarDigital Library
- Towles H., Chen W., Yang R., Kum S., Fuchs H. Kelshikar., Mulligan J., Holden L., Zeleznik B., Sadagic A., Forman J. L., 3D Tele-collaboration over internet2. (2002). International workshop on tele presence 2002, Juan Les PinsGoogle Scholar
- Ott, D. E. and Mayer-Patel, K., 2004. Coordinated multi-streaming for 3D tele-immersion (2004). 12th Annual ACM International Conference on Multimedia, pp. 596--603. Google ScholarDigital Library
- Yang Z., Wu W., Nahrstedt K., Kurillo G., and Bajcsy R.. 2010. Enabling multi-party 3D tele-immersive environments with ViewCast. ACM Trans. Multimedia Comput. Commun. Appl. 6, 2, Article 7 (March 2010), 30 pages. Google ScholarCross Ref
- Huang, Z., Wu, W., Nahrstedt, K., Rivas, R. and Arefin, A., SyncCast: Synchronized dissemination in multi-site interactive 3D tele-immersion (2011). 2nd Annual ACM Conference on Multimedia Systems (MMSys '11)., pp. 69--80. Google ScholarDigital Library
- Peng J., Kim C. S., C-C jay Kuo. Technologies for 3D Mesh Compression: A survey. Elsevier journal of visual communication and image representation(2005) pp. 688--733 Google ScholarDigital Library
- Isenburg M., Lindstrom P., 2005. Streaming meshes (2005). IEEE Visualization (VIS 05) pp 231--238Google Scholar
- Mamou, K., Zaharia, T. and Prêteux, F. TFAN: A low complexity 3D mesh compression algorithm (2009),. Comp. Anim. Virtual Worlds, 20: 343--354. Google ScholarDigital Library
- Jang E.s., Lee S., Koo B., Kim D., Son K. Fast 3D Mesh Compression using Shared Vertex Analysis (2010) ETRI Journal, Volume 32, Number 1, February 2010Google Scholar
- AlRegib G. and Altunbasak Y., 3TP: An application-layer protocol for streaming 3-D graphics, (2005) IEEE Trans. on Multimedia, Vol. 7, No. 6, pp. 1149--1156 Google ScholarDigital Library
- Li H., Li M., and Prabhakaran B.. Middleware for streaming 3D progressive meshes over lossy networks (2006.). ACM Trans. Multimedia Comput. Commun. Appl. 2, 4 (November 2006), 282--317. Google ScholarDigital Library
- Wei Cheng, Wei Tsang Ooi, Sebastien Mondet, Romulus Grigoras, and Géraldine Morin. 2011. Modeling progressive mesh streaming: Does data dependency matter?.(2011) ACM Trans. Multimedia Comput. Commun. Appl. 7, 2, Article 10 (March 2011), 24 pages. Google ScholarDigital Library
- John W. Byers, Michael Luby, Michael Mitzenmacher, and Ashutosh Rege. A digital fountain approach to reliable distribution of bulk data(1998). SIGCOMM Comput. Commun. Rev. 28, 4 pp. 56--67. Google ScholarDigital Library
- Smolic, A., 3D Video and Free-viewpoint Video-From Capture to Display (2011). Elsevier Pattern Recognition Volume 4 Issue 9 pp. 1958--1968 Google ScholarDigital Library
- Touma C., Gotsman T., Triangle mesh compression, (1998) Proceedings of Graphics Interface, 1998,pp. 26--34.Google Scholar
- Gumhold S., Straßer W., Real time compression of triangle mesh connectivity(1998), in: ACM SIGGRAPH'98 pp. 133--140. Google ScholarDigital Library
- Pajarola R., Rossignac J., Compressed progressive meshes, IEEE Trans. Vis. Comput. Graph. 6 (1) (2000) pp. 79--93. Google ScholarDigital Library
- Taubin G., Rossignac J., Geometric compression through topological surgery, ACM Trans. Graph.17 (2) (1998) pp.84--115. Google ScholarDigital Library
- Khodakovsky A., Schroeder P., Sweldens W. Progressive geometry compression(2000). In Proceedings of the 27th annual conference on Computer graphics and interactive techniques SIGGRAPH '00. pp. 271--278. Google ScholarDigital Library
- Aspert, N. Santa-Cruz D., Ebrahimi T., MESH: Measuring Error between Surfaces using the Hausdorff distance (2002), in Proceedings of the IEEE International Conference on Multimedia and Expo 2002 ICME pp. 705--708Google Scholar
- M. Luby (2002). "LT Codes". Proceedings of the IEEE Symposium on the Foundations of Computer Science: pp. 271--280. Google ScholarDigital Library
- M. Luby, A. Shokrollahi M. Watson, T. Stockhammer, RFC 5053 Raptor Forward Error Correction Scheme for Object Delivery (2007)Google Scholar
- M. Luby A. Shokrallahi, M. Watson, T. Stockhammer, L. Minder RFC 6330 RaptorQ Raptor Forward Error Correction Scheme for Object Delivery(2011)Google Scholar
- Li, S.-Y. R.; Yeung, R. W.; Ning Cai;, "Linear network coding,"(2003) Information Theory, IEEE Transactions on, vol.49, no.2, pp.371--381, Feb. 2003 Google ScholarDigital Library
- Chou P. A., Wu Y., and Jain K.. Practical network coding. In Allerton Conference in Communication, Control and Computing, Monticello, IL, Oct. 2003.Google Scholar
Index Terms
- A 3D tele-immersion system based on live captured mesh geometry
Recommendations
Geometry-guided progressive lossless 3D mesh coding with octree (OT) decomposition
A new progressive lossless 3D triangular mesh encoder is proposed in this work, which can encode any 3D triangular mesh with an arbitrary topological structure. Given a mesh, the quantized 3D vertices are first partitioned into an octree (OT) structure, ...
Tetrahedral mesh compression with the cut-border machine
VIS '99: Proceedings of the conference on Visualization '99: celebrating ten yearsIn recent years, substantial progress has been achieved in the area of volume visualization on irregular grids, which is mainly based on tetrahedral meshes. Even moderately fine tetrahedral meshes consume several mega-bytes of storage. For archivation ...
Spectral compression of mesh geometry
SIGGRAPH '00: Proceedings of the 27th annual conference on Computer graphics and interactive techniquesWe show how spectral methods may be applied to 3D mesh data to obtain compact representations. This is achieved by projecting the mesh geometry onto an orthonormal basis derived from the mesh topology. To reduce complexity, the mesh is partitioned into ...
Comments