Abstract
The increasing adoption of remote working practices and internet-based systems highlights the need for a new mode of interaction and communication interfaces between humans and machines. Current human-machine interfaces (HMI) are based on complex controllers that are not intuitive for a novice or remote operators. The increase in skills required in today's manufacturing environment allows the use of telexistence capabilities to facilitate human-robot collaboration. We propose a digital-twin based framework with a telexistence interface as a means to reduce the complexity of operating and programming an industrial robot. To remotely control the industrial robot, sensors data are shared via a data distribution service (DDS). The immersive virtual reality (VR) interface is deployed for effective control and monitoring of the industrial robot. Telexistence capabilities allow intuitive manipulation and are combined with real-time data visualization of the robot through the digital twin. The interface is implemented with the Unity 3D engine and connected to a console application that collects sensor data and shares them via a DDS connectivity framework. A simple experiment with a physical FANUC M20-IA industrial robot and Azure Kinect RGBD cameras shows the reliable performance when a robot path request was sent by a remote operator through DDS and the immersive VR interface. We then discuss future work and use cases for this telexistence platform to support maintenance activities in manufacturing contexts.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Notes
- 1.
Jogging the industrial robot is the term used to describe the act of manually moving the robot via the user interface.
- 2.
Lower-level language similar to Pascal and used to write FANUC robots programs.
- 3.
Offline motion and command simulation for FANUC branded robots.
- 4.
References
Rosen, R., von Wichert, G., Lo, G., Bettenhausen, K.D.: About the importance of autonomy and digital twins for the future of manufacturing. IFAC-PapersOnLine 48, 567–572 (2015). https://doi.org/10.1016/j.ifacol.2015.06.141
Grieves, M., Vickers, J.: Digital twin: mitigating unpredictable, undesirable emergent behavior in complex systems. In: Kahlen, F.-J., Flumerfelt, S., Alves, A. (eds.) Transdisciplinary Perspectives on Complex Systems, pp. 85–113. Springer, Cham (2017). https://doi.org/10.1007/978-3-319-38756-7_4
El Saddik, A.: Digital twins: the convergence of multimedia technologies. IEEE Multimed. 25(2), 87–92 (2018). https://doi.org/10.1109/MMUL.2018.023121167
Fung, W.K., Lo, W.T., Liu, Y.H., Xi, N.: A case study of 3D stereoscopic vs. 2D monoscopic tele-reality in real-time dexterous teleoperation. In: 2005 IEEE/RSJ International Conference on Intelligent Robots and Systems, IROS, pp. 181–186 (2005). https://doi.org/10.1109/IROS.2005.1545299
Linn, C., Bender, S., Prosser, J., Schmitt, K., Werth, D.: Virtual remote inspection - a new concept for virtual reality enhanced real-time maintenance. In: Proceedings of the 2017 23rd International Conference on Virtual Systems and Multimedia, VSMM, pp. 1–6 (2018). https://doi.org/10.1109/VSMM.2017.8346304
Masoni, R., et al.: Supporting remote maintenance in industry 4.0 through augmented reality. Procedia Manuf. 11, 1296–1302 (2017). https://doi.org/10.1016/J.PROMFG.2017.07.257
Burdea, G.C.: Invited review: the synergy between virtual reality and robotics. IEEE Trans. Robot. Autom. 15, 400–410 (1999). https://doi.org/10.1109/70.768174
Stotko, P., et al.: A VR system for immersive teleoperation and live exploration with a mobile robot. In: IEEE International Conference on Intelligent Robots and Systems, pp. 3630–3637 (2019).https://doi.org/10.1109/IROS40897.2019.8968598
Chen, J., Glover, M., Li, C., Yang, C.: Development of a user experience enhanced teleoperation approach. In: ICARM 2016 International Conference on Advanced Robotics and Mechatronics, pp. 171–177 (2016). https://doi.org/10.1109/ICARM.2016.7606914
Tachi, S.: Telexistence. In: Brunnett, G., Coquillart, S., van Liere, R., Welch, G., Váša, L. (eds.) Virtual Realities. LNCS, vol. 8844, pp. 229–259. Springer, Cham (2015). https://doi.org/10.1007/978-3-319-17043-5_13
Talaminos-Barroso, A., Estudillo-Valderrama, M.A., Roa, L.M., Reina-Tosina, J., Ortega-Ruiz, F.: A machine-to-machine protocol benchmark for eHealth applications – use case: respiratory rehabilitation. Comput. Methods Programs Biomed. 129, 1–11 (2016). https://doi.org/10.1016/J.CMPB.2016.03.004
Connext Product Suite for Intelligent Distributed Systems. https://www.rti.com/products
Fanuc - ROS Wiki. http://wiki.ros.org/fanuc
Garg, G., Kuts, V., Anbarjafari, G.: Digital twin for Fanuc robots: industrial robot programming and simulation using virtual reality. Sustainability 13, 10336 (2021). https://doi.org/10.3390/SU131810336
Acknowledgment
This research is funded by the joint Anglo-French PhD scheme, organized by the respective nations’ Ministry of Defence, with Dstl acting as the Technical Partner.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2023 IFIP International Federation for Information Processing
About this paper
Cite this paper
Mazeas, D., Erkoyuncu, J.A., Noël, F. (2023). A Telexistence Interface for Remote Control of a Physical Industrial Robot via Data Distribution Service. In: Noël, F., Nyffenegger, F., Rivest, L., Bouras, A. (eds) Product Lifecycle Management. PLM in Transition Times: The Place of Humans and Transformative Technologies. PLM 2022. IFIP Advances in Information and Communication Technology, vol 667. Springer, Cham. https://doi.org/10.1007/978-3-031-25182-5_38
Download citation
DOI: https://doi.org/10.1007/978-3-031-25182-5_38
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-25181-8
Online ISBN: 978-3-031-25182-5
eBook Packages: Computer ScienceComputer Science (R0)
