Abstract
Mass personalization—a megatrend in industrial manufacturing and production—requires fast adaptations of robotics and automation solutions to continually decreasing lot sizes. In this paper, the challenges of applying robot-based automation in a highly individualized production are highlighted. To face these challenges, a framework is proposed that combines latest machine learning (ML) techniques, like deep learning, with high-end physics simulation environments. ML is used for programming and parameterizing machines for a given production task with minimal human intervention. If the simulation environment realistically captures physical properties like forces or elasticity of the real world, it provides a high-quality data source for ML. In doing so, new tasks are mastered in simulation faster than in real-time, while at the same time existing tasks are executed. The functionality of the simulation-driven ML framework is demonstrated on an industrial use case.
Zusammenfassung
Die Massenpersonalisierung – ein Megatrend in der industriellen Fertigung und Produktion – erfordert eine schnelle Anpassung von Robotik- und Automatisierungslösungen an immer kleinere Losgrößen. In dieser Arbeit werden die Herausforderungen der roboterbasierten Automatisierung in einer hochgradig individualisierten Produktion aufgezeigt. Um diesen Herausforderungen zu begegnen, wird ein Framework vorgeschlagen, welches neueste Methoden des maschinellen Lernens (ML), wie beispielsweise Deep Learning, mit High-End-Physiksimulationen kombiniert. ML dient zur Programmierung und Parametrierung von Maschinen für eine bestimmte Produktionsaufgabe mit minimalem menschlichen Eingriff. Wenn die Simulationsumgebung physikalische Eigenschaften wie Kräfte oder Elastizität der realen Welt realitätsnah erfasst, stellt sie eine hochwertige Datenquelle für ML dar. Dabei werden neue Aufgaben in der Simulation schneller als in Echtzeit gemeistert, während gleichzeitig bestehende Aufgaben ausgeführt werden. Die Funktionalität des simulationsgesteuerten ML-Frameworks wird anhand eines industriellen Anwendungsfalls demonstriert.
Award Identifier / Grant number: 017-192996
Award Identifier / Grant number: Cyberprotect
Funding source: Baden-Württemberg Stiftung
Award Identifier / Grant number: NEU016/1
Funding statement: This work was partially supported by the Ministry of Economic Affairs of the state Baden-Württemberg (Zentrum für Cyber Cognitive Intelligence – Grant No. 017-192996 and Cyberprotect) and the Baden-Württemberg Stiftung (Deep Grasping – Grant No. NEU016/1).
About the authors
Mohamed El-Shamouty received his B. Sc. and M. Sc. in Communication Engineering and Media Technology from German University in Cairo, in 2015 and University of Stuttgart, in 2018 respectively. Since 2018, he is a research associate in the robotics department at the Fraunhofer Institute for Manufacturing Engineering and Automation IPA with the focus of using Machine Learning for safe and intelligent collaborative robots in production.
Kilian Kleeberger received his B. Sc. and M. Sc. degree in production engineering from the Friedrich-Alexander-Universität Erlangen-Nürnberg. During his studies, he focused on automation engineering, image processing and robotics. Since 2017, he is a research associate in the robotics department at the Fraunhofer Institute for Manufacturing Engineering and Automation IPA. His research focuses on computer vision and robotics.
Arik Lämmle received his B. Sc. and M. Sc. degree in mechanical engineering from the University of Stuttgart. During his studies, he focused on industrial robotics, production processes and machine tools. Since 2017, he is a research associate in the robotics department at the Fraunhofer Institute for Manufacturing Engineering and Automation IPA. His main focus is the development of intelligent systems for autonomous execution of assembly operations using a combination of advanced simulation environments, artificial intelligence and industrial robots.
Marco Huber received his diploma and Ph. D. in computer science from the Karlsruhe Institute of Technology (KIT), Germany, in 2006 and 2009, respectively. From 2009 until 2011 he was with Fraunhofer IOSB, Karlsruhe, Germany, where he was leading a research group on computer vision and information fusion. He then worked as Senior Researcher with AGT International in Darmstadt until 2015. From April 2015 to September 2018, Marco Huber was responsible for product development and data science services of the Katana division at USU Software AG in Karlsruhe. Since October 2018, he is full professor for cognitive production systems with University of Stuttgart and also head of the Center for Cyber Cognitive Intelligence (CCI) at the Fraunhofer Institute for Manufacturing Engineering and Automation IPA. His research focuses on machine learning, sensor data analysis, and robotics for production.
Acknowledgment
The authors would like to also acknowledge Ramez Awad, Xinyang Wu, Dennis Lamaj, Jiacheng Yang and Karin Röhricht for fruitful discussions and support with implementations.
References
1. S. J. Hu, “Evolving paradigms of manufacturing: From mass production to mass customization and personalization”, in CMS, Elsevier, 2013.Search in Google Scholar
2. G. Thomas, M. Chien, A. Tamar, J. Ojea, P. Aparicio, and P. Abbeel, “Learning Robotic Assembly from CAD”, in IEEE International Conference on Robotics and Automation (ICRA), 2018.10.1109/ICRA.2018.8460696Search in Google Scholar
3. W. Zou, M. Andulkar, and U. Berger, “Development of Robot Programming System through the use of Augmented Reality for Assembly Tasks”, in 50th International Symposium on Robotics, VDE, 2018.Search in Google Scholar
4. L. Wang, B. Schmidt, M. Givehchi, and G. Adamson, “Robotic assembly planning and control with enhanced adaptability through function blocks”, The International Journal of Advanced Manufacturing Technology, vol. 77, 2015.10.1007/s00170-014-6468-1Search in Google Scholar
5. A. Billard, S. Calinon, R. Dillmann, and S. Schaal, Robot Programming by Demonstration, B. Siciliano and O. Khatib (eds), Springer Berlin Heidelberg, 2008.10.1007/978-3-540-30301-5_60Search in Google Scholar
6. OpenAI, M. Andrychowicz, B. Baker, M. Chociej, R. Jozefowicz, B. McGrew, J. Pachocki, A. Petron, M. Plappert, G. Powell, A. Ray, J. Schneider, S. Sidor, J. Tobin, P. Welinder, L. Weng and W. Zaremba, “Learning dexterous in-hand manipulation”, 2018.10.1177/0278364919887447Search in Google Scholar
7. R. Selizki, Computer Vision: Algorithms and Applications, Springer, 2010.Search in Google Scholar
8. F. Vicentini, M. Giussani, and L. M. Tosatti, “Trajectory-dependent safe distances in human-robot interaction”, in IEEE Emerging Technology and Factory Automation (ETFA), 2014.10.1109/ETFA.2014.7005316Search in Google Scholar
9. S. James, P. Wohlhart, M. Kalakrishnan, D. Kalashnikov, A. Irpan, J. Ibarz, S. Levine, R. Hadsell, and K. Bousmalis, “Sim-to-real via sim-to-sim: Data-efficient robotic grasping via randomized-to-canonical adaptation networks”, in IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2019.10.1109/CVPR.2019.01291Search in Google Scholar
10. N. Fazeli, M. Oller, J. Wu, Z. Wu, J. B. Tenenbaum, and A. Rodriguez, “See, feel, act: Hierarchical learning for complex manipulation skills with multisensory fusion”, Science Robotics, vol. 4, no. 26, 2019.10.1126/scirobotics.aav3123Search in Google Scholar PubMed
11. L. Johannsmeier, M. Gerchow, and S. Haddadin, “A framework for robot manipulation: Skill formalism, meta learning and adaptive control”, arXiv:1805.08576, 2018.10.1109/ICRA.2019.8793542Search in Google Scholar
12. S. Phaniteja, P. Dewangan, P. Guhan, A. Sarkar, and K. M. Krishna, “A deep reinforcement learning approach for dynamically stable inverse kinematics of humanoid robots”, in ROBIO, 2017.10.1109/ROBIO.2017.8324682Search in Google Scholar
13. J. Hwangbo, J. Lee, A. Dosovitskiy, D. Bellicoso, V. Tsounis, V. Koltun, and M. Hutter, “Learning agile and dynamic motor skills for legged robots”, Science Robotics, vol. 4, no. 26, 2019.10.1126/scirobotics.aau5872Search in Google Scholar PubMed
14. S. Levine, P. Pastor, A. Krizhevsky, and D. Quillen, “Learning hand-eye coordination for robotic grasping with deep learning and large-scale data collection”, in International Symposium on Experimental Robotics (ISER), 2016.10.1007/978-3-319-50115-4_16Search in Google Scholar
15. T. Johannink, S. Bahl, A. Nair, J. Luo, A. Kumar, M. Loskyll, J. A. Ojea, E. Solowjow, and S. Levine, “Residual reinforcement learning for robot control”, arXiv:1812.03201, 2018.10.1109/ICRA.2019.8794127Search in Google Scholar
16. S. Stanev, R. Awad, M. Prieur, W. Walla, and J. Ovtcharova, “Production-oriented product validation method as support for the reuse of production lines in the automotive industry”, in 3rd International Conference on Changeable, Agile, Reconfigurable and Virtual Production (CARV), Herbert Utz Verlag, 2009.Search in Google Scholar
17. R. Awad, M. Fechter, and J. van Heerden, “Integrated risk assessment and safety consideration during design of hrc workplaces”, in 22nd IEEE International Conference on Emerging Technologies and Factory Automation (ETFA), IEEE, 2017.10.1109/ETFA.2017.8247648Search in Google Scholar
18. E. Todorov, T. Erez, and Y. Tassa, “MuJoCo: A physics engine for model-based control”, in IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), 2012.10.1109/IROS.2012.6386109Search in Google Scholar
19. T. Erez, Y. Tassa, and E. Todorov, “Simulation tools for model-based robotics: Comparison of bullet, havok, mujoco, ode and physx”, in IEEE International Conference on Robotics and Automation (ICRA), 2015.10.1109/ICRA.2015.7139807Search in Google Scholar
20. E. Rohmer, S. P. N. Singh, and M. Freese, “V-REP: A versatile and scalable robot simulation framework”, in IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), 2013.10.1109/IROS.2013.6696520Search in Google Scholar
21. V. Mnih, K. Kavukcuoglu, D. Silver, A. A. Rusu, J. Veness, M. G. Bellemare, A. Graves, M. Riedmiller, A. K. Fidjeland, G. Ostrovski, S. Petersen, C. Beattie, A. Sadik, I. Antonoglo, H. King, D. Kumaran, D. Wierstra, S. Legg and D. Hassabis, “Human-level control through deep reinforcement learning”, Nature, vol. 518, no. 7540, pp. 529–533, 2015.10.1038/nature14236Search in Google Scholar PubMed
22. M. Andrychowicz, F. Wolski, A. Ray, J. Schneider, R. Fong, P. Welinder, B. McGrew, J. Tobin, P. Abbeel, and W. Zarembah, “Hindsight experience replay”, in 31st Conference on Neural Information Processing Systems (NIPS), 2017.Search in Google Scholar
23. M.-A. Zöller and M. F. Huber, “Survey on Automated Machine Learning”, arXiv:1904.12054v1, 2019.Search in Google Scholar
24. X. B. Peng, M. Andrychowicz, W. Zaremba, and P. Abbeel, “Sim-to-real transfer of robotic control with dynamics randomization”, in IEEE International Conference on Robotics and Automation (ICRA), 2018.10.1109/ICRA.2018.8460528Search in Google Scholar
25. J. Tobin, R. Fong, A. Ray, J. Schneider, W. Zaremba, and P. Abbeel, “Domain randomization for transferring deep neural networks from simulation to the real world”, in IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), 2017.10.1109/IROS.2017.8202133Search in Google Scholar
26. K. Bousmalis, A. Irpan, P. Wohlhart, Y. Bai, M. Kelcey, M. Kalakrishnan, L. Downs, J. Ibarz, P. Pastor, K. Konolige, S. Levine, and V. Vanhoucke, “Using simulation and domain adaptation to improve efficiency of deep robotic grasping”, in IEEE International Conference on Robotics and Automation (ICRA), 2018.10.1109/ICRA.2018.8460875Search in Google Scholar
27. P. Dhariwal, C. Hesse, O. Klimov, A. Nichol, M. Plappert, A. Radford, J. Schulman, S. Sidor, Y. Wu, and P. Zhokhov, “Openai baselines”, https://github.com/openai/baselines, 2017.Search in Google Scholar
28. I. Caspi, G. Leibovich, G. Novik, and S. Endrawis, “Reinforcement learning coach”, https://doi.org/10.5281/zenodo.1134899, 2017.Search in Google Scholar
29. Y. Duan, X. Chen, R. Houthooft, J. Schulman, and P. Abbeel, “Benchmarking deep reinforcement learning for continuous control”, in 33rd International Conference on Machine Learning (ICML), vol. 48, JMLR, 2016.Search in Google Scholar
30. F. Aichele, B. Schenke, B. Eckstein, and A. Groz, “A framework for robot control software development and debugging using a real-time capable physics simulation”, in 47st International Symposium on Robotics (ISR), 2016.Search in Google Scholar
31. F. Khatami and C. Müller, “World Robotics 2018: Industrial Robots”, 2018.Search in Google Scholar
32. H. ElMaraghy and W. ElMaraghy, “Smart adaptable assembly systems”, CIRP, vol. 44, 2016.10.1016/j.procir.2016.04.107Search in Google Scholar
33. P. Tempel, F. Eger, A. Lechler, and A. Verl, “Schaltschrankbau 4.0: Eine Studie über die Automatisierungs- und Digitalisierungspotenziale in der Fertigung von Schaltschränken und Schaltanlagen im klassischen Maschinen- und Anlagenbau”, https://discover.eplan.eu/schaltschrankbau-studie-anwender, last accessed on July 2019.Search in Google Scholar
34. D. de Gregorio, R. Zanella, G. Palli, S. Pirozzi, and C. Melchiorri, “Integration of robotic vision and tactile sensing for wire-terminal insertion tasks”, IEEE Transactions on Automation Science and Engineering, vol. 16, no. 2, 2019.10.1109/TASE.2018.2847222Search in Google Scholar
35. P. Heisler, P. Steinmetz, I. S. Yoo, and J. Franke, “Automatization of the cable-routing-process within the automated production of wiring systems”, in Energy Efficiency in Strategy of Sustainable Production III, Applied Mechanics and Materials, Trans Tech Publications, 2017.10.4028/www.scientific.net/AMM.871.186Search in Google Scholar
36. K. Kleeberger, C. Landgraf, and M. F. Huber, “Large-scale 6D Object Pose Estimation Dataset for Industrial Bin-Picking”, in IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), 2019.10.1109/IROS40897.2019.8967594Search in Google Scholar
37. D. Kalashnikov, A. Irpan, P. Pastor, J. Ibarz, A. Herzog, E. Jang, D. Quillen, E. Holly, M. Kalakrishnan, V. Vanhoucke, and S. Levine, “QT-Opt: Scalable deep reinforcement learning for vision-based robotic manipulation”, in Conference on Robot Learning (CoRL), 2018.Search in Google Scholar
38. D. Quillen, E. Jang, O. Nachum, C. Finn, J. Ibarz, and S. Levine, “Deep reinforcement learning for vision-based robotic grasping: A simulated comparative evaluation of off-policy methods”, in IEEE International Conference on Robotics and Automation (ICRA), 2018.10.1109/ICRA.2018.8461039Search in Google Scholar
39. J. Mahler, J. Liang, S. Niyaz, M. Laskey, R. Doan, X. Liu, J. A. Ojea, and K. Goldberg, “Dex-Net 2.0: Deep learning to plan robust grasps with synthetic point clouds and analytic grasp metrics”, in Robotics: Science and Systems (RSS), 2017.10.15607/RSS.2017.XIII.058Search in Google Scholar
40. A. Depierre, E. Dellandréa, and L. Chen, “Jacquard: A large scale dataset for robotic grasp detection”, in IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), 2018.10.1109/IROS.2018.8593950Search in Google Scholar
41. J. Redmon and A. Angelova, “Real-time grasp detection using convolutional neural networks”, in IEEE International Conference on Robotics and Automation (ICRA), 2015.10.1109/ICRA.2015.7139361Search in Google Scholar
42. S. Kumra and C. Kanan, “Robotic grasp detection using deep convolutional neural networks”, in IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), 2017.10.1109/IROS.2017.8202237Search in Google Scholar
43. I. Lenz, H. Lee, and A. Saxena, “Deep learning for detecting robotic grasps”, in International Journal of Robotics Research (IJRR), 2015.10.1177/0278364914549607Search in Google Scholar
44. D. Morrison, J. Leitner, and P. Corke, “Closing the loop for robotic grasping: A real-time, generative grasp synthesis approach”, in Robotics: Science and Systems (RSS), 2018.10.15607/RSS.2018.XIV.021Search in Google Scholar
45. L. Halt, F. Nägele, P. Tenbrock, and A. Pott, “Intuitive constraint-based robot programming for robotic assembly tasks”, in IEEE International Conference on Robotics and Automation (ICRA), 2018.10.1109/ICRA.2018.8462882Search in Google Scholar
46. X. Zhang, A. S. Polydoros, and J. Piater, “Learning movement assessment primitives for force interaction skills”, arXiv:1805.04354, 2018.Search in Google Scholar
47. J. Schulman, F. Wolski, P. Dhariwal, A. Radford, and O. Klimov, “Proximal policy optimization algorithms”, arXiv:1707.06347, 2017.Search in Google Scholar
48. W. Bauer, M. Bender, M. Braun, P. Rally, and O. Scholtz, “Lightweight robots in manual assembly—best to start simply”, Examining companies’ initial experiences with lightweight robots, Stuttgart, 2016.Search in Google Scholar
49. ISO 10218-1:2011, Robots and robotic devices – Safety requirements for industrial robots – Part 1: Robots, ISO, 2011.Search in Google Scholar
50. ISO 10218-2:2011, Robots and robotic devices – Safety requirements for industrial robots – Part 2: Robot systems and integration, ISO, 2011.Search in Google Scholar
51. N. Schaaf and M. F. Huber, “Enhancing Decision Tree based Interpretation of Deep Neural Networks through L1-Orthogonal Regularization”, arXiv:1904.05394, 2019.10.1109/ICMLA.2019.00016Search in Google Scholar
© 2019 Walter de Gruyter GmbH, Berlin/Boston
