Paper Titles

Sustainability Aspects of Current Market Developments, Different Product Types and Innovative Manufacturing Processes of Electric Motors
p.64

An Information Processing Framework Facilitating the Implementation of Condition Monitoring in Cyber-Physical Systems
p.75

Transparent Acquisition and Processing of Energy Data in the Field of Industrial Production - Requirements and Applications
p.81

Implementation of an Intralogistics Routing-Service Basing on Decentralized Workspace Digitization
p.90

An Artificial Intelligence Approach for Online Optimization of Flexible Manufacturing Systems
p.96

Towards an Energy Efficient Series Production of High Performance Permanent Magnet Synchronous Motors by Selective Magnet Assembly
p.111

Optimization of the Process Reliability of the Ultrasonic Crimping Process by Evaluating the Mounting Conditions for Tubular Cable Lugs
p.119

Qualification of Direct Diode Lasers for Laser Beam Welding in Order to Enhance Process Efficiency
p.127

Contribution of Additive Manufacturing of Rare Earth Material to the Increase in Performance and Resource Efficiency of Permanent Magnets
p.135

HomeApplied Mechanics and MaterialsApplied Mechanics and Materials Vol. 882An Artificial Intelligence Approach for Online...

An Artificial Intelligence Approach for Online Optimization of Flexible Manufacturing Systems

Article Preview

Abstract:

This paper addresses the problem of efficiently operating a flexible manufacturing machine in an electricity micro-grid featuring a high volatility of electricity prices. The problem of finding the optimal control policy is formulated as a sequential decision making problem under uncertainty where, at every time step the uncertainty comes from the lack of knowledge about fu-ture electricity consumption and future weather dependent energy prices. We propose to address this problem using deep reinforcement learning. To this purpose, we designed a deep learning architecture to forecast the load profile of future manufacturing schedule from past production time series. Combined with the forecast of future energy prices, the reinforcement-learning algorithm is trained to perform an online optimization of the production ma-chine in order to reduce the long-term energy costs. The concept is empirical-ly validated on a flexible production machine, where the machine speed can be optimized during the production.

You might also be interested in these eBooks

Energy Efficiency in Strategy of Sustainable Production IV

Info:

Periodical:

Applied Mechanics and Materials (Volume 882)

Pages:

96-108

DOI:

https://doi.org/10.4028/www.scientific.net/AMM.882.96

Citation:

Cite this paper

Online since:

July 2018

Authors:

Jupiter Bakakeu*, Schirin Tolksdorf, Jochen Bauer, Hans-Henning Klos, Jörn Peschke, Adrian Fehrle, Werner Eberlein, Johannes Bürner, Matthias Brossog, Lars Jahn, Jörg Franke

Keywords:

Load Management, OPC UA, Reinforcement Learning

Export:

RIS, BibTeX

Price:

Permissions:

Request Permissions

* - Corresponding Author

[1] Faktenpapier atypische Netznutzung. DIHK und VEA - Bundesverband der Energie-Abnehmer e. V. Hannover, April (2015).

[2] Roesch M., Brugger M., Braunreuther S., Reinhart G.: Klassifizierung von Energieflexibilitätsmaßnahmen. ZWF. (2017).

DOI: 10.3139/104.111774

[3] R. S. Sutton and A. G. Barto, Introduction to Reinforcement Learning,, 1st ed. Cambridge, MA, USA: MIT Press, (1998).

[4] Gholian, H. Mohsenian-Rad and Y. Hua, Optimal Industrial Load Control in Smart Grid,,IEEE Trans. Smart Grid, vol. 7, no. 5, pp.2305-2316, (2016).

DOI: 10.1109/tsg.2015.2468577

[5] R. Chen, H. Sun, Q. Guo, H. Jin, W. Wu and B. Zhang, Profit-seeking energy–intensive enterprises participating in power system scheduling: Model and mechanism,,Appl. Energy , vol. 158, pp.263-274, (2015).

DOI: 10.1016/j.apenergy.2015.08.018

[6] Z. Wang, F. Gao, Q. Zhai, X. Guan, K. Liu and D. Zhou, An integrated optimization model for generation and batch production load scheduling in energy intensive enterprise,, in Proc. IEEE Power Eng. Soc. Gen. Meet., 2012, pp.1-8.

DOI: 10.1109/pesgm.2012.6345296

[7] A. Middelberg , J. F. Zhang and X. H. Xia, An optimal control model for load shifting – with application in the energy management of a colliery,,Appl. Energy, vol. 86, pp.1266-1273, (2009).

DOI: 10.1016/j.apenergy.2008.09.011

[8] J. Wang et al, Optimal vehicle batching and sequencing to reduce energy consumption and atmospheric emissions in automotive paint shops,, Int. J. Sustainable Manuf., vol. 2, (2011).

DOI: 10.1504/ijsm.2011.042149

[9] F. Shrouf et al, Optimizing the production scheduling of a single machine to minimize total energy consumption costs,, J. Cleaner Prod., vol. 67, p.197–207, (2014).

DOI: 10.1016/j.jclepro.2013.12.024

[10] G. Chen, L. Zhang, J. Arinez, and S. Biller, Energy-efficient production systems through schedule-based operations,, IEEE Trans. Autom. Sci. Eng., vol. 10, no. 1, p.27–37, Jan. (2013).

DOI: 10.1109/tase.2012.2202226

[11] Z. Sun and L. Li, Opportunity estimation for real-time energy control of sustainable manufacturing systems,, IEEE Trans. Autom. Sci. Eng., vol. 10, no. 1, p.38–44, Jan. (2013).

DOI: 10.1109/tase.2012.2216876

[12] L. Li and Z. Sun, Dynamic energy control for energy efficiency improvement of sustainable manufacturing systems using Markov decision process,, IEEE Trans. Syst., Man, Cybern. Syst, vol.43, no.5, p.1195–1205, Sep (2013).

DOI: 10.1109/tsmc.2013.2256856

[13] J. R. Duflou et al, Towards energy and resource efficient manufacturing: A processes and systems approach,, CIRP Annals—Manuf. Technol., vol. 61, no. 2, p.587–609, (2012).

DOI: 10.1016/j.cirp.2012.05.002

[14] Pechmann and I. Schler, Optimizing energy costs by intelligent production scheduling,, in Glocalized Solutions for Sustainability Manufacturing. New York, NY, USA: Springer, (2011).

[15] Bruzzone et al, Energy aware scheduling for improving manufacturing process sustainability: A mathematical model for flexible flow shops,, CIRP Annals—Manuf. Technol. , vol. 61, (2012).

DOI: 10.1016/j.cirp.2012.03.084

[16] K. Fang, et al, A new approach to scheduling in manufacturing for power consumption and carbon footprint reduction,, J. Manuf. Syst., (2011).

[17] S. Ashok, Peak-load management in steel plants,, Appl. Energy, vol. 83, p.413–424, (2006).

DOI: 10.1016/j.apenergy.2005.05.002

[18] Middelberg, J. Zhang, and X. Xia, An optimal control model for load shifting with application in the energy management of a colliery,, Appl. Energy , vol. 86, p.1266–1273, (2009).

DOI: 10.1016/j.apenergy.2008.09.011

[19] Z. Sun, L. Li, M. Fernandez, and J. Wang, Inventory control for peak electricity demand reduction of manufacturing systems considering the tradeoff between production loss and energy savings,, J. Cleaner Prod., vol. 82, p.84–93, (2014).

DOI: 10.1016/j.jclepro.2014.06.071

[20] M. R. Alam, M. St-Hilaire, and T. Kunz, Computational methods for residential energy cost optimization in smart grids: A survey,, ACM Comput. Surv., vol. 49, p.2:1–2:34, Apr. (2016).

DOI: 10.1145/2897165

[21] Barbato and A. Capone, Optimization models and methods for demand-side management of residential users: A survey,, Energies, vol. 7, no. 9, p.5787 – 5824, (2014).

DOI: 10.3390/en7095787

[22] E. Loukarakis, C. J. Dent, and J. W. Bialek, Decentralized multi-period economic dispatch for real-time flexible demand management,, IEEE Transactions on Power Systems, vol. 31, (2016).

DOI: 10.1109/tpwrs.2015.2402518

[23] H. Mohsenian-Rad and A. Leon-Garcia, Optimal residential load control with price prediction in real-time electricity pricing environments,, IEEE Transactions on Smart Grid, (2010).

DOI: 10.1109/tsg.2010.2055903

[24] N. G. Paterakis et al, Coordinated operation of a neighborhood of smart households comprising electric vehicles, energy storage and distributed generation,, IEEE Transactions on Smart Grid, vol. 7, no. 6, p.2736–2747, Nov (2016).

DOI: 10.1109/tsg.2015.2512501

[25] J. Vardakas et al, A survey on deman response programs in smart grids: Pricing methods and optimization algorithms,, IEEE Comunication Surveys Tutorials, vol. 17, no. 1, p.152–178, (2015).

DOI: 10.1109/comst.2014.2341586

[26] L. A. Hurtado et al, Comfort-constrained demand flexibility management for building aggregations using a decentralized approach,, in International Conference on Smart Cities and Green ICT Systems, May 2015, p.1–10.

DOI: 10.5220/0005444101570166

[27] Liu, X. Xu and D. Hu, Multiobjective reinforcement learning: A comprehensive overview,, IEEE Transactions on Systems, Man, and Cybernetics: Systems, vol. 45, no. 3, pp.385-398, (2015).

DOI: 10.1109/tsmc.2014.2358639

[28] R. S. Sutton and A. G. Barto, Introduction to Reinforcement Learning,, Third ed. Cambridge, MA, USA: MIT Press, (2017).

[29] V. Mnih et al, Human-level control through deep reinforcement learning,, Nature, (2015).

[30] S. Gu, E. Holly, E. Timothy, P. Lillicrap and S. Levine, Mastering the game of Go without human knowledge,, Springer Nature, (2016).

[31] J. Hwangbo, I. Sa, R. Siegwart, and M. Hutter, Control of a Quadrotor with Reinforcement Learning,, IEEE Robotics And Automation Letters, (2017).

DOI: 10.1109/lra.2017.2720851

[32] W. Xia, H. Li and B. Li, A Control Strategy of Autonomous Vehicles Based on Deep Reinforcement Learning,, 9th International Symposium on Computational Intelligence and Design (ISCID), Hangzhou, 2016, pp.198-201.

DOI: 10.1109/iscid.2016.2054

[33] N.N, Measurement instruction to determine the energy- and resource demand ofmachine tools for mass production,, VDMA 34179, Beuth Verlag GmbH, (2015).

[34] Arens, T. Hopfgartner, T. Jensen, M. Lamping, M. Pieper, Packaging Machine Language V3.0 Mode & States Definition Document,, http://www.omac.org, (2006).