Meta-modeling game for deriving theory-consistent, microstructure-based traction–separation laws via deep reinforcement learning

https://doi.org/10.1016/j.cma.2018.11.026Get rights and content

Abstract

This paper presents a new meta-modeling framework that employs deep reinforcement learning (DRL) to generate mechanical constitutive models for interfaces. The constitutive models are conceptualized as information flow in directed graphs. The process of writing constitutive models is simplified as a sequence of forming graph edges with the goal of maximizing the model score (a function of accuracy, robustness and forward prediction quality). Thus meta-modeling can be formulated as a Markov decision process with well-defined states, actions, rules, objective functions and rewards. By using neural networks to estimate policies and state values, the computer agent is able to efficiently self-improve the constitutive model it generated through self-playing, in the same way AlphaGo Zero (the algorithm that outplayed the world champion in the game of Go) improves its gameplay. Our numerical examples show that this automated meta-modeling framework does not only produces models which outperform existing cohesive models on benchmark traction–separation data, but is also capable of detecting hidden mechanisms among micro-structural features and incorporating them in constitutive models to improve the forward prediction accuracy, both of which are difficult tasks to do manually.

Introduction

Constitutive responses of interfaces are important for a wide spectrum of problems that involve spatial domain with embedded strong discontinuity, such as fracture surfaces [1], [2], [3], [4], slip lines [5], [6], joints [7] and faults [8], [9], [10]. While earlier modeling efforts, in particular those involving the modeling of cohesive zones, often solely focus on mode I kinematics, the mixed mode predictions of traction–separation law relations are critical for numerous applications, ranging from predicting damage upon impacts [11], to predicting seismic events [12]. Park et al. [2] provide a comprehensive account of the major characteristic of traction–separation laws and conclude that, while there are differences in details, most of the traction–separation laws obey a number of universal principles, such as the indifference of any superimposed rigid-body motion, the finite work required to create new surface, the existence of characteristic length scales, and the vanishing of cohesive traction with sufficient separations.

In the case where the loading history is not monotonic, constitutive responses of interfaces often become path-dependent. For instance, geomaterials, such as fault gauges, are known to exhibit rate- and state-dependent frictional responses [13], [14], [15], [16], [17]. While there are phenomenological models designed to capture the path-dependent responses of the interfaces, a recent trend that gains increasing popularity is to replace the phenomenological traction–separation laws with a computational homogenization procedure to capture the responses of materials with heterogeneous microstructures (cf. Moës et al. [18], Hirschberger et al. [19], [20]). Nevertheless, as pointed out previously in Wang and Sun [9], the major issue of applying hierarchical multiscale coupling on interfacial problems is the increasing computational demand due to the large number of required representative elementary simulations, a trade-off that is widely known in FEM2 [21] and other homogenization-based multiscale methods, such as DEM-FEM [22], [23], [24], [25], [26], [27], [28], [29], [30].

To overcome this computational barrier, surrogate models are often derived to replicate the homogenized responses of sub-scale simulations [31], [32], [33], [34], [35], [36], [37]. Nevertheless, since surrogate models are often constitutive laws hand-crafted by modelers to incorporate morphology-dependent features [36], deriving, verifying and validating a surrogate model that can incorporate the essential information to yield macroscopic predictions with sufficient accuracy and robustness remain difficult and time-consuming. Data-driven models such as Le et al. [38], Bessa et al. [39], Versino et al. [40], Kafka et al. [41] and Wulfinghoff et al. [42] attempted to overcome this issue via supervised machine learning (e.g. neural network [43], symbolic regression model [40]) and unsupervised machine learning (e.g. dimensional reduction, feature extraction and clustering [39], [42]).

Recent work by Wang and Sun [9] attempted to resolve this issue by building a generic recurrent neural network that can easily incorporate different types of sub-scale information (e.g. porosity, fabric tensor, and relative displacement) to predict traction. This technique uses the concept of directed graph on the transfer learning approach (cf. Pan et al. [44]) in which multiple neural networks trained to make predictions on other physical quantities (e.g. relationship between porosity and fabric tensor) are re-used to generate additional inputs for predicting traction. However, the determination of the optimal input information (in addition to the displacement jump history) and configurations of information flow that enhances the prediction accuracy still requires a time-consuming trial-and-error procedure (cf. Section 4.3 Wang and Sun [9]).

In this work, we introduce a general artificial intelligence approach to automate the creation and validation of traction–separation models. Unlike the previous approach in which neural networks are often used to either identify material parameters or create black-box constitutive laws, this work focuses on leveraging the capacity of a computer to improve via self-playing, a technique commonly referred as (deep) reinforcement learning in the computer science community [45], [46], [47]. In the past two years, the functionality of algorithms automatically generated from deep reinforcement learning have achieved remarkable success. In many cases, the demonstrated capacities were thought to be impossible in the past. For instance, the algorithm trained by deep reinforcement learning created by a company called DeepMind is able to outperform human experts in Go, Chess and Atari games. The most exciting part of this achievement is that, unlike previous AI such as the IBM Deep Blue, the deep reinforcement learning does not rely on hand-crafted policy evaluation functions and is therefore applicable to different kinds of games once they are defined and implemented.

This success motivates us to propose a meta-modeling approach where deep reinforcement learning may generate constitutive laws for (1) a given set of data, (2) a well-defined objective, and (3) a given set of universal principles. To achieve this goal, we recast the process of writing a constitutive model as a game with components suitable for deep reinforcement learning, involving a sequence of actions completely compatible with the stated rules (i.e., the law of physics). First, we define the model score, which could be any objective function suitable for a given task. For instance, this objective can be minimizing the discrepancy between calibrated experimental results and blind predictions measured by a norm, or a constrained optimization problem that gives considerations on other attributes such as consistency, speed, and robustness [16]. Once the score (i.e., the objective) is clearly defined, we then implement the rules, which are the universal principles of mechanics, such as material frame indifference, laws of thermodynamics. These rules are applied in an environment in which scores are sampled. In the case of traction–separation law, the environment is simply the validation process itself.

Following this, we then define the action space which consists of a number of actions available for the modelers to write constitutive models. Once the action space and the model score are defined, we leverage the directed graph modeling technique to generate a state. The state at the end of each game represents a constitutive model automatically generated from the computer algorithm. In reality, the action space could be of very high dimensions such that manually deriving, implementing, verifying and validating all possible configurations are not feasible. This situation is similar to playing the games of chess and Go where the number of possible combinations of decisions or moves (each can be represented by a decision tree) remains finite but is so enormous that it is not possible to seek the optimal moves by exhausting all possibilities [48].

With the state, action, rule and objective defined, the most critical part is to assign reward for each action. In principle, if the action space is of very low dimension, i.e., there are not many ways to model the physical processes, then the reward for each action can be determined by exhausting all the possible model configurations. However, in the case of writing a complex traction–separation model, we cannot evaluate the quality of the model until its predictions are compared with benchmark data. Therefore, the ability to approximate the reward for each action (in our case the modeling choices) without the need to evaluate all the available options becomes crucial for the success of the meta-modeling approach.

The deep reinforcement learning is therefore ideal for us to achieve this goal. We can approximate the rewards via neural networks and the Bellman expectation equation [49], [50]. By repeatedly generating new constitutive laws (i.e., playing the game of writing models), the agent will use the reward obtained from each played game, in analogy with the binary game result (win/loss) at the end of a Go game, to update the action probabilities and value functions to improve the agent’s ability to write good constitutive laws. Through sufficient self-plays, the reinforcement learning algorithm then improves the modeling choices it made over time until it is ready for predictions. (See Fig. 1.)

There are a few major upshots for this approach. First, once the reinforcement learning algorithm is established, it can serve as a model generator without any human intervention. Second, since we regard the validation process as the environment component of the reinforcement learning, the performance of a resultant model is simultaneously evaluated and therefore validations are always a part of the model writing process. Third, the meta-modeling approach may easily embed any existing model previously hand-crafted by domain experts into the action space without re-implementing a new model. These unique capabilities enable us to have an unbiased tool to evaluate how well existing models fulfill a particular objective. Furthermore, since the model generation procedure is automated once an objective function is defined, this work may potentially eliminate the need of writing multiple incremental models for the same materials over time. Finally, this modeling approach is particularly powerful for discovering hidden physical coupling mechanisms that are otherwise too subtle to detect with human observation.

The rest of the paper is organized as follows. We first review the directed graph approach that enables us to generate and utilize a decision tree to represent the modeling process (Section 2). The definition of model scores is then described in Section 3. We then provide a formal definition of a game invented to generate traction–separation laws for predictions (Section 4). This is followed by a description on how to use the reinforcement learning for the traction–separation law generation (Section 5). Two numerical experiments are then used to showcase the performance of the automated meta-modeling approach using synthetic data from microscale discrete element simulations (Section 6). The major findings are then summarized in the conclusions.

Section snippets

Representing traction–separation law in directed graph

In this section, we introduce a building block for a simplified and extensible game that generates traction–separation laws by considering the relationships among different types of data collected from sub-scale simulations. In this game, the goal is to find a specific way to link different types of data such that a score function is maximized. Before we introduce the formal definition of the game, one necessary step is to recast the algorithm that leads to predictions from constitutive laws as

Score system for model evaluation and objective function

A score system must be introduced to evaluate the generated directed graphs for constitutive models such that the accuracy and credibility in replicating the mechanical behavior of real-world materials can be assessed. This score system may also serve as the objective function that defines the rewards for the deep reinforcement learning agent to improve the generated digraphs and resultant constitutive laws. In this work, we define the score as a positive real-valued function of the range [0,1]

Game of the traction–separation law

Our focus in this paper is primarily on the meta-modeling game invented for generating traction–separation models. Nevertheless, similar games can be defined for generating other types of constitutive models based on the ideas presented in this work. With the directed graph representations of traction–separation models as presented in Section 2, the process of developing a model can be recast as a game of making a sequence of decisions in generating edges between nodes in the digraphs. The

Deep reinforcement learning for generating constitutive laws

With the game of constitutive modeling completely defined, a deep reinforcement learning (DRL) algorithm is employed as a guidance of taking actions in the game to maximize the final model score (Fig. 5). This tactic is considered one of the key ideas leading to the major breakthrough in AI playing the game of Go (AlphaGo Zero) [47], Chess and shogi (Alpha Zero) [66] and many other games. The learning is completely free of human interventions. It does not need previous human knowledge in

Numerical experiments and applications

In this section, we present two traction–separation modeling games with different digraph complexities to demonstrate the intelligence, robustness and efficiency of the deep reinforcement learning algorithm on improving the accuracy and consistency of the generated traction–separation models through self-plays. For both examples in Sections 6.1 Numerical Experiment 1: Determining optimal physical relationships for traction–separation laws, 6.2 Numerical Experiment 2: Data-driven discovery for

Conclusion

This paper presents a meta-modeling approach in which we attempt to generate traction–separation laws not through explicitly writing a particular model but to provide the computer with modeling options such that it can explore on its own through self-practicing. Unlike previous deep-learning models thatonly leverage supervised learning techniques to train neural networks that makes black-box predictions, this new approach focuses on reinforcement learning technique to discover hidden

Acknowledgments

The corresponding author’s work is supported by the Earth Materials and Processes program from the US Army Research Office under grant contract W911NF-15-1-0442 and W911NF-15-1-0581, the Dynamic Materials and Interactions Program from the Air Force Office of Scientific Research under grant contract FA9550-17-1-0169, the nuclear energy university program from the Department of Energy under grant contract DE-NE0008534 as well as the Mechanics of Material program at National Science Foundation

References (84)

  • KiraneKedar et al.

    A cold dwell fatigue crack nucleation criterion for polycrystalline Ti-6242 using grain-level crystal plasticity FE model

    Int. J. Fatigue

    (2008)
  • KeshavarzShahriyar et al.

    Multi-scale crystal plasticity finite element model approach to modeling nickel-based superalloys

    Acta Mater.

    (2013)
  • PanchalJitesh H. et al.

    Key computational modeling issues in integrated computational materials engineering

    Comput. Aided Des.

    (2013)
  • FaisalTanvir R. et al.

    Computational study of the elastic properties of Rheum rhabarbarum tissues via surrogate models of tissue geometry

    J. Struct. Biol.

    (2014)
  • BessaM.A. et al.

    A framework for data-driven analysis of materials under uncertainty: countering the curse of dimensionality

    Comput. Methods Appl. Mech. Engrg.

    (2017)
  • VersinoDaniele et al.

    Data driven modeling of plastic deformation

    Comput. Methods Appl. Mech. Engrg.

    (2017)
  • WulfinghoffStephan et al.

    Model order reduction of nonlinear homogenization problems using a Hashin–Shtrikman type finite element method

    Comput. Methods Appl. Mech. Engrg.

    (2018)
  • LefikM. et al.

    Artificial neural network for parameter identifications for an elasto-plastic model of superconducting cable under cyclic loading

    Comput. Struct.

    (2002)
  • HornikKurt et al.

    Multilayer feedforward networks are universal approximators

    Neural Netw.

    (1989)
  • OliverJ. et al.

    From continuum mechanics to fracture mechanics: the strong discontinuity approach

    Eng. Fract. Mech.

    (2002)
  • RiceJames R.

    A path independent integral and the approximate analysis of strain concentration by notches and cracks

    J. Appl. Mech.

    (1968)
  • BryantEric C. et al.

    A mixed-mode phase field fracture model in anisotropic rocks with consistent kinematics

    Comput. Methods Appl. Mech. Engrg.

    (2018)
  • RabczukTimon et al.

    A new approach for modelling slip lines in geological materials with cohesive models

    Int. J. Numer. Anal. Methods Geomech.

    (2006)
  • BorjaRonaldo I. et al.

    Continuum mathematical modeling of slip weakening in geological systems

    J. Geophys. Res. Solid Earth

    (2007)
  • OhnakaMitiyasu et al.

    A cohesive zone model for dynamic shear faulting based on experimentally inferred constitutive relation and strong motion source parameters

    J. Geophys. Res. Solid Earth

    (1989)
  • OrtizMichael et al.

    Finite-deformation irreversible cohesive elements for three-dimensional crack-propagation analysis

    Internat. J. Numer. Methods Engrg.

    (1999)
  • RudnickiJohn W.

    Fracture mechanics applied to the Earth’s crust

    Ann. Rev. Earth Planet. Sci.

    (1980)
  • PatersonMervyn S. et al.

    Experimental Rock Deformation-The Brittle Field

    (2005)
  • SunWaiChing

    A unified method to predict diffuse and localized instabilities in sands

    Geomech. Geoeng.

    (2013)
  • BorjaRonaldo I.

    Plasticity: modeling & computation

    (2013)
  • WangKun et al.

    Identifying material parameters for a micro-polar plasticity model via x-ray micro-ct images: lessons learned from the curve-fitting exercises

    Int. J. Multiscale Comput. Eng.

    (2016)
  • HirschbergerClaudia Britta et al.

    Computational homogenization of material layers with micromorphic mesostructure

    Phil. Mag.

    (2008)
  • SunWaiChing et al.

    Connecting microstructural attributes and permeability from 3D tomographic images of in situ shear-enhanced compaction bands using multiscale computations

    Geophys. Res. Lett.

    (2011)
  • SunWai Ching et al.

    Multiscale method for characterization of porous microstructures and their impact on macroscopic effective permeability

    Internat. J. Numer. Methods Engrg.

    (2011)
  • FishJacob

    Practical Multiscaling

    (2013)
  • SunWaiChing et al.

    A multiscale DEM-LBM analysis on permeability evolutions inside a dilatant shear band

    Acta Geotech.

    (2013)
  • WangKun et al.

    Anisotropy of a tensorial Bishop’s coefficient for wetted granular materials

    J. Eng. Mech.

    (2015)
  • LiuYang et al.

    A nonlocal multiscale discrete-continuum model for predicting mechanical behavior of granular materials

    Internat. J. Numer. Methods Engrg.

    (2016)
  • KuhnMatthew R. et al.

    Stress-induced anisotropy in granular materials: fabric, stiffness, and permeability

    Acta Geotech.

    (2015)
  • WuHuanran et al.

    Multiscale modeling and analysis of compaction bands in high-porosity sandstones

    Acta Geotech.

    (2018)
  • VerhooselClemens V. et al.

    Computational homogenization for adhesive and cohesive failure in quasi-brittle solids

    Internat. J. Numer. Methods Engrg.

    (2010)
  • LiuYang et al.

    Determining material parameters for critical state plasticity models based on multilevel extended digital database

    J. Appl. Mech.

    (2016)
  • Cited by (109)

    • A framework for neural network based constitutive modelling of inelastic materials

      2024, Computer Methods in Applied Mechanics and Engineering
    View all citing articles on Scopus
    View full text