Meta-modeling game for deriving theory-consistent, microstructure-based traction–separation laws via deep reinforcement learning
Introduction
Constitutive responses of interfaces are important for a wide spectrum of problems that involve spatial domain with embedded strong discontinuity, such as fracture surfaces [1], [2], [3], [4], slip lines [5], [6], joints [7] and faults [8], [9], [10]. While earlier modeling efforts, in particular those involving the modeling of cohesive zones, often solely focus on mode I kinematics, the mixed mode predictions of traction–separation law relations are critical for numerous applications, ranging from predicting damage upon impacts [11], to predicting seismic events [12]. Park et al. [2] provide a comprehensive account of the major characteristic of traction–separation laws and conclude that, while there are differences in details, most of the traction–separation laws obey a number of universal principles, such as the indifference of any superimposed rigid-body motion, the finite work required to create new surface, the existence of characteristic length scales, and the vanishing of cohesive traction with sufficient separations.
In the case where the loading history is not monotonic, constitutive responses of interfaces often become path-dependent. For instance, geomaterials, such as fault gauges, are known to exhibit rate- and state-dependent frictional responses [13], [14], [15], [16], [17]. While there are phenomenological models designed to capture the path-dependent responses of the interfaces, a recent trend that gains increasing popularity is to replace the phenomenological traction–separation laws with a computational homogenization procedure to capture the responses of materials with heterogeneous microstructures (cf. Moës et al. [18], Hirschberger et al. [19], [20]). Nevertheless, as pointed out previously in Wang and Sun [9], the major issue of applying hierarchical multiscale coupling on interfacial problems is the increasing computational demand due to the large number of required representative elementary simulations, a trade-off that is widely known in FEM2 [21] and other homogenization-based multiscale methods, such as DEM-FEM [22], [23], [24], [25], [26], [27], [28], [29], [30].
To overcome this computational barrier, surrogate models are often derived to replicate the homogenized responses of sub-scale simulations [31], [32], [33], [34], [35], [36], [37]. Nevertheless, since surrogate models are often constitutive laws hand-crafted by modelers to incorporate morphology-dependent features [36], deriving, verifying and validating a surrogate model that can incorporate the essential information to yield macroscopic predictions with sufficient accuracy and robustness remain difficult and time-consuming. Data-driven models such as Le et al. [38], Bessa et al. [39], Versino et al. [40], Kafka et al. [41] and Wulfinghoff et al. [42] attempted to overcome this issue via supervised machine learning (e.g. neural network [43], symbolic regression model [40]) and unsupervised machine learning (e.g. dimensional reduction, feature extraction and clustering [39], [42]).
Recent work by Wang and Sun [9] attempted to resolve this issue by building a generic recurrent neural network that can easily incorporate different types of sub-scale information (e.g. porosity, fabric tensor, and relative displacement) to predict traction. This technique uses the concept of directed graph on the transfer learning approach (cf. Pan et al. [44]) in which multiple neural networks trained to make predictions on other physical quantities (e.g. relationship between porosity and fabric tensor) are re-used to generate additional inputs for predicting traction. However, the determination of the optimal input information (in addition to the displacement jump history) and configurations of information flow that enhances the prediction accuracy still requires a time-consuming trial-and-error procedure (cf. Section 4.3 Wang and Sun [9]).
In this work, we introduce a general artificial intelligence approach to automate the creation and validation of traction–separation models. Unlike the previous approach in which neural networks are often used to either identify material parameters or create black-box constitutive laws, this work focuses on leveraging the capacity of a computer to improve via self-playing, a technique commonly referred as (deep) reinforcement learning in the computer science community [45], [46], [47]. In the past two years, the functionality of algorithms automatically generated from deep reinforcement learning have achieved remarkable success. In many cases, the demonstrated capacities were thought to be impossible in the past. For instance, the algorithm trained by deep reinforcement learning created by a company called DeepMind is able to outperform human experts in Go, Chess and Atari games. The most exciting part of this achievement is that, unlike previous AI such as the IBM Deep Blue, the deep reinforcement learning does not rely on hand-crafted policy evaluation functions and is therefore applicable to different kinds of games once they are defined and implemented.
This success motivates us to propose a meta-modeling approach where deep reinforcement learning may generate constitutive laws for (1) a given set of data, (2) a well-defined objective, and (3) a given set of universal principles. To achieve this goal, we recast the process of writing a constitutive model as a game with components suitable for deep reinforcement learning, involving a sequence of actions completely compatible with the stated rules (i.e., the law of physics). First, we define the model score, which could be any objective function suitable for a given task. For instance, this objective can be minimizing the discrepancy between calibrated experimental results and blind predictions measured by a norm, or a constrained optimization problem that gives considerations on other attributes such as consistency, speed, and robustness [16]. Once the score (i.e., the objective) is clearly defined, we then implement the rules, which are the universal principles of mechanics, such as material frame indifference, laws of thermodynamics. These rules are applied in an environment in which scores are sampled. In the case of traction–separation law, the environment is simply the validation process itself.
Following this, we then define the action space which consists of a number of actions available for the modelers to write constitutive models. Once the action space and the model score are defined, we leverage the directed graph modeling technique to generate a state. The state at the end of each game represents a constitutive model automatically generated from the computer algorithm. In reality, the action space could be of very high dimensions such that manually deriving, implementing, verifying and validating all possible configurations are not feasible. This situation is similar to playing the games of chess and Go where the number of possible combinations of decisions or moves (each can be represented by a decision tree) remains finite but is so enormous that it is not possible to seek the optimal moves by exhausting all possibilities [48].
With the state, action, rule and objective defined, the most critical part is to assign reward for each action. In principle, if the action space is of very low dimension, i.e., there are not many ways to model the physical processes, then the reward for each action can be determined by exhausting all the possible model configurations. However, in the case of writing a complex traction–separation model, we cannot evaluate the quality of the model until its predictions are compared with benchmark data. Therefore, the ability to approximate the reward for each action (in our case the modeling choices) without the need to evaluate all the available options becomes crucial for the success of the meta-modeling approach.
The deep reinforcement learning is therefore ideal for us to achieve this goal. We can approximate the rewards via neural networks and the Bellman expectation equation [49], [50]. By repeatedly generating new constitutive laws (i.e., playing the game of writing models), the agent will use the reward obtained from each played game, in analogy with the binary game result (win/loss) at the end of a Go game, to update the action probabilities and value functions to improve the agent’s ability to write good constitutive laws. Through sufficient self-plays, the reinforcement learning algorithm then improves the modeling choices it made over time until it is ready for predictions. (See Fig. 1.)
There are a few major upshots for this approach. First, once the reinforcement learning algorithm is established, it can serve as a model generator without any human intervention. Second, since we regard the validation process as the environment component of the reinforcement learning, the performance of a resultant model is simultaneously evaluated and therefore validations are always a part of the model writing process. Third, the meta-modeling approach may easily embed any existing model previously hand-crafted by domain experts into the action space without re-implementing a new model. These unique capabilities enable us to have an unbiased tool to evaluate how well existing models fulfill a particular objective. Furthermore, since the model generation procedure is automated once an objective function is defined, this work may potentially eliminate the need of writing multiple incremental models for the same materials over time. Finally, this modeling approach is particularly powerful for discovering hidden physical coupling mechanisms that are otherwise too subtle to detect with human observation.
The rest of the paper is organized as follows. We first review the directed graph approach that enables us to generate and utilize a decision tree to represent the modeling process (Section 2). The definition of model scores is then described in Section 3. We then provide a formal definition of a game invented to generate traction–separation laws for predictions (Section 4). This is followed by a description on how to use the reinforcement learning for the traction–separation law generation (Section 5). Two numerical experiments are then used to showcase the performance of the automated meta-modeling approach using synthetic data from microscale discrete element simulations (Section 6). The major findings are then summarized in the conclusions.
Section snippets
Representing traction–separation law in directed graph
In this section, we introduce a building block for a simplified and extensible game that generates traction–separation laws by considering the relationships among different types of data collected from sub-scale simulations. In this game, the goal is to find a specific way to link different types of data such that a score function is maximized. Before we introduce the formal definition of the game, one necessary step is to recast the algorithm that leads to predictions from constitutive laws as
Score system for model evaluation and objective function
A score system must be introduced to evaluate the generated directed graphs for constitutive models such that the accuracy and credibility in replicating the mechanical behavior of real-world materials can be assessed. This score system may also serve as the objective function that defines the rewards for the deep reinforcement learning agent to improve the generated digraphs and resultant constitutive laws. In this work, we define the score as a positive real-valued function of the range
Game of the traction–separation law
Our focus in this paper is primarily on the meta-modeling game invented for generating traction–separation models. Nevertheless, similar games can be defined for generating other types of constitutive models based on the ideas presented in this work. With the directed graph representations of traction–separation models as presented in Section 2, the process of developing a model can be recast as a game of making a sequence of decisions in generating edges between nodes in the digraphs. The
Deep reinforcement learning for generating constitutive laws
With the game of constitutive modeling completely defined, a deep reinforcement learning (DRL) algorithm is employed as a guidance of taking actions in the game to maximize the final model score (Fig. 5). This tactic is considered one of the key ideas leading to the major breakthrough in AI playing the game of Go (AlphaGo Zero) [47], Chess and shogi (Alpha Zero) [66] and many other games. The learning is completely free of human interventions. It does not need previous human knowledge in
Numerical experiments and applications
In this section, we present two traction–separation modeling games with different digraph complexities to demonstrate the intelligence, robustness and efficiency of the deep reinforcement learning algorithm on improving the accuracy and consistency of the generated traction–separation models through self-plays. For both examples in Sections 6.1 Numerical Experiment 1: Determining optimal physical relationships for traction–separation laws, 6.2 Numerical Experiment 2: Data-driven discovery for
Conclusion
This paper presents a meta-modeling approach in which we attempt to generate traction–separation laws not through explicitly writing a particular model but to provide the computer with modeling options such that it can explore on its own through self-practicing. Unlike previous deep-learning models thatonly leverage supervised learning techniques to train neural networks that makes black-box predictions, this new approach focuses on reinforcement learning technique to discover hidden
Acknowledgments
The corresponding author’s work is supported by the Earth Materials and Processes program from the US Army Research Office under grant contract W911NF-15-1-0442 and W911NF-15-1-0581, the Dynamic Materials and Interactions Program from the Air Force Office of Scientific Research under grant contract FA9550-17-1-0169, the nuclear energy university program from the Department of Energy under grant contract DE-NE0008534 as well as the Mechanics of Material program at National Science Foundation
References (84)
- et al.
A unified potential-based cohesive model of mixed-mode fracture
J. Mech. Phys. Solids
(2009) - et al.
A unified variational eigen-erosion framework for interacting brittle fractures and compaction bands in fluid-infiltrating porous media
Comput. Methods Appl. Mech. Engrg.
(2017) - et al.
The cohesive zone model: advantages, limitations and challenges
Eng. Fract. Mech.
(2002) - et al.
A multiscale multi-permeability poroplasticity model linked by recursive homogenizations and deep learning
Comput. Methods Appl. Mech. Engrg.
(2018) - et al.
Prediction of permeability and formation factor of sandstone with hybrid lattice Boltzmann/finite element simulation on microtomographic images
Int. J. Rock Mech. Mining Sci.
(2018) - et al.
Computational thermo-hydro-mechanics for multiphase freezing and thawing porous media in the finite deformation range
Comput. Methods Appl. Mech. Engrg.
(2017) - et al.
A computational approach to handle complex microstructure geometries
Comput. Methods Appl. Mech. Engrg.
(2003) - et al.
Computational multiscale modelling of heterogeneous material layers
Eng. Fract. Mech.
(2009) A multilevel finite element method (FE2) to describe the response of highly non-linear structures using generalized continua
Comput. Methods Appl. Mech. Engrg.
(2003)- et al.
A semi-implicit discrete-continuum coupling method for porous media based on the effective stress principle at finite strain
Comput. Methods Appl. Mech. Engrg.
(2016)
A cold dwell fatigue crack nucleation criterion for polycrystalline Ti-6242 using grain-level crystal plasticity FE model
Int. J. Fatigue
Multi-scale crystal plasticity finite element model approach to modeling nickel-based superalloys
Acta Mater.
Key computational modeling issues in integrated computational materials engineering
Comput. Aided Des.
Computational study of the elastic properties of Rheum rhabarbarum tissues via surrogate models of tissue geometry
J. Struct. Biol.
A framework for data-driven analysis of materials under uncertainty: countering the curse of dimensionality
Comput. Methods Appl. Mech. Engrg.
Data driven modeling of plastic deformation
Comput. Methods Appl. Mech. Engrg.
Model order reduction of nonlinear homogenization problems using a Hashin–Shtrikman type finite element method
Comput. Methods Appl. Mech. Engrg.
Artificial neural network for parameter identifications for an elasto-plastic model of superconducting cable under cyclic loading
Comput. Struct.
Multilayer feedforward networks are universal approximators
Neural Netw.
From continuum mechanics to fracture mechanics: the strong discontinuity approach
Eng. Fract. Mech.
A path independent integral and the approximate analysis of strain concentration by notches and cracks
J. Appl. Mech.
A mixed-mode phase field fracture model in anisotropic rocks with consistent kinematics
Comput. Methods Appl. Mech. Engrg.
A new approach for modelling slip lines in geological materials with cohesive models
Int. J. Numer. Anal. Methods Geomech.
Continuum mathematical modeling of slip weakening in geological systems
J. Geophys. Res. Solid Earth
A cohesive zone model for dynamic shear faulting based on experimentally inferred constitutive relation and strong motion source parameters
J. Geophys. Res. Solid Earth
Finite-deformation irreversible cohesive elements for three-dimensional crack-propagation analysis
Internat. J. Numer. Methods Engrg.
Fracture mechanics applied to the Earth’s crust
Ann. Rev. Earth Planet. Sci.
Experimental Rock Deformation-The Brittle Field
A unified method to predict diffuse and localized instabilities in sands
Geomech. Geoeng.
Plasticity: modeling & computation
Identifying material parameters for a micro-polar plasticity model via x-ray micro-ct images: lessons learned from the curve-fitting exercises
Int. J. Multiscale Comput. Eng.
Computational homogenization of material layers with micromorphic mesostructure
Phil. Mag.
Connecting microstructural attributes and permeability from 3D tomographic images of in situ shear-enhanced compaction bands using multiscale computations
Geophys. Res. Lett.
Multiscale method for characterization of porous microstructures and their impact on macroscopic effective permeability
Internat. J. Numer. Methods Engrg.
Practical Multiscaling
A multiscale DEM-LBM analysis on permeability evolutions inside a dilatant shear band
Acta Geotech.
Anisotropy of a tensorial Bishop’s coefficient for wetted granular materials
J. Eng. Mech.
A nonlocal multiscale discrete-continuum model for predicting mechanical behavior of granular materials
Internat. J. Numer. Methods Engrg.
Stress-induced anisotropy in granular materials: fabric, stiffness, and permeability
Acta Geotech.
Multiscale modeling and analysis of compaction bands in high-porosity sandstones
Acta Geotech.
Computational homogenization for adhesive and cohesive failure in quasi-brittle solids
Internat. J. Numer. Methods Engrg.
Determining material parameters for critical state plasticity models based on multilevel extended digital database
J. Appl. Mech.
Cited by (109)
Self-consistency Reinforced minimal Gated Recurrent Unit for surrogate modeling of history-dependent non-linear problems: Application to history-dependent homogenized response of heterogeneous materials
2024, Computer Methods in Applied Mechanics and EngineeringDiscovering interpretable elastoplasticity models via the neural polynomial method enabled symbolic regressions
2024, Computer Methods in Applied Mechanics and EngineeringA framework for neural network based constitutive modelling of inelastic materials
2024, Computer Methods in Applied Mechanics and EngineeringData-driven multiscale modelling of granular materials via knowledge transfer and sharing
2023, International Journal of PlasticityData-driven constitutive modelling of granular soils considering multiscale particle morphology
2023, Computers and Geotechnics