Selecting simple, transferable models with the supremum principle

Letter
Open Access

Selecting simple, transferable models with the supremum principle

Cody Petrie, Christian Anderson, Casie Maekawa, Travis Maekawa, and Mark K. Transtrum

Phys. Rev. Research 4, L032044 – Published 23 September 2022

Abstract

We consider how mathematical models enable predictions for conditions that are qualitatively different from the training data. We propose techniques based on information topology to find models that can apply their learning in regimes for which there is no data. The first step is to use the manifold boundary approximation method to construct simple, reduced models of target phenomena in a data-driven way. We consider the set of all such reduced models and use the topological relationships among them to reason about model selection for new, unobserved phenomena. Given minimal models for several target behaviors, we introduce the supremum principle as a criterion for selecting a new, transferable model. The supremal model, i.e., the least upper bound, is the simplest model that reduces to each of the target behaviors. We illustrate how to discover supremal models with several examples; in each case, the supremal model unifies causal mechanisms to transfer successfully to new target domains. We use these examples to motivate a general algorithm that has formal connections to theories of analogical reasoning in cognitive psychology.

Received 23 September 2021
Revised 11 February 2022
Accepted 1 June 2022

DOI:https://doi.org/10.1103/PhysRevResearch.4.L032044

Published by the American Physical Society under the terms of the Creative Commons Attribution 4.0 International license. Further distribution of this work must maintain attribution to the author(s) and the published article's title, journal citation, and DOI.

Published by the American Physical Society

Physics Subject Headings (PhySH)

Cell signaling Scientific reasoning & problem solving Signal transduction

Cells

Information theory Network Models Time series analysis

Statistical Physics & ThermodynamicsPhysics of Living SystemsGeneral Physics

Authors & Affiliations

Cody Petrie ^*, Christian Anderson , Casie Maekawa, Travis Maekawa , and Mark K. Transtrum^†

Department of Physics and Astronomy, Brigham Young University, Provo, Utah 84602, USA

^*codypetrie89@gmail.com
^†mktranstrum@byu.edu

Article Text

Click to Expand

Supplemental Material

Click to Expand

References

Click to Expand

Issue

Vol. 4, Iss. 3 — September - November 2022

Subject Areas

Reuse & Permissions

Author publication services for translation and copyediting assistance advertisement

Images

Figure 1
(a) Infections vs time for a hypothetical epidemic. Data taken at different times—early (red), late (blue), and intermediate (purple)—exhibit qualitatively different types of behaviors. These data carry information about different aspects of the generating process. The “True” curve represents the model from which data was generated, without the corruption of Gaussian noise. (b) Information geometry quantifies how data identify a model's parameters. The full trajectory completely identifies an SIR model, manifest by a model manifold that is not compressed along either direction. Restricting to data at early times compresses the directions related to a recovery rate so that the model is well approximated by an effective SI model. Similarly, restricting to late times compresses information about infection, leading to an effective IR model. (c) Candidate models of varying complexity can be arranged hierarchically in a directed graph. The SIR model is the supremum of the SI and IR models, i.e., the simplest model that combines information from both early and late times. It can use the information from data in two of the regimes to accurately predict a third.
Reuse & Permissions
Figure 2
(a) A network diagram showing the “full” model of the Wnt pathway. The red indicates mechanisms that are unique to the accumulation model, and blue indicates mechanisms unique to the oscillation model. (b) Three possible behaviors of $β$ -catenin in response to Wnt. The accumulation [19] and oscillation [20] behaviors are known to occur naturally, while behaviors similar to the interrupted behavior have been reported in Refs. [21, 22, 23, 24, 25], but not previously expressed mathematically.
Reuse & Permissions
Figure 3
(a) The Hasse diagram of key reduced models for the Wnt system. The nodes (models) are labeled by “A” for accumulation, “O” for oscillation, “C” for combined, as well as the number of parameters each model contains. The black line represents the original sequence of reduced accumulation models and the blue line represents the new sequence with parameter limits reordered to match those in the oscillation models. The central, blue arrow shows that the original model can be reduced to the supremal model using the same set of limits used to arrive at A9 and O9. Details about the model equations and sequence of limits can be found in the Supplemental Material [42]. (b) A general example of the diamond property. Consider a model with $N$ parameters containing two parameters $ε_{1}$ and $ε_{2}$ . The diamond property states that the order in which parameters can be removed commutes. The same model with $N - 2$ parameters can be reached by first taking either $ε_{1}$ or $ε_{2}$ to zero, and then taking the other to zero. The diamond property is used to reorder the limits of a sequence, to build a supremal model.
Reuse & Permissions

Physical Review Research