NLP-inspired structural pattern recognition in chemical application

Highlights

• NLP-inspired structural pattern recognition was applied to predict chemical activity.

• It is combinable with statistical pattern recognition with early or late fusion.

• The new method allows searching for “structural alerts” and their combinatorial patterns.

• A biodegradability prediction system serves to illustrate the new method.

• The source code, data sets and the list of found detected structural alerts are available from the corresponding author on request.

Abstract

In this paper we report on a new structural pattern recognition approach for in silico prediction of chemical activity. It is based on grammatical inference on strings representing chemical compounds and string edit distance between a chemical compound and a formal grammar generalizing an activity class. In the late 1980s Weininger published a chemical language with a very simple and natural grammar. Recently, the algorithms suitable to process this language have been developed. From modeling of chemical activity with formal grammars and chemical compounds as words, a functionality is derivable to search for “structural alerts”, that is, molecular substructures and their combinatorial patterns that cause a molecule to have properties of interest. A biodegradability prediction system has been constructed to serve as an example throughout the paper. The source code and various files from the experiment are available from the corresponding author on request.

Introduction

New chemicals keep appearing and require certification with respect to their activity properties. In order to substitute costly wet-lab testing, computational alternatives are sought. Apart from that, in silico methods are necessary to guide chemical design in order to discard candidate chemical structures that can give rise to unwanted properties as cheaply as possible and as early as possible. Biodegradable activity will be used as an example throughout this paper and is a measure of how readily a chemical compound can be put to use by microorganisms and decomposed in nature. Biodegradability is a quality necessary to take into account on in a wide range of contexts, for example, its high value is a must for modern packing material. A good predictive model with high probability discards unsuitable candidate materials and thus cuts down on the exploration space.

A fundamental concept of chemistry is that the structural characteristics of the molecule are responsible for its properties. The task of in silico prediction is as follows: given formula or graph of a chemical compound, predict whether it is active with respect to a property of interest. For example,–Will it be toxic? or –Will it be biodegradable? An early comprehensive review of the problem, with the key concepts defined, and classical solution can be found in [1]. Twenty years of efforts since then have resulted in big progress and distinct methodologies. Despite that, there are still calls out for the improvement of the existing predictive models on the side of wet-lab practitioners [2] and active work in the search for good new features (“descriptors”) [3], [4].

Statistical pattern recognition is often successfully undertaken: many numeric features are extracted, for example with the DRAGON system [5], then significant ones are selected, and a classification function is trained. Statistical pattern recognition is straightforward, but may not be ideal for many realistic problems, where patterns include structural or relational information and are difficult to quantify as feature vectors.

Graphs would be an ideal representation of molecules, but unfortunately the graph space has no strong algebraic properties, – neither a group nor a vector space. This lack of mathematical properties does not allow to readily combine structural and statistical pattern recognition methods. Graph embedding methods bridge the expressive (and prohibitively expensive) graph-based representation to statistical machine learning models. These methods are categorized into implicit and explicit graph embedding [6].

Implicit graph embedding refers to kernel methods, e.g. [7], [8], [9]. A graph kernel is a function that can be thought of as a dot product in some implicitly existing vector space. Kernel methods handle non-linear complex tasks in a new space and the separating function has the form

$$f(\mathbf{u})=\mathit{sign}\left({\displaystyle \sum _{i=1}^l}{\alpha }_i{y}_i〈\varphi ({\mathbf{u}}_i)\text{,}\varphi (\mathbf{u})〉+b\right)\text{,}$$

where $\mathbf{u}$ is a new sample about which the decision needs to be outputted. The training set $S=\{({\mathbf{u}}_1\text{,}{y}_1)\text{,}\dots \text{,}({\mathbf{u}}_l\text{,}{y}_l)\}\text{,}({\mathbf{u}}_i\text{,}{y}_i)$ is in X $\times$ Y and $i=1\text{,}\dots \text{,}l$. X is the inner product space and Y is the class value. $Y=\{-1\text{,}1\}$. The solution to the hyperplane (a set of values for ${\alpha }_i$) is found via quadratic programming, and ${\alpha }_i$ are non-zero for support vectors only. The decision function f outputs a prediction depending on which side of the hyperplane the input pattern $\mathbf{u}$ lies. The hyperplane is

$${\displaystyle \sum _{i=1}^l}{\alpha }_i{y}_i〈\varphi ({\mathbf{u}}_i)\text{,}\varphi (\mathbf{u})〉+b)=0\text{.}$$

A good kernel method maps graphs having a small edit-distance to close values. A valid kernel function must be a positive semi-definite.

Implicit graph embedding is not an optimal choice if interpretation of the decision is wanted, say for a banker or a chemist. Also kernel expressivity is significantly lower than wanted (that of graphs), and thus their use implies loss of potentially important structural information. For example, tree-let kernels take into account labeled subtrees with $⩽6$ nodes, while some groups of atoms predefining typical chemical relations are bigger. Combinatorial patterns of these subtrees are also out of reach for the kernel. Sometimes graph-distance is incorporated into a kernel, e.g. in Laplacial kernel [10], but it has complexity problems and needs to rely on an approximation.

Explicit graph embedding methods explicitly embed an input graph into a feature vector and thus enable the use of methodologies and techniques devised for vector spaces. The selection of the axis of the feature vector requires thus a fine analysis of the dataset in order to select representative features. Explicit graph embedding can be naturally combined with statistical pattern recognition (if at all needs to be separated from it).

A conclusion on the comparison of different methods achieved in [6] is that implicit and explicit graph methods are competitive and comparable. Certainly, depending on the data some variations can appear but they are small and they rather not be a criterion for choosing one method or the other, while other considerations should be taken into account, such as computational complexity or how a method is thought to be used.

Our idea is a natural language inspired approach, in which grammars are used to create a definition of the structure of each pattern class. In the late 1980s Weininger proposed “a chemical language with a very small and natural grammar” [11], called SMILES (Simplified Molecular Input Line Entry System). Although the generative grammars were not explicitly mentioned, the allusion is strong and implied that grammar inference, in the style of natural language processing communities, might be feasible for the chemical language too. The objects under modeling, i.e. formal expressions with tree structure, have good algebraic properties [13].

Grammatical features were found to be useful for activity prediction in our earlier work [14], [15], but then in both cases they were used within the explicit graph embedding scheme with other features containing key chemical information known to be highly relevant to activity, such as the graph shape of the compound and counts of the specific atomic groups predefining typical reactions.¹ The method was demonstrated to be naturally combinable with statistical pattern recognition in the form of early or late fusion. Under the scheme with early fusion, the edit distances are used as features with other graph features in the explicit scheme. For late fusion distinct predictive models (physicochemical and structural) compare their answers and their probabilities. Should the models disagree, a more confident method overtakes the decision.

The contributions of this submission are:

• NLP-inspired structural pattern recognition approach for chemical activity prediction, and

• a successful attempt to have expressiveness close to that of a graph without running into notorious complexity problems or poor mathematical characterizations of the objects, and

• a functionality derivable from it to search for “structural alerts”, i.e. the substructures and their combinatorial patterns that make a molecule active.

The rest of the paper is organized as follows. Section 2 covers the chemical language SMILES. In Section 3 the proposed pattern recognition based on grammatical inference and distances to grammars is explained together with the functionality to search for activity alerts. In Section 4 an experiment of biodegradability prediction serves as an example for the previous sections.

The source code, various files used in the experiment and the list of obtained structural alerts are available from the corresponding author on request.