Elsevier

Neural Networks

Volume 11, Issue 4, June 1998, Pages 699-707
Neural Networks

CONTRIBUTED ARTICLE
Neural computations of algebraic and geometrical structures

https://doi.org/10.1016/S0893-6080(97)00152-4Get rights and content

Abstract

How Artificial Neural Networks (ANN) can be used to solve problems in algebra and geometry by modelling specific subnetwork nodes and connections is considered. This approach has the benefit of producing ANNs with well-defined hidden units and reduces the search to parameters which satisfy known model constraints—yet still gains from the benefits inherent in neural computing architectures.

Introduction

Over the past fifty years or so there has been considerable interest in how perceptron-type neuron models can be combined to solve problems in logic, pattern recognition, language understanding and perception. This paper extends such inquiries to include an analysis of how such models can also be used to solve problems in algebra and geometry. In order to accomplish this we show how the more traditional formulations of neural networks need to be replaced by what has been called “model-based neural networks” (MBNN) (Caelli et al., 1993) or “ explanation-based neural networks” (Mitchell, 1997). In MBNNs, input–output relations, connections and weights, are all subject to modelling via constraints which control the search procedures where applicable, or simply guide the information flow through the network in such a way to guarantee given solutions. Further, MBNNs typically consider the type of network “algebra” which is necessary for the achievement of specific goals: the way in which specific subnets can be combined or concatenated to map inputs to known output goals. Central to such developments is the consideration of how specific subnets can be modelled and how they can be concatenated to produce solutions to more complex problems. Such subnetwork aglebras play an important role in the current work.

In classical numerical analysis solutions to problems are either obtained via approximation theory, linear programming or search methods, the former being analytic, the latter being based upon enumeration techniques which satisfy specific cost functions (Press et al., 1988). Our proposed approach combines both aspects of numerical computation but also restricts itself to perceptron-like computational units, in the spirit of “neural computing”. However, of central importance to this work is the notion that parameterizing the connections, weights, etc., constrains the associated search problem and so results in solutions which are guaranteed to satisfy the problem constraints and domain knowledge.

In more standard applications of neural networks the input is typically numerical data and the weights or parameters to be estimated typically correspond to variable states. In the investigations to be reported here the reverse is sometimes more appropriate: inputs correspond to model variable states and the estimation problem is concerned with finding data to produce an output function value (e.g. the output is zero). In this way, we note some duality between data and model parameters: problems can be formulated in terms of either estimating parameters to fit data or estimating data to fit parameter states. The choice between these two alternatives can vary with respect to the actual problem to be computed within a given subnetwork.

To this stage we (Caelli et al., 1993; Squire and Caelli, 1997) have shown how by creating specific types of connections and parameterizations of the weighting functions we can obtain invariant pattern recognition with much fewer variables than traditional ANNs. In this paper we show how general this idea is in investigating how quite different mathematical computations can be solved within standard perceptron-like neural architectures. In particular, we shall consider how to implement them in MBNNs:

  • 1.

    Taylor expansion for functional approximation.

  • 2.

    Differentiation and integration of functions.

  • 3.

    Solution of algebraic equations.

  • 4.

    Calculus of eigenvalues of matrices.

  • 5.

    Metrics in the sense of Minkowski and Riemann.

  • 6.

    Surface representations.

  • 7.

    Computation of first and second fundamental forms of surfaces.

  • 8.

    Computation of Principal curvatures and of Gaussian and Mean curvatures.

Section snippets

Some basic computations

Our perspective to neural computing involves determining the ways in which the subnets are modelled and combined where, in particular, different representations entail different net architectures. The subnetwork models define individual or “local” network behaviour while the concatenation rules and, where appropriate, cost functions, determine the “global” output of the system.

For instance, consider two vectors x, y, and consider the problem of designing a neural net with non-linearities,

Polynomials, functions and the calculus

In this section we show how MBNNs can be used to explicitly represent polynomials and more general functions, and to perform operations such as differentiation and integration using algebraic and finite difference methods. It should be noted, at this stage, that our approach here is not concerned with well-known results that any function can be approximated by a neural network. Rather, in MBNN we are concerned with the formal and explicit algebraic definitions and derivations within neural

Solution of algebraic equations

The problem of determining roots of polynominals has many different approaches varying from direct algebraic solutions through to search techniques. In this section we propose to formulate the problem in terms of MBNN and show how a recurrent architecture can actually implement solutions. Consider a polynominal P(x,n) and assume that the coefficient ai are such that there exist n real roots of the polynominal, that is, n solutions to the equation P(x,n)=0, where these roots need not be distinct.

Minkowski and Riemann metrics

To this stage we have considered how to implement aglebraic representations and solve algebraic problems explicitly using neural network architectures.

In this section we apply these formulations to some geometric problems to open up questions related to the use of a MBNN as a “geometric engine” for both solid and differential geometry. First, let us consider the computation of distances.

Let x, y be two n-dimensional vectors, their distance is given, according to the Minkowski metric lp, byd(x

Curvature of surfaces

The Fundamental Theorem of Surface Theory states that a surface is uniquely determined within a rigid motion by the coefficients gij and bij of the first and second fundamental forms, provided that they satisfy certain compatibility conditions (Stoker, 1969; do Carmo, 1976). As we have shown in the previous section, the first fundamental form is computable in an explicit way using the MBNN approach, in this section we shall show that the coefficients of the second fundamental form can also be

Conclusions

In this paper we propose a novel way of developing and interpreting neural networks. Consistent with our previous work on Model-based Neural Networks, we have extended the idea of modelling connections, developing subnet architectures and their concatenations which can probably result in required algebraic structures—and all within perceptron-like neural network architectures.

Further to this, we hopefully, have raised issues about more general application of neural computing, which, in turn,

Acknowledgements

This project was funded by a grant from the Australian Research Grants Committee.

References (12)

  • Bareiss, E.H. (1967). The numerical solution of polynomial equations and the resultant procedures. In A. Ralston & W.S....
  • T. Caeli et al.

    Model-based neural network

    Neural Networks

    (1993)
  • Conte, S., & de Boor, C. (1981). Elementary Numerical Analysis. New York:...
  • Dieudonné, J. (1960). Foundations of Modern Analysis. New York: Academic...
  • do Carmo, M. (1976). Differential Geometry of Curves and Surfaces. Englewood Cliffs, New Jersey:...
  • Korn, G., & Korn, T. (1968). Mathematical Handbook for Scientists and Engineers. New York:...
There are more references available in the full text version of this article.

Cited by (0)

View full text