Abstract
The problem of minimizing a sum of squares of nonlinear functions is studied. To solve this problem one usually takes advantage of the fact that the objective function is of this special form. Doing this gives the Gauss-Newton method or modifications thereof. To study how these specialized methods compare with general purpose nonlinear optimization routines, test problems were generated where parameters determining the local behaviour of the algorithms could be controlled. The order of 1000 test problems were generated for testing three algorithms: the Gauss-Newton method, the Levenberg-Marquardt method and a quasi-Newton method.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Y. Bard,Comparison of gradient methods for the solution of nonlinear parameter estimation problems, SIAM J. Numer. Anal. 7 (1970) 157–185.
P. E. Gill and W. Murray,Safeguarded steplength algorithms for optimization using descent methods, NPL Rep. NAC 37, Teddington, Middlesex (1974).
P. E. Gill and W. Murray, ed.,Numerical methods for constrained optimization, Academic Press, London and New York (1974).
R. J. Hanson,Certifying linear equation solvers, Signum newsletter vol. 4, No. 3 (1969) 21–29.
K. Levenberg,A method for the solution of certain nonlinear problems in least squares, Quart. Appl. Math. 2 (1944) 164–168.
D. W. Marquardt,An algorithm for least-squares estimation of nonlinear parameters, J. SIAM 11 (1963) 431–441.
J. J. McKeown,Specialised versus general-purpose algorithms for minimising functions that are sums of squared terms, Tech. Rep. No. 50, The Hatfield Polytechnic (1973).
W. Murray, ed.,Numerical methods for unconstrained optimization, Academic Press, London and New York (1972).
J. M. Ortega and W. C. Rheinboldt,Iterative solution of nonlinear equations in several variables, Academic Press, New York (1970).
Ostrowski, A. M.Solution of equations in Euclidean and Banach spaces, Academic Press, New York (1973).
M. J. D. Powell,A view of unconstrained optimization, Rep. C.S.S. 14, Harwell (1975).
M. J. D. Powell,Some global convergence properties of a variable metric algorithm for minimization without exact line searches, Rep. C.S.S. 15, Harwell (1975).
P.-Å. Wedin,The non-linear least squares problem from a numerical point of view, I Geometrical properties, Tech. Rep. Dept. of Comp. Sc., Lund University (1972).
P.-Å. Wedin,On the Gauss-Newton method for the non-linear least squares problem, Rep. 24, The Swedish Institute of Applied mathematics, Box 5073, Stockholm (1974).
Author information
Authors and Affiliations
Rights and permissions
About this article
Cite this article
Ramsin, H., Wedin, PÅ. A comparison of some algorithms for the nonlinear least squares problem. BIT 17, 72–90 (1977). https://doi.org/10.1007/BF01932400
Received:
Revised:
Issue Date:
DOI: https://doi.org/10.1007/BF01932400